Rethinking and Improving Robustness of Convolutional Neural Networks: a Shapley Value-based Approach in Frequency Domain
Authors: Yiting Chen
Qibing Ren
Junchi Yan
Venue: NeurIPS 2022 Accept (2022)
Keywords: Convolutional Neural Network
adversarial robustness
frequency domain
Shapley value

Abstract: The existence of adversarial examples poses concerns for the robustness of convolutional neural networks (CNN), for which a popular hypothesis is about the frequency bias phenomenon: CNNs rely more on high-frequency components (HFC) for classification than humans, which causes the brittleness of CNNs. However, most previous works manually select and roughly divide the image frequency spectrum and conduct qualitative analysis. In this work, we introduce Shapley value, a metric of cooperative game theory, into the frequency domain and propose to quantify the positive (negative) impact of every frequency component of data on CNNs. Based on the Shapley value, we quantify the impact in a fine-grained way and show intriguing instance disparity. Statistically, we investigate adversarial training(AT) and the adversarial attack in the frequency domain. The observations motivate us to perform an in-depth analysis and lead to multiple novel hypotheses about i) the cause of adversarial robustness of the AT model; ii) the fairness problem of AT between different classes in the same dataset; iii) the attack bias on different frequency components. Finally, we propose a Shapley-value guided data augmentation technique for improving the robustness. Experimental results on image classification benchmarks show its effectiveness.

Rethinking and Improving Robustness of Convolutional Neural Networks: a Shapley Value-based Approach in Frequency Domain
Yiting Chen, Qibing Ren, Junchi Yan'
Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence
Shanghai Jiao Tong University
{sjtucyt, renqibing, janjunchi]ssjtae edu. cn
Abstract
The existence of adversarial examples poses concerns for the robustness of convolutional neural networks (CNN), for which popular hypothesis is about the frequency bias phenomenon: CNNs rely more on high-frequency components (HFC) for classification than humans, which causes the brittleness of CNNs. However, most previous works manually select and roughly divide the image frequency spectrum and conduct qualitative analysis. In this work, we introduce Shapley value, a metric cooperative game theory, into the frequency domain and propose to quantify the positive (negative) impact of every frequency component data on CNNs. Based on the Shapley value, we quantify the impact in fine-gradd fine-grained way and show intriguing instance disparity. Statistically, we investigate adversarial training(AT) and the adversarial attack the frequency domain. The observations motivate us to perform an in-depth analysis and lead to multiple novel hypotheses about i) the cause of adversarial robustness the AT model; ii) the fairness problem of AT between different classes in the same dataset; iii) the attack bias on different frequency components. Finally, we propose Shapley-value guided data augmentation technique for improving the robustness. Experimental results on image classification benchmarks show its effectiveness. The code for this paper is at httpp:: http//gtttttttce//e htt.auu..com/ytccenn88///////
Introduction
Though convolutional neural networks (CNNs) have shown their great generalization power in various show that CNNs prone to be affected by adversarial noises that are imperceptible to humans. As inspecting the impact of frequency components, i.e. signals of a certain frequency in the frequency domain, one popular hypothesis about the brittleness of CNNs is that CNNs can exploit high frequency components (HFCs) that are perceivable humans 41 42 21 Based on this hypothesis. line of works propose utilize HFC filtering to improve the attack or defense methods [33 22 51]. However, towards the above frequency bias phenomenon, most works [4 42 21 intuitively divide the image frequency spectrum into two parts as low frequency components (LFCs) and HFCs then evaluate the contribution of each part based on the performance of models on the dataset, overlooking the possible disparity in sample-level and frequent component-level. Moreover, current works [17 32 reveal that some successful adversarial attacks also depend on LFCs, which reveals the limitation of previous works: i) overlooking the contribution of each individual frequency component in LFCs and HFCs; ii) only do qualitative dataset-level analysis.
vision tasks, the existence of adversarial examples [77 16 24
Junchi Yan is the correspondence author. who is also with Shanghai AI Laboratory.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Instead of studying the impact of HFCs and LFC's at the overall dataset level, in this paper we resort to quantify the contribution of each frequency component with the help of Shapley value [T. Specifically, Shapley value is metric from cooperative game theory which has been widely used in explainable machine learning 99 prediction in the spatial domain, our work is the first to apply Shapley value to the frequency domain. Shapley value satisfies several desirable properties for quantifying the contribution of each frequency component, and we justify that applying Shapley value in the frequency domain rather than spatial domain is a more natural approach on CNNs (c.f. Remark D
2
23 36. Since these works measure pixel-level importance for
Based on the Shapley value, we quantify the contribution of each frequency component by measuring the average expected difference between the model's s output brought by the absence of the frequency component regarding all possible circumstances where other frequency components are either masked or not. An illustration of the quantifying method is presented in Fig. [ Through our quantification, one is able to inspect the contribution of each frequency components of each sample (instead of at dataset level). For simplicity, we refer to the frequency components with positive contribution as positive frequency components (PFC) and frequency components with negative contribution as negative frequency components (NFC).
Our quantification results provide insights in various perspectives. At the instance level, we present intriguing disparity between different data samples, e.g. model may rely more on HFCs inference on some instances, as shown in the first row of Fig. [] or rely more on LFCs to inference on the other instances, as shown in the second row of Fig. 2 Statistically, we inspect and analyze the adversarial training including the cause of the adversarial robustness achieved by adversarial training and the fairness problem in adversarial training (AT), which quantitatively proves that HFCs plays an important role in adversarial training. We further inspect and analyze the adversarial attack. We present the attack bias on different frequency components and provide a possible explanation for adversarial attacks focusing LFCs 17 32).
Based on our quantification, we propose simple yet principled data augmentation method named Class-wise Shapley value- galue-guieed Augmentation (CSA) to correct the misaligmment of features between different classes. By augmenting the data sample with features composed by NFCs of other data samples from the same class, the method boosts the adversarial robustness efficiently. Our major contributions are as follows:
Instead of inspecting CNNs based on the manually divided frequency spectrum as done in existing literature [1),, we introduce the Shapley value metric to the frequency domain and directly quantify the contribution of each individual frequency component on CNNs, with the advantage of desirable theoretical properties from the Shapely value itself (Section 3 To our knowledge, this is the first work to apply the Shapley value to the frequency domain on DNNs (specifically CNNs in this paper).
As inspecting the impact of frequency components on models, we address the possible cause of adversarial robustness achieved by adversarial training and demonstrate that AT models improve robustness by sacrificing the utilization of HFessSec. (.2).. We further address the faimess problem in adversarial training and demonstrate the negative relationship between the impact HFCs have on the ST model and the robust accuracy of the AT model(Sec. 4.3)
We inspect the impact of different frequency components on adversarial attacks and demonstrate that adversarial attacks on PFCs and HFCs are relatively more effective than attacks on NFCs and LFCs. We further demonstrate that contribution of frequency components is not necessarily determined by its frequency and provide possible explanation for successful adversarial attacks focusing on LFCSs [1 32 (Sec.
Motivated by our explanatory results, we assume that NFCs compose features that are not correctly exploited by the model. Therefore we propose to augment data with NFCs found on data samples of its class, entitled by Class-wise Shapley value-guidee Augmentation (CSA), which successfully boosts robustness in our empirical studies (Section
2 Notations and Preliminaries
We present our model in the context of image classification, while we think the concept can still be applied to other backbones e.g. GNNs.
2

---

Spatial Domain
Frequency Domain
f(.)
Model;
f(
)-f( ,,
Unmasked Frequency;
f( )-f(
Average
Shapley Value of
:
Masked Frequency;
Target Frequency
f(
)- f(
Figure Illustration of the Shapley value calculated frequency domain when the frequency components are devided into 2 x 2 patches. The Shapley value of a frequency component is the average marginal effect that the frequency component contributes to the output of the model.
Notation: (X, denotes data sample where X E Rdixda is an image and E [1,.... C). label where is the number of classes. For simplicity, we omit the channel of images. f(,,0) 0) Rdixxd + RC denotes a neural network that takes image input and output prediction where represent its parameters. We use .(:,-) to denote loss function and 1|p to denote lp norm. F(-): Rdixxd + Xdixda denotes the Discrete Fourier Transfom (DFT). F (-) Edixda Rd x denotes the inverse Discrete Fourier Transform.
Discrete Fourier Transform (DFT), DFT transforms finite signals into complex- valued functions of frequency. The 2D-Discrete (for image data considered in this paper) Fourier Transform is given by:
da
F())(u,o)=
= Σ
X(m,
12m( 2, +5)
(1)
m=0 n=0
The inversion of DFT is computed as:
X(m,n)
F(X)(u,
22 (mm
(2)
d1d2
u=D
==u
Adversarial Attack. aims find a small noise 8 bounded in lp space to cause failure 37
16
= arg max {(f( + +),y), s.l. ||0|p S
Defense. Adversarial training becomes the most effective defense method B which typically conducted by minimizing loss directly on adversarial training data.
arg min EJ max l(f(X + 8). "1
Definition of the Shapley Value: Let V = (1. n2} denotes the finite set of players. Each
non-empty subset C N is called coalition. A cooperative game is defined by the pair (N, V). where V 2.V + R is a mapping that assigns a real number to each coalition and satisfies V(O)) 0.
Let स E II()) denotes permutation of player set V and xi)) is the position of player N, where II()) is the set of permutations. Then the predecessor set of player i E W in permutation is
P. = (j E (j) (1)). The Shapley value Φ of player is defined as:
N.V
Σ REII(N)
[V(P] U (1)) - V(PT))
(5)
IT(A
3

---

se000maii 31235567
Origin Image
Image Reconstructed Image Reconstructed by PFC
Heatmap of Shapley Value
by NFC
on Frequency domain
Shapley value based quantification of two example images in ImageNet. Left to right: origin image, image reconstructed with positive frequency components (PFC), images reconstructed with negative frequency components (NFC), the heat map of Shapley value in frequency domain. Note that the heatmap shifted, the closer to the center the lower the frequency.
Figure 2:
The Shapley value 31 of the player the average marginal contribution of the player the value of predecessor set in every possible permutation.
Properties of Shapley Value: It satisfies many desirable properties including Null player, Efficiency, Symmetry, Linearity. See Appendix D for more details about the properties of Shapley value.
Shapley Value-based Quantification
3.1 Contribution quantification for frequency component
We assume that frequency components in the data sample cooperate with each other and compose the feature that the model process to infer. To apply the Shapley value to the frequency components, firstly construct a cooperative game this scenario. Considering frequency components as players, define the set of players for data sample (X,y) as:
Nx = (F() 1), ... (7x)4411444 -.d2-1))
(6)
Note that (0,)(000))) determines the mean value of the image, exclude this frequency component from the player set and keep the mean value of the image unchanged to fairly calculate the marginal
contribution of the frequency components.
For model [()), we define the output of the model on S C as:
S(S) 1 2 F(X)))
where f(-)y denotes the y-th output of the model and denotes Hadamard product. ds [0, 1] xd2 is the mask generated according to S satisfying:
as(u, v) I[F(X( S]
where I(-) denotes an indicator that output 1 when the condition is satisfied and 0 otherwise. Specifi- cally, (0, The corresponding characteristic function of the game is defined as:
V(S)
- S
By definition, we calculate the Shapley value .f of frequency component F(X) (u, v) according to Eq.5] The Shapley value of frequency component evaluates the average marginal contribution of the frequency component the neural network's ' output of the ground-truth class, which takes desirable properties including Null player, Efficiency, Symmetry and Linearity.

---

Sumple
Adv Sumple
average Shapley value
0.08
0.08
0.06
average Shapley
0.02
0.04
0.00
0.02
-0.02
0.00
10111213141516
1
1011
frequency basis
frequency basis
(a) ST model
(b) AT model
Figure 3: Average shapley value of frequency componets in clean samples and adversarial samples over frequency basis of ResNet18 on CIFARIO.
3.2
Justification for frequency based Shapley value on CNNs
When it comes to measuring the contribution of input features in images, the common approach is decomposing features into pixels in the spatial domain and considering each pixel as player [ 23 36). Missing players that not in coalition S replaced with reference value. The reference value are often chosen intuitively as zero or the mean value the image. The choice of reference value may affect the explanatory results on CNNs.
Remark. 1 (Decomposition of convolution) The output of a convolution operation (.) on X can decomposed into a linear function of output on frequency components on X. Formally,
d1-1d2 d2-1
Jamn X
Σ
[T)) Doman
[F
di X )]]
(10)
where I xd, E RRd xd is a Notch pass filter i.e. it contains zeros except the element on the u-th row and v-th column is one. According to Remark [ the output of convolution operation linear function w.rlf each frequency component. Hence the reference value for frequency components is naturally zero.
Furthermore, the Shapley value intuitively aligns more with what is important an additive setting, mentioned in previous works [20]. For non-additive models, Shapley value may fail to explain the importance players. For example, function fx) = x1 22 with reference value set to be zero. The Shapley value of each variable is 3,2 fz) while the value of each variable may vary. For frequency components, the CNN is approximately a linear function. See Appendix D for more details.
Rethinking Robustness in Frequency Domain
In this section, we conduct extensive experiments give fundamental analysis of the contributions of different frequency components. In Sec, [.1]] we introduce details and results of instance-level sampling for Shapley value calculation. In Sec. 4.2 inspect the contribution of frequency components of clean (adversarial) data with standard (adversarial) training paradigms respectively, followed by an in-depth analysis of the way how adversarial attacks work in Sec. 4.4 the discussion of fairness in Sec. 4.3
We demonstrate
4.1 The Shapley Value of Frequency Components and The Instance Disparity
To get the Shapley value of each frequency component, employ Monte Carlo Permutation Sampling to approximate the Shapley value [II. We sample the Shapley value on CIFAR1O, CIFAR100, Tiny ImageNet and ImageNet.
Quantification result We demonstrate our quantification result on ImageNet Fig Each row contains the results of data sample. We show the original image, the reconstructed image from
5

---

(a) Class (Plane)
(b) Class (Car)
(c) Class 2 (Bird)
(d) Class (Cat)
(e) Class (Deer)
(f) Class 5 (Dog)
(g) Class 6 (Frog)
(h) Class (Horse)
(i) Class (Ship)
( Class 9 (Truck)
Figure 4: Average Shapley value over frequencies of clean samples ST model from CIFAR1O.
positive frequency components (PFCs), the reconstructed image from negative frequeney components (NFCs), and the heatmap of Shapley value from left to right.
The Shapley value of each frequency component varies between different data samples. As shown Fig-ß the NFCs the first image are mainly of high frequency or medium frequency while the PFCs are mostly low frequency. As a result, the NFCs compose the contour of the goldfish and other textures while PFCs recover the main body of the fish just like how humans perceive However, the "shark" image in the second row shows the opposite direction to "goldfish": PFCs mainly contains the contour of the shark, i.e., the HFC information, which means the frequency bias varies at instance-level. See Appendix A for more details and more results.
4.2 The Cause of the Adversarial Robustness of AT model
Observations in Sec. are based on clean examples. We further inspect the difference between ST and AT models in the frequency domain based on our Shapley value metric. For CIFAR10, adversarial examples are generated with PGD-20 with a Linf bound of 8 82255 and step size of 2,255. We show results of ResNet-18 on CIFAR10, 10. Refer to Appendix B for more results of vgg16 on CIFARI 10 and ResNet- 18 on CIFAR 100 and refer Appendix A for more details of the model training.
4.2.1 Standard Trained Models Mainly Exploit LFC but Are More Vulnerable on HFC
As shown in Fig. ((a) first, for clean examples, LFCs have significant positive Shapley value on average for standard trained (ST) model while the average Shapley value of HFCs are nearly zero, which means ST model barely exploits HFC; Second, for adversarial examples, the mean Shapley value HFC are significantly negative, which shows that ST model is more vulnerable on HFC.
4.2.2 Adversarial Trained Models Improve Robustness by Sacrificing Information in HFCs
As shown in Fig. 3(b) LFCs have high positive Shapley value on both clean and adversarial examples and the average Shapley value HPCs are nearly zero on both clean and adversarial examples, which may be the key to improved robustness of AT. We hypothesize that AT tends to filter out the negative impact HFC and the outcome is that AT focuses more on LFC, overlooking information HFC. As Fig. 3(a) shows, the HFC of clean images plays a remarkable positive impact. Thus we conjecture that the trade-off between the clean accuracy and robust accuracy AT possibly the result of whether the model utilizes the information in HFCs.
4.3 Fairness of AT in Frequency Domain
The class-wist wise disparity of robustness of AT model a troublesome phenomenon even on the balanced dataset such as CIFAR-10 10 43 38) which results in faimess concerns. Though the clean accuracy of the ST model on each classes similar, the robust accuracy of the AT model differs between classes. For 10, the class 0, 1, 8, 9 that belong to the same coarse class "Transportation" exhibit relatively high accuracy and robustness while other classes related "Animals" less robust except the class 7 "horse"".

---

To investigate the difference between different classes, we demonstrate the average Shapley value of clean data samples from each class for ST model in Fig. We generally find the trend that the ab- solute Shapley value of HFC is relatively larger on non-robust classes (e.g. class 2,3,4 and 5) than robust classes (e.g. class 1, 7, 8 and 9).
clast
Adv accuracy of AT modol(%)
60 50
clapi
Clas
We further calculate the average absolute Shapley value of HFC (we intuitively take components with frequency higher than 70% of the largest frequency) for ST model. As shown in Fig. E there is neg- ative relationship between the adversarial accuracy of AT model and the average absolute Shapley value of HPC for ST model (the Pearson correlation co- efficient is -0.8765). As shown in Sec. 4.2 HFCs compose useful but vulnerable information while AT tends to sacrifice HFCs for robustness. Therefore, we conjecture that: classes, where HFCs have a larger impact on the ST model, would have relatively lower adversarial accuracy under AT.
20 0.005 0.030 0.040 Avg. absolute Shapley value HF ST model
0,060
Figure 5: the adversarial accuracy of AT model and the average absolute Shapley value of HFC for ST model.
The negative relationship between
4.4 Inspecting Adversarial Attack in Frequency Domain
4.4.1 Adversarial Attacks on PFCs and HFCs Are More Effective
In this section, we inspect the performance of adversarial attacks over different frequency components. Specifically, for an adversarial noise 8 generated by Eq. on ST model and data sample (X. we transform the noise into frequency domain as F()). which is further applied by masks based on different metrics to determine the impact of adversarial noise on different frequency components.
We first build masks based on the division between PFCs and NFCs of the data sample and the masked noise is defined as follows, where Hadamard product:
8PPC = F F @ F())), ÓNFC= FF F(8)),
OPFC(u, v) = 1(d > 0)
(11)
ONFC(U, I(dx../ 0)
Similar to Eq. 11 we define adversarial noises 8 on HFCs and LFC
SLFC = (OLFO F(8)). =
OLFC(U, v) = I[r(u, 8) < Clow Tmax OFFC(u, = [1[r(u, v) > Ehigh Tmaa]]
(12)
O F(6)),
where r(u, v) is the frequency of the frequency component, Tmar is the maximum frequency in this data sample, flow and Ehigh the intuitively determined threshold for low frequency and high frequency. In our experiment, Gow = 0.4, Chigh = 0.6.
We test the error rate of ST model as the input data sample is attacked with various noises, as shown in Fig. 6 The error rate is nearly zero for origin data sample X and nearly one hundred percent for data samples attacked with the unmasked noises X 8. For adversarial noise PFCs and NFCs, X + FCFC reaches a similar error rate of ST model as X + which is significantly larger than the rate with X FC. It indicates that the adversarial attack PFCs is more effective. Similar for HFCs and LFCs, the adversarial attack on HFCs is more effective than NFCs.
4.4.2 A Possible Explanation for Adversarial Attacks Focusing on LFCs
As demonstrated in Sec. 4.2 and Sec. we quantitatively proved that models are more vulnerable
on HFCs than on LFCs. Previous works hypothesize that exploiting HFCs causes the brittleness of However, it conflicts with the success of the adversarial attacks that are limited
CNNs [4 40
45
on LFCs 17
32
addressed in 4 25 and demands closer look.
To explain such an intriguing conflict, we calculate the ratio of PFCs and NFCs for each frequency over the spectrum. As shown in Fig, both PFCs and NFCs that have significant impact take up
7

---

100
ratio of frequency companent
0.4
PFC
NFC
0.3
error rate
0.2
0.1
10111213141516
X X+5
of model
X-Bax
X46ec
X+8uc
frequency basis
Figure 6: The enror rate of ST model on CI- FARIO with different input. X represent ori- gin data, the adversarial noise generated against the model. SpFC, FNFC, FCFC and ÔLFC are the masked adversarial noise that are constrained PFC, NFC, HFC and LFC.
Figure 7: Ratio of the component PFC and NFC with significantly large Shapley value (ab- solute value greater than 20% the maximum absolute Shapley value of the same frequency). Results are averaged over clean examples on ST model (ResNet18) CIFARIO.
fair ratio (at least 0.1) over the whole frequency spectrum. Therefore, we argue that the contribution frequency components on the prediction of CNNs are not necessarily related to frequency.
The fact that PFCs and NFCs actually co-exist on every frequency basis provides a possible explana- tion for the success of the adversarial attacks that are limited LFCs [17 [2]. Since the ratio of PFCs and NFCs are even higher at low frequency, is possible for adversarial attacks that focus on LFCS reduce the impact of PFCs and boost the impact of NFCs.
5 Improving Robustness via Class-wise Shapley Value-guided Augmentation
Motivation. As defined in Sec. frequency components with negative Shapley value make the model output lean to other classes rather than the ground-truth class, which implies that the distribution of image features composed by NFC is mistakenly aligned with that of other classes. Intuitively, by proactively correcting the misalignment of features of the model, we could improve the adversarial robustness for future attacks. Moreover, disparity at the class level distribution of NFC. Therefore, we propose to actively augment data with features composed by NFC of the same class, named Class-wise Shapley value-guided Augmentation (CSA). To be specific, based CNN follows, with the indicator function T[-]:
observations in Sec. 4.3 reveal that there exists obvious
obtain the CSA p of data X by masking its PFC as
x (u, v) X <0 F(X)(u,)))
(13)
Implementation
Since it takes a large number of samplings for obtaining a stable Shapley value (c.f. Eq. 5) for each sample, it is necessary to downgrade the time complexity. Fortunately, we find that for
Table Test accuracy of ResNet- trained on CIFARIO and CIFAR100 with CSA. 'Fina'' denotes
the performance the last training epoch. 'Best' refers the best snapshot model in training.
Natural Accuracy w/o attack
attacked by PGD-20
attacked by AutoAttack
CIFARIO
Best
Final
Diff
Best
Final
Diff
Best
Final
Diff
buscline
83.83%
84,49%
-0.66
49.87%
46.38%
3.49
46.52%
44.06%
2.46
PGD-AT
CSA
81,57 %
82.91 %
-1.34
51.13%
49.42%
1.71
47.17%
46.56%
0.61
baseline
81.23% 80.84%
81.71%
-0.48
51.18%
51.08%
0.10
47.94%
47.74%
0.20
TRADES
CSA
81.62%
-0.78
52.57 %
52.06%
0.51
48.94%
49.16%
-0.22
CIFAR100
Best
Final
Diff
Best
Final
Dift
Best
Final
Diff
baseline
56.55%
57.13%
40.58
24.38%
21.70%
2.68
21.66%
10%
1.56
PGD-AT
CSA
56.31 %
56.72 %
-0.41
26.14%
23.63%
2.51
23.02% %
21.64%
1.38
baselinc
53.09% 54.88%
53.09%
0.00
27.01%
27.01%
0.00
22.76%
22.76%
0.00
TRADES
CSA
54.46%
0.42
27.81%
27.77%
0.04
23.61 %
23.73%
-0.12

---

Table 2: Test accuracy of VGG16 and WRN-28-10 trained on CIFARIO with CSA. 'Final' denotes
the performance the last training epoch. 'Best' refers to the best snapshot model in training.
Natural Accuracy w/o attack
attacked PGD-20
attacked AutoAttack
WRN-28-10
Best
Final
Diff
Best
Final
Diff
Best
Final
Diff
bascline
85,90%
85.67%
0.23
50.18%
47.51%
2.67
47.86%
45.69%
2.17
PGD-AT
CSA
85.48 %
85.87 %
-0.39
53.00%
50.83%
3.50
49.61% %
48.75%
0.86
baseline
84.13%
84.73%
-0.60
53.17%
48.72%
4.45
50.21%
46.65%
3.56
TRADES 49
CSA
84.41 %
84.92%
-0.51
54.32%
50.53%
3.79
51.40%
48.65%
2.75
VGG16
Best
Final
Diff
Best
Final
Diff'
Best
Final
Diff
baseline
77.90%
80.06 %
-2.16
48.03%
46.96%
1.07
43.68%
42.70%
0.98
PGD-AT 2
CSA
75.42 %
75.40 %
-0.02
48.22%
48.06%
0.16
43.68%
43.53%
0.15
baseline
79.86%
79.69%
0.17
46.92%
46.63%
0.29
42.50%
42.67%
-0.17
TRADES 49
CSA
80.09%
80.23%
-0.14
48.58
48.23%
0.35
44.13%
43.97%
0.16
Table 3: Testing accuracy of ResNet18 trained on CIFARIO with high frequency suppressing methods. 'baseline' denotes the original TRADES method, 'sup HFC' denotes combining the suppressing high frequency method with TRADES and 'CSA' denotes combining our CSA with TRADES.
Methods
Natural
Accuracy w/o attack
attacked by PGD-20
attacked by AutoAtaak
Best
Final
Diff
Best
Final
Diff
Best
Final
Diff
baseline sup HFC
81.23%
81.71% %
-0.48
51.18%
51.08%
0.10
47.94%
47.74%
0.20
TRADES 4
78. 14%
80.26%
-2.12
50.66%
50.42%
0.24
45.47%
45.14%
0.33
CSA
80.84%
81.62%
-0.78
52.57% %
52.06%
0.51
48.94%
49.16%
-0.22
the same class, the feature distribution composed by NFC aligns well between samples, and depending on only a small ratio of whole data (3% in our experiment) brings out an obvious improvement. Detailed analyses at a statistical level and experimental level can be found in Appendix B
For each class e, randomly take data samples (X1,c), (),,)),.. (Xn: e) and generate set of CSAs with a pretrained model f as: pf- fpxx PX, .xx.) } and train our model follows:
0 = arg min E max e (FIX +ax))))xpp)))) + +0%,u) o p/j
(14)
[<<|e|l
where the a is a coefficient of the CSA pf.
Results We conduct experiments on CIFARIO 10 and CIFAR 100 19 to verify the effectiveness of our CSA. We train R [8] on CIFARIO and CIFAR 100 under PGD-AT 24 and TRADES [49]. Attacks are crafted within Lo bound with 8/2 8/255. We select PGD-20 20 attack 124] and AutoAttack (AA) [12 as the attack baselines. As Table D shows, under various attacks, our CSA consistently achieves better robustness over defense baselines with little cost of natural accuracy. With the Shapley value calculated with ResNet-18, 18, we further apply our CSA on VGG16 333 and WideRes 10 [46] on CIFARIO. As shown in Table 2 our CSA also greatly improves the adversarial robustness of VGG16 (3 and WRN-28- 10 [46). It indicates that our CSA method is model-agnostic and could be applied to various model architectures. Note that the CSA set of each class contains the NFC of the first 150 images without shuffling for CIFARIO and 30 images for CIFARI00, i.e,, 3% of the training set in CIFARIO and 6% of the training set in CIFAR 100, which shows that obtaining CSA only takes little overhead and the feasibility of our method in realistic settings. Training details and ablation study can be found in Appendix C
Comparison between CSA and high frequency filtering defense method High frequency filtering methods have been proposed to defend adversarial attacks [33 22 5T]. Most high frequency filtering methods are proposed to deal with l2 norm adversarial attack and are compared with other non- adversarial training defend methods. We take the suppressing high frequency method proposed in [5T] as baseline, which masks the high frequency components and trains the model with TRADES, and test the robust accuracy of ResNet18 CIFARIO with lx attack where 8/255.
As demonstrated in Tab. for loc. attack, suppressing high frequency components even slightly degrade the testing accuracy CIFAR10.

---

6 Related Work
Adversarial Attack Since the seminal works [37 16] have been proposed that reveal the vulnerability of DNNs, there is an increasing interest in studying stronger adversarial attacks, such as FGSM [16], C&W attack [ PGD attack (24), and AutoAttack [[]]. The robustness of CNNs in this work has been thoroughly evaluated on these above attack baselines.
Adversarial Defense The development between attack and defense is just like an arms race: the existence of adversarial examples promotes the development of defense methods, among which adversarial training (AT) has been considered the most effective one [3] However, AT has its own
limitations: i) the trade-off between accuracy and robustness [3 39
48 44 ii) robust overfitting [30
I0], which brings a large robustness generalization gap. Many works have been proposed to improve
AT by sample-wist wise importance re-weighting 50 adding more unlabeled data (27 6 or regularizing
the embedding space [29]. A recent work (28 proposes to augment data with the approximated confounders from causal perspective for improved robustness while these confounders are class- agnostic and lack theoretical guarantee. In this work, we quantify the frequency importance via the theoretically stable Shapley value and design a class-specific data augmentation strategy, which achieves great improvement in robustness with better robustness generalization.
Understanding Robustness in Frequency Domain One way to inspect the intriguing nature of adversarial attack and defense is through the lens of frequency spectrum. Many works explore the sensitivity of CNNs and frequency properties of the model 40 45 I and study whether convolution
itself has an intrinsic bias in frequency domain [7]. A hypothesis is that CNNs exploit HFCs which leads to the lack of robustness [4.. Driven by this hypothesis, many pre pee-processing defend methods
have been proposed [3
22 51. Meanwhile, some works also come up with adversarial attacks
focusing on LFC 17
which leads to assumptions that constituent frequencies of adversarial
examples are dependent on the dataset [4, 25 Efforts to quantify and inspect the contribution of each
frequency component have been made 42 21 We employ Shapley value, tool that has preferable
properties, to quantify the contribution of each frequency component on the fine-grained level.
Explainability with Shapley Value Shapley Value [31] proposed to distribute contributions among players in a game. It has been widely applied for the explainability of deep learning. Some works propose its approximations 23 Shapley value [3). Also, the interaction between input features [36 35 47 is recently studied. In our work, we apply Shapley value in the frequency domain to quantify the contribution of frequency components. The quantification have many desirable theoretical properties.
While another line of works focus on the condition to apply
7 Conclusion and Outlook
We have made efforts to quantify the contribution of each frequency component on CNNs. We employ Shapley value, a method with theoretically desirable properties, to quantify the contribution of a single frequency component on the instance-level. Through our quantification, we explore the difference between standard trained (ST) model and adversarial trained (AT) model. Our analysis of the fairness in AT provides insights of the cause of robustness difference over classes. We further examine the attack effect of adversarial noises on different frequency components. Based on our findings, we then propose simple yet effective class-wise data augmentation method CSA which augments data with the NFC found in this class. Experimental results have shown its effectiveness.
Limitation & future work: Our current results are limited mainly to CNNs while we believe it is also rewarding to develop new techniques to other backbone e.g. Transformer and GNNs based on our framework, which we leave for immediate future work. We hope that this work could inspire future explanations and defense algorithms of adversarial attacks. Due to the nature of this work, there may not be any potential negative social impact that is easily predictable
Acknowledgement
This work was partly supported by National Key Research and Development Program of China A2100A,0 National Natural Science Foundation of China (61972250, 1270033, and Shanghai Municipal Science and Technology (Major) Project SHZDX0000,,,,, 2251 1 1001000).
10

---

References
[1] A. A. Abello, R. Hirata, and Z. Wang. Dissecting the high-frequency bias in convolutional neural networks. In CVPR, 2021.
[2] M. Ancona, C. Öztireli, and M. H. Gross. Explaining deep neural networks with a polynomial time algorithm for shapley value approximation. In ICML, volume 97 of Proceedings of Machine Learning Research, pages 272-281. PMLR, 2019.
[3] A. Athalye, N. Carlini, and D. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning. PMLR, 2018.
[4] R. Bernhard, P. Moëllic, M. Mermillod, Y. Bourrier, R. Cohendet, M. Solinas, and M. Reyboz. Impact of spatial frequeney based constraints on adversarial robustness. In IJCNN, pages 1-8. IEEE, 2021.
[5] N. Carlini and D. A. Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pages 39-57. IEEE Computer Society, 2017.
[6] Y. Carmon, A. Raghunathan, L. Schmidt, J. C. Duchi, and P. S. Liang. Unlabeled data improves adversarial robustness. In NewrIPS, volume 32, 2019.
[7] J. O. Caro, Y. Ju, R. Pyle, S. Dey, W. Brendel, F. Anselmi, and A. Patel. Local convolutions cause an implicit bias towards high frequency adversarial examples, 2020.
[8] J. Castro, D. Gómez, and J. Tejada. Polynomial calculation of the shapley value based on sampling. Comput. Oper. Res., 36(5):1726-1730, 2009.
[9] J. Chen, L. Song. J. Wainwright, and M. I. Jordan. L-shapley and c-shapley: Efficient model interpretation for structured data. ICLR (Poster). OpenReview.net, 2019.
[10] T. Chen, Z. Zhang, S. Liu, S. Chang, and Z. Wang. Robust overfitting may be mitigated by properly learned smoothening. In ICLR, 2020.
[11] F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein. Robustbeneh: a standardized adversarial robustness benchmark. In NeurIPS Datasets and Benchmarks, 2021.
[12] Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020.
[13] N. Das, M. Shanbhogue, S. Chen, F. Hohman, S. Li, L. Chen, M. E. Kounavis, and D. H. Chau. SHIELD: fast, practical defense and vaccination for deep learning using JPEG compression. In KDD, pages 196-204, ACM, 2018.
[14] C. Frye, D. de Mijolla, T. Begley, L. Cowton, M. Stanley, and I. Feige. Shapley explainability on the data manifold. In ICLR. OpenReview.net, 2021.
[15] C. Frye, C. Rowat, and I. Feige. Asymmetric shapley values: incorporating causal knowledge into model-agnostic explainability. In NeurlPS, 2020.
[16] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In ICLR (Poster), 2015.
[17] C. Guo, S. Frank, and K. Q. Weinberger. Low frequency adversarial perturbation. In UAI, volume 115 of Proceedings of Machine Learning Research, pages 1127-1137. AUAI Press, 2019.
[18] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, IEEE Computer Society, 2016.
[19] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.

---

[20] I. E. Kumar, S. Venkatasubramanian, C. Scheidegger. and S. A. Friedler. Problems with shapley- value-based explanations as feature importance measures. ICML, volume 119 of Proceedings of Machine Learning Research, pages 5491-5500. PMLR, 2020.
[21] Z. Lin, Y. Gao, and J. Sang. Investigating and explaining the frequency bias in image classifica- tion. In IJCAI, 2022.
[22] Z. Liu, Q. Liu, T. Liu, N. Xu, X. Lin, Y. Wang, and W. Wen. Feature distillation: Dnn-oriented JPEG compression against adversarial examples. In CVPR, pages 860-868 Computer Vision Foundation / IEEE, 2019.
[23] S. M. Lundberg and S. Lee. A unified approach to interpreting model predictions. In NIPS, pages 4775-4774, 2017.
[24] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR (Poster). OpenReview.net, 2018.
[25] S. R. Maiya, M. Ehrlich, V. Agarwal, S. Lim, T. Goldstein, and A. Shrivastava. A frequency perspective of adversarial robustness. CoRR, abs/21 0tts/2111.0086.. 2021.
[26] S. Malcki, L. Tran-Thanh G. Hines, T. Rahwan, and A. Rogers. Bounding the estimation error of sampling- bampling-based shapley value approximation with/without stratifying. CoRR, abs/1306.4265, 2013.
[27] A. Najafi, S.-i. Maeda, M. Koyama, and T. Miyato. Robustness to adversarial perturbations in learning from incomplete data. In NeurIPS, volume 32, 2019.
[28] Y. M. Q. W. Qibing Ren, Yiting Chen and J. Yan. Dice: aomain-attaci invariant causal learning for improved data privacy protection and adversarial robustness. In KDD, 2022.
[29] Q. Ren, Q. Bao, R. Wang, and J. Yan. Appearance and structure aware robust deep visual graph matching: Attack, defense and beyond. In CVPR, 2022.
[30] L. Rice, E. Wong, and Z. Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
[31] L. S. Shapley. A value for n-person games. In Contributions to the Theory of Games, page 307-317, 1953.
[32] Y. Sharma, G. W. Ding. and M. Brubaker. On the effectiveness of low frequency perturbations. In IJCAI, pages 3389-3396. ijeai.org, 2019.
[33] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[34] D. Su, H. Zhang, H. Chen, J. P.-Y. Chen, and Y. Gao. Is robustness the cost of accuracy? ?-a comprehensive study on the robustness of 18 deep image classification models. In ECCV, 2018.
[35] M. Sundararajan, K. Dhamdhere, and A. Agarwal. The shapley taylor interaction index. In ICML, volume 1 119 of Proceedings of Machine Learning Research, pages 9259-9268. PMLR, 2020.
[36] M. Sundararajan and A. Najmi. The many shapley values for model explanation. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 9269-9278. PMLR, 2020.
[37] C. Szegedy, W. Zaremba, I. Sutskever, Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In ICLR (Poster), 2014.
[38] Q. Tian, K. Kuang, K. Jiang, F. Wu, and Y. Wang. Analysis and applications of class-wise robustness in adversarial training. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1561-1570, 2021.
[39] D. Tsipras, S. Santurkar, I.. Engstrom, A. Turner, and A. Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv.1805.121533 2018.
12

---

[40] Y. Tsuzuku and I. Sato. On the structural sensitivity of deep convolutional networks to the directions of fourier basis functions. In CVPR, pages 51-60. Computer Vision Foundation / IEFF, 2019.
[41] H. Wang, X. Wu, Z. Huang, and E. P. Xing. freh-frrquency component helps explain the generalization of convolutional neural networks. In CVPR, pages 8681-8691. Computer Vision Foundation IEBE, 2020.
[42] Z. Wang, Y. Yang, A. Shrivastava, V. Rawal, and Ding. Towards frequency-based explanation for robust CNN. CoRR, abs/2005.03141, 2020.
[43] H. Xu, X. Liu, Y. Li, A. Jain, and J. Tang. To be robust to be fair: Towards fairness in adversarial training, In International Conference on Machine Learning, pages 1 11492-11501. PMLR, 2021.
[44] Y.-Y. Yang, C. Rashtchian, H. Zhang, R. R. Salakhutdinov, and K. Chaudhuri. A closer look at accuracy vs. robustness. In NeurIPS, 2020.
[45] D. Yin, G. Lopes, Shlens, D. Cubuk, and J. Gilmer. fourier perspective on model robustness in computer vision. In NeurlPS, 2019.
[46] S. Zagoruyko and N. Komodakis. Wide residual networks. In BMVC. BMVA Press, 2016.
[47] H. Zhang. Y. Xie, Zheng, D. Zhang, and Q. Zhang. Interpreting multivariate shapley interactions in dnns. In AAAI, pages 10877-10886. AAAI Press, 2021.
[48] H. Zhang, Y. Yu, J. Jiao, E. Xing, L. El Ghaoui, and M. Jordan. Theoretically principled trade-off between robustness and accuracy. In JCML, 2019.
[49] H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. El Ghaoui, and M. I. Jordan. Theoretically principled trade-off between robustness and accuracy. ICML, volume 97 of Proceedings of Machine Learning Research, pages 7472-7482. PMLR, 2019.
[50] J. Zhang. J. Zhu, G. Niu, B. Han, M. Sugiyama, and M. Kankanhalli, Geometry aware instance- reweighted adversarial training. In ICLR, 2021.
[51] Z. Zhang, C. Jung, and X. Liang. Adversarial defense by suppressing high-frequency compo- nents. CoRR, abss/1908.0555 2019.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes]
13

---

(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (e.g., code, data, models) curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'le 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "pl446hfOgky", "reviewer": "Reviewer_tXAo", "summary": "This paper presents a method for using Shapley values to estimate the importance of different frequency components of input images for the predictions of a CNN. In addition they propose a new method to train more robust classifiers by reducing the effect of frequency components that usually reduce the classification chance of target class.", "questions": "Would like clarification on the confusing sentences disucssed in weaknesses, as well as any arguments for why claims I had issue with are indeed well supported by data.", "limitations": "Only one sentence discussing limitations and no discussion regarding potential negative social impact. Would like to see at least some effort made to discuss potential impact, or at least acknowledge the fact that due to the nature of the work there may not be any that are easily predictable.\n\nIn addition I had some issues with the Checklist which was filled incorrectly. Specifically, authors state they have included code and data but this is not the case. In additions authors claim to have reported compute times hardware used which is also false. They also included a theoretical result but marked the corresponding sections in the checklist as N/A.\n\n\n### Edit:\nThank you for the detailed response. I think the updated submission is noticeable stronger, and has largely addressed my concerns regarding soundness of the results, however I still have some concerns regarding the contributions and clarity of this submission. In response I have updated my review as follows:\n\nSoundness: 2->3, Presentation 1->2, Rating 3->4.\nIn addition I have updated my confidence from 4->3 as my remaining concerns are more subjective and I feel less confident about them.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Relatively original combination of feature importance tools such as Shapley values and the analysis of adversarial examples in the frequency domain.\n- Some interesting analysis, such as the average Shapley values of different frequencies in Figure 3, and analysis of accuracy in different classes in section 4.5.\n- Related work and background explained quite clearly.\n\nWeaknesses:\n\nMost claims not sufficiently backed by evidence:\n- Most claims are based on one experiment with one network and with margins that do not look significant to me. Some examples:\nLine 186-188: It seems like this should be the case based on definitions alone but experimental data does not support it very well, with very small difference between PFC and NFC.  This to me suggests there may be issues with the methodology. \n- Fig 4. also forms the basis for one of the main claims of the paper claiming that positive parts of HFC and negative parts of LFC are most easily attacked which I don't think is displayed in the figure at all.\n- Analysis of Figure 7 not well supported by data. While there may be some trend towards \"classes that rely on HFC would have relatively low accuracy and robustness under AT\", it is not clear if this is the case and I would like to see a more quantitave measure or a simpler graph focused on this.\n- Experimental results of Table 1 are not very convincing. Authors get a slightly more robust model than the baseline they have chosen at the cost of clean accuracy. Similar results could probably be achieved by changing some parameters of the baseline method. In addition performance is far below published state-of-the-art on RobustBench, even for methods using same ResNet-18 backbone.\n\nClarity:\n- Overall the papers was very hard to follow. It has many typos and weird word choices especially in sections describing main findings of the paper which made it hard to understand what these findings are claimed to be. Examples:\n- Line 56-57: “we find that ST models merely exploit HFC but are more vulnerable on HFC under adversarial attacks”, what does this sentence mean? The phrase “merely exploits” is used many times in the paper such as the title of section 4.2 but it is unclear to me what the authors mean by this.\n- Line 169-171: “Thus we assume that the tension between accuracy and robustness possibly accounts for the overlooking of AT on HFC, especially positive ingredients.” What? Extremely confusing sentence.\n- Line 181: We future split → we further split\n- Line 208: “NFC take up a fairly certain ratio“, what does certain mean in this context?\n- Figure 1 is hard to understand.\n- Figure 7 should have same x-axis for all subfigures\n\nContributions:\nI don't think the papers contributions are very significant. As explained above robust training results don't seem too useful, and while the paper claims to extend previous work based on dataset level statistics of different features into instance based analysis, I only found the dataset level analysis discussed in the paper useful. The main takeaway that I think is adequately supported by this paper is that $l_{\\infty}$-norm adversarial attacks target high frequency features but this was already known from previous work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "j-f7VZTrfZ9", "reviewer": "Reviewer_mEHT", "summary": "In this paper, the authors mainly focus on the problem of explaining the relationship of CNN's robustness and frequency bias by proposing a new method utilizing the Shapley-value as the major tool. They assign the Shapley value to the frequency domain components and futher examine its relationship with HFC and LFC. The main contributions are:\n1. Propose a new method to explain the robustness--frequency bias relationship;\n2. Conduct instance-level, class-level and dataset-level test on the frequency-component explanation experiment, reveal the relationship between HFC/LFC and PFC/NFC;\n3. Propose a data augmentation method based on the explanation to improve the model robustness.\n", "questions": "Please refer to the previous section.", "limitations": "The authors conclude the only limitation as limited to CNNs. This does not look like a reasonable limitation, since the main topic is discussing the frequency bias of CNN. CNN is more likely to suffer from such biases becuase of its locality. Moreover, the limitations that I can think of are:\n1. Lack of motivatiuon of using such a framework;\n2. Lack of theoretical analysis on robustness-PFC/HFC. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The main strengths are:\n1. The problem is of broad interest and the angle is somehow novel, since the are few previous works focusing on instance-level explanation for CNN frequency bias;\n2. The overall method is reasonable. Assigning Shapley value to the frequency domain components is also novel;\n3. According to the results reported in the paper, the augmentation method also succeed in improving the model robustness.\n\nThe main weaknesses are:\n1. The motivation of using Shapley value is not clear. In fact, analyzing the importance of each indivisual frequency components has been discussed before[1, 2]. The most intuitive explanation method would be simply replace the target frequency component with zero or mean value, and the results reported in the papers also align with each other. This also lead to another problem that this paper lacks baseline method to compare with, considering this is not a novel task;\n2. Some explanation does not sound reasonable. For example, in section 4.5 the authors try to explain class-wise fairness with the Shapley values. However, the authors state that class 0 is both robust HFC-negative because the label of which is \"transportation\". This obviously make the previous robustness-HFC explanation less strong;\n3. Lack of more experimental results. The authors argue that the HFC/LFC dependency is inconsistent across instances within the same dataset by giving the Shapley heatmap of a shark and a gold fish. It would be great if the authors could show more results, and also avergae results across the class/dataset to better illustrate, considering the \"shark\" image is kind of blurred and even hard to tell by humans;\n4. Minor issues. For Figure 3, please double check the x-axis. For line 168, there is a typo with \"LHC\". For line 248, did you mean four classes? For Figure4, what do you mean by model output?\n\n\n[1] Visualizing and Understanding Convolutional Networks, Zeiler et al;\n\n[2] Towards Frequency-Based Explanation for Robust CNN, Wang er al.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "NN-gSVh8mrF", "reviewer": "Reviewer_w1Z4", "summary": "This work provides an interesting perspective to the robustness of CNNs in frequency domain with the help of Shapley value via a theoretically stable quantification of the contribution of frequency components. \n\nSpecifically, the authors bring a new hypothesis about the trade-off between accuracy and robustness on adversarial training and an in-depth analysis of attack bias.\n\nThe authors also investigate the fairness problem in adversarial training and propose to boost robustness by proactively augmenting inputs with negative frequency components of their class. Experimental results verify the effectiveness of their method.", "questions": "Questions:\n\n1. [Major concern] On line 34, the authors talk about a series of works improve robustness by filtering out HFC, while these baselines are missing in their experiment section.\n\n2. [Minor concern] On line 261-265, the authors claim that though calculating Shapley value requires large sampling steps, their strategy only depends on a small ratio of data. It may be better to provide a detailed time complexity of this overhead. \n", "limitations": "Yes, the authors addressed the limitations.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n+ This work performs a novel fine-grained analysis of the contribution of every frequency component in an instance-level with Shapley value, while previous works only do the coarse division of frequency spectrum at dataset-level. \n+ Based on the quantization of Shapley value, the authors propose novel and interesting explanations about the existence of adversarial examples and the success and limitations of adversarial training on CNNs. \n+ This work also delves into the class-wise robustness and proposes novel explanations, which may shed light on mitigating the fairness problem on adversarial training. Their proposed data augmentation defense strategy gets evaluated under strong attack baselines that verify its effectiveness. \n\nCons:\n- Lack of comparison with baselines which imporve robustness by filtering out HFC.\n- Lack of time complexity analysis. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "C9jroly0dJc", "reviewer": "Reviewer_em14", "summary": "This work study the adversarial robustness of convolutional neural networks (CNNs) based on the popular hypothesis about the frequency bias phenomenon in CNNs. By introducing a Shapley value, they quantify the impacts of every frequency component at an instance-level. They also propose a Shapley-value guided method for adversarial training.", "questions": "Please address questions in Weaknesses.", "limitations": "No potential negative societal impact observed.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Using the Shapley value to analysis frequency components in each instance is an interesting idea. Their experiments also demonstrate that this metric is meaningful to improve the adversarial robustness of CNNs.\n2. Their observations in Section 4 lead to instructive and constructive arguments on adversarial robustness. For example, their explanations about standard trained models (4.2) and adversarial trained models (4.3) are inspiring.\n\nWeaknesses:\n1. The experiment in Table 1 only use a small ResNet-18 model. Experiments on larger models, e.g., WideResNet-32, can be more convincing than this one.\n2. In Figure 7, only class 3 shows large negative values on high-frequency components, while others only show small negative values or all positive values. Is there any intuitive or theoretical explanation for this?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept