Smooth Kernels Improve Adversarial Robustness and Perceptually-Aligned Gradients
Authors: Haohan Wang
Xindi Wu
Songwei Ge
Zachary C. Lipton
Eric P. Xing
Venue:  (2020)
Keywords: adversarial robustness
computer vision
smoothness regularization

Abstract: Recent research has shown that CNNs are often overly sensitive to high-frequency textural patterns. Inspired by the intuition that humans are more sensitive to the lower-frequency (larger-scale) patterns we design a regularization scheme that penalizes large differences between adjacent components within each convolutional kernel. We apply our regularization onto several popular training methods, demonstrating that the models with the proposed smooth kernels enjoy improved adversarial robustness. Further, building on recent work establishing connections between adversarial robustness and interpretability, we show that our method appears to give more perceptually-aligned gradients. 

Under review as a conference paper at ICLR 2020
SMOOTH KERNELS IMPROVE ADVERSARIAL ROBUST- NESS AND PERCEPTUALLY-ALIGNE GRADIENTS
Anonymous authors Paper under double-blind review
ABSTRACT
Recent research has shown that CNNs are often overly sensitive to high-fregue sigh-frequeccy textural patterns. Inspired by the intuition that humans tend to be more sensitive to lower-frequency (larger-scale) patterns, we design regularization scheme that penalizes large differences between adjacent components within each convolu- tional kernel. We apply our regularization onto several popular training methods, demonstrating that the models with the proposed smooth kernels enjoy improved adversarial robustness. Further, building recent work establishing connections between adversarial robustness and interpretability, we show that our method ap- pears to give more perceptually-aliget gradients.
INTRODUCTION
In recent years, deep learning models have demonstrated remarkable capabilities for predictive mod- cling in computer vision, leading some to liken their abilities on perception tasks those of humans (e.g., Weyand et al. 2016). However, under closer inspection, the limits of such claims to the nar- row scope of i.i.d. data become clear. For example, when faced with adversarial examples (Szegedy) et al. 2013) Goodfellow et al.) (015) or even in aon-adversarial domain-agnosse agnostic cross-domain evaluations (Wang 2019 b; Carlucci et al., 2019), performance collapses, dispelling claims of human-like perceptive capabilities and calling into doubt more ambitious applications of this technology in the wild.
A long line of recent research has investigated the robustness of neural networks, including inves- tigations of the high-dimension nature of models (Fawzi et al. 2018), enlarging the gaps between decision boundaries (Zhang et al. 2019a), training the models with augmented examples through at- tack methods (Madry et al. 2018), , and even guaranteeing the robustness of models within given radii of perturbation (Wong & Kolter 2018 Cohen et al.) (011)). Compared to earlier methods, these cent works enjoy stronger robustness both as assessed via theoretical guarantees and empirically via quantitative performance against strong attacks. However, despite the success of these techniques, vulnerabilities to new varieties of attacks are frequently discovered (Zhang et al. 2019b).
In this paper, we aim to lessen the dependency of neural networks on high-frequency patterns in im- ages, regularizing CNNs to focus on the low-frequency components. Therefore, the main argument of this paper that: by regularizing the CNN to be most sensitive to the low-frequency components of an image, we can improve the robustness of models. Interestingly, this also appears to lead to more pereeptually-ales
gradients. Further, as Wang et al.) (2019c) explicitly defined the low (or freeunccy components as images reconstructed from the low (or high)-end of the image fre- quency domain (as is frequently discussed in neuroscience literature addressing human recognition of shape (Bar (204) or face (Awasthi al. 2011)), we continue with this definition and demon- strate that a smooth kernel can filter out the high-frequency components and improve the models' robustness.
We test our ideas and show the empirical improvement over popular adversarial robust methods with standard evaluations and further use model interpretation methods to understand how the models make decisions and demonstrate that the regularization helps the model to generate more prreptualll--alineen gradients.

---

Under review as a conference paper at ICLR 2020
2
RELATED WORK
Adversarial examples are samples with small perturbations applied that are imperceptible to humans but can nevertheless induce misclassification in machine learning models (Szegedy et al., 2013 )
The discovery of adversarial examples spurred a torrent of research, much it consisting of an arm race between those inventing new attack methods and others offering defenses to make classifiers bust to these sorts of attacks. We refer to survey papers such as (Akhtar & Mian 2018 Chakraborty al. 2018) and only list few most relevant works about applying regularizations the networks to improve the adversarial robustness, such as regularizations constraining the Lipschitz constant of the network (Cisse al. 2017) (Lipschitz smoothness), regularizing the scale of gradients (Ross) & Doshi- Velez, 2018 Jakubovitz & Giryes| 2018) (smooth gradients), regularizing the curvature of the loss surface Aioossavi-Dzzfoo
al.) 2019) (smooth loss curvature), and promoting the smooth- ness of the model distribution (Miyato et al., 2015). These regularizations also use the concept of "smoothness,"" " but different from ours (small differences among the adjacent weights).
Recently, adversarial training (Goodfellow et al. 2015 Madry et al. 2018) has become one of the most popular defense methods, based on the simple idea augmenting the training data with sam- ples generated through attack methods (i.e., threat models). While adversarial training excels across many evaluations, recent evidence exposes its new limitations (Zhang et al. 2019b). suggesting that adversarial robustness remains challenge.
Key differences: In this paper, we present a new technique penalizing differences among the adjacent components of convolutional kernels. Moreover, we expand upon the recent literature demonstrating connections between adversarial robustness and perceptually-alignel gradients.
3 SMOOTH KERNEL REGULARIZATION
Intuition. High-frequency components of images are those reconstructed from the high-end of the image reqquenc-doomale through inverse Fourier transform. This definition was also verified previ- ously by neuroscientists who demonstrated that humans tend to rely on the low-frequency compo- nent of images to recognize shapes (Bar) 2004) and faces (Awasthi et al. (01T). Therefore, we argue that the smooth kernel regularization is effective because it helps to produce models less sensitive to high-freee frequeryy patterns in images. We define a smooth kernel as a convolutional kernel whose weight at each position does not differ much from those of its neighbors, i.e., (Wa,) W a small number, where w denotes the convolutional kernel weight, i.j denote the indices of the convolutional kernel w, and (i.,)) j) denotes the set of the spatial neighbors of i,
We note two points that support our intuition.
1. The frequency domain of a smooth kernel has only negligible high-freyuency components. This argument can be shown with Theorem in (Platonov] 2005). Roughly, the idea is to view the weight matrix w as a function that maps the index of weights to the weights: w(ii.) + Wa.j. then a smooth kernel can be seen as a Lipschitz function with constant a. As pointed out by Platonov| (2005), >, Titchmarsh (9948) showed that when 0 < 1, in the frequency domain, the sum of all the high frequency components with a radius greater than will converge a small number, suggesting that the fighfreeuency components (when r is large) are negligible.
2. The kernel with negligible high-frequency components will weigh the high-frequency com- ponents of input images accordingly. This argument can be shown through Convolution Theorem (Bracewell] 1986), which states w 8 x = FF & F(x)), where F(.) stands for Fourier transform, @ stands for con- volution operation, and @ stands for point-wise multiplication. As the theorem states, the convolution operation of images is equivalent to the glement-wise multiplication of image frequency domain. Therefore, roughly, if w has negligible high-frequency components in the frequency domain. it will weigh the high-frequency components of x accordingly with negligible weights. Naturally, this argument only pertains single convolution, and we rely on our intuition that repeated applications of these smooth kernels across multiple convolution layers in nonlinear deep network will have some cumulative benefit.
2

---

Under review as a conference paper at ICLR 2020
Formally, calculate regularization term (o(w) as follows:
Ro(w) = = E
(Wa. wh,a)
s. (.AEN(
We also aim to improve this regularization by trying a few additional heuristics:
First, we notice that directly appending (o(w) will sometimes lead to models that achieve the small value of (w(w) by directly scaling down the every coefficient of propor- tionally, without changing the fluctuation pattern of the weights. To fix this problem, we directly subtract the scale of (i.e. Ei, w after Ro (w)).
Another heuristic to fix this same problem is to directly divide (o(w) by the scale of w. Empirically, we not observe significant differences between these two heuristics. We settle with the first heuristic because of the difficulty in calculating gradient when matrix is the denominator.
Finally, we empirically observe that the regularization above will play a significant role during the early stage of training, but may damage the overall performances later when the regularization pulls towards smoothness too much. To mitigate this problem, we use an exponential function to strengthen the effects of the regularization when the value is big and to weaken it when the value is small.
Overall, our final regularization is:
R(w) exp
Σ Σ (WiJ - WN,K) J.J h,KEN(iJ))
Σ w 2,J
In practice, the convolutional kernel is usually a 4-dimensional tensor, while our method only en- courages smoothness over the two spatial dimensions corresponding the 2D images. Thus, only regularize through these two dimensions broadcasting the operation through the channels.
Because repeated calculation of each kernel component's distance with its neighbors will double count some pairs, our implementation instead enumerates over all pairs of neighbors, counting each squared difference only once towards the total penalty.
We can directly append the regularization X/((w) ) to most loss functions, where À is a tuning hy- perparameter. In the following experiments, append AR(w) the vanilla loss function (cross- entropy loss), Trades loss (Zhang et al., 2019a), adversarial training loss (Madry et al. 2018). and variation of logit pairing loss (Kannan al. [0188), as introduced in the following paragraphs.
Adversarial training works by fitting the model using adversarial examples generated on the fly at train time by the threat model. Trades loss fits the model with clean examples while regularizing the softmax of augmented adversarial examples to be close to that produced for corresponding clean examples, a natural alternative to the model with augmented adversarial examples while regu- larizing the softmax of clean examples to be close to that of the corresponding adversarial examples, which is related to logit pairing. However, to make the comparison consistent, we use a variation of logit pairing, penalizing the KL divergence of softmax (rather than & distance over logits), following the Trades loss, which also uses KL divergence over softmax as the distance metric.
To be specific, with the standard notations such (X, Y) denoting data set, (x,y) denoting sample, the logit pairing loss is formalized as:
min (x (): + (),
where x' = arg max (x(;)))):
d(x' S)
where d(..-) and -). are distance functions, fi(::)) denotes the model f(:-) but outputs the softmax instead of prediction, +) is a cost function, 7 is tuning hyperparameter, and is the upper bound perturbation. In our following experiments, we consider d(.,)) as E norm following popular adversarial training set-up and k(,,)) KI divergence following standard Trades loss.
3

---

Under review as a conference paper at ICLR 2020
Intuitively, usage of KL divergence in logit pairing loss is argued be advantageous because Pinsker's inequality suggests that KL divergence upper-bounds the total variation (TV) distance (e.g.. Csiszar & Körner 2011), the usage of KL divergence limits the hypothesis space to the parameters that yield small TV distance over perturbations of samples, which is linked to the robustmess of estimator, topic that has been studied by the statistics community for over decades (e.g., see (Diakonikolas et al. 2019) and references within).
[..g.(Csiszaar
be seen
regularization that
4 EXPERIMENTS
To empirically validate our methods. we first consider a simple synthetic experiment to demonstrate the effectiveness of our proposed solutions. Then, with standard data sets such MNIST (LeCun 1998). FaSH (Xiao et al. 2017). CIFARIO (Krizhevsky & Hinton 2009) and Restricted ImageNet (Tsipras al. 2019). we evaluate our methods with well-established criteria, such as fo bounded accuracy. We also leverage saliency-based visualization methods to understand how the model understands each class. Most experiments are conducted with a simple basic convolutional neural network with two convolution layers and two fully connected layers, while the CIFARIO experiment is conducted with ResNet18 and Restricted ImageNet experiment is conducted with ResNet50 (more details of the models are in the Appendix). As mentioned previously, we apply the new regularization to four different losses: the vanilla loss (denoted as V), Trades loss (denoted as T) (Zhang et al.] 2019a), adversarial training (denoted as A) (Madry al. 2018), and our variation of logit pairing (denoted as L). T, A, L all adopt foo norm bounded PGD as the threat model. We use VR, TR, AR, LR to denote the methods after our regularization is plugged in. We evaluate our methods against a wide range of adversarial attack methods, including FGSM (Goodfellow et al., 2015 PGD (Madry et al. 2018), C&W (Carlini & Wagner 2017). DeepFool (both &g and txo) MMosavi-Dezzooli et al. (016)) ADef, a method that iteratively applies small deformations to the image (Alaifari et al. 2019) and Salt&Pepper, black-box method that adds noise to the image. For these attack methods, we use the default parameters in Foolbox (Rauber et al. (017), and our experiments suggest that these default parameters are effective enough in most cases. For every data set, we first tune the foo norm perturbation bound of adversarial training method and then use the same setting for Trades loss and variation of logit pairing. We tune within {0.1, 1.0, 10.0} and tune À within f0.01, 0.1, 1.0, 10.0, 100.0}.
4.1 SYNTHETIC EXPERIMENTS FOR SANITY CHECKING
We first use basic data set of four shapes to test whether our proposed method helps regularize the model behave as we desire. Each image in this data set has white background and a black foreground depicting one of the four shapes: circle, star, square, and triangle. Our goal is to train convolutional neural network to classify the images into one of these four shapes.
We compare the models trained the four basic losses V. T, A, L and these models with our regu- larization, denoted as VR, TR, AR, and LR, when À 100.0. To further test our idea, we also test the regularization with the hyperparameter set to negative value À - 100.00 to inspect the consequences when we regularize the model towards high-frequency kernels. Resulting models are denoted as VH, TH, AH. LH respectively according the basic losses.
We report our inspections in Figure D Figure TTa visualizes of the convolution kernel (due to the limitation of space, we only visualize the first four convolutional kernels); Figure[[i) visualizes the corresponding frequency domain in absolute values; Figure [c) visualizes the internal representa- tion after image depicting star is passed through the kernels.
Figure T (a) shows that our regularization guides the model towards a smooth kernel, across all the basic losses. Also, if we apply our regularization with negative parameter, then the weights of the resulting kernel tend to fluctuate more dramatically. Figure (b) validates our argument that a smooth kernel only has negligible high-frequency components. As we can see, the frequency do- main corresponding to the kernels when our regularization is applied shows significant differences in low-frequency components (center of the visualization) and high-frequency components (periph- ery of the visualization). Figure (c) further validates our intuition, showing that in comparison to
4

---

Under review as a conference paper at ICLR 2020
VR
VH
TR
TH
AR
AH
L
LR
LH
(a) Visualization the four convolutional kernels at the first layer
VR
VH
T
TR
TH
A
AR
AH
L
LR
LH
(b) Visualization the corresponding frequency domain
*
VR
VHI
T
TR
TH
A
AR
AH
L
LR
LH
(c) The internal representation after "star"
image passed through the kernels
Figure I: Visualization of the first four convolutional kernels of each model (first row), correspond- ing frequency domain (second row), and the internal representation when an image is passed into the kernel (third row).
AR
LR
CH
Figure 2: The averaged most deceptive images (one for each class) generated by FGSM for the testing models. O stands for the original example.
internal representations summarized by kernels from basic losses, those influenced by our regular- ization are more sensitive to the low-frequency signal (e.g. the shape of the input), and the internal representation with our regularization when the parameter is negative tends to focus more on the high-frequency signals.
Further, we check the mechanism of our model by inspecting how adversarial examples deceive the models. Figure 2 shows the four on-average most deceptive adversarial examples (the models predict incorrectly with the highest confidence) generated by FGSM. Notations follow the same convention the previous case, and O denotes the original image.
While many images have be perturbed with human perceivable amount to deceive the model, we notice that the adversarial examples for models with our regularization (+R) tend to behave in way that can be understood by human. The most convincing examples are at the first row for A and AR, where
clearly that the adversarial examples alter the decisions from star to circle. Other adversarial examples for +R models also introduce large areas that shall be interpreted as the shape. In contrast, adversarial examples for other models tend to introduce scattered patches, which

---

Under review as a conference paper at ICLR 2020
Table 1: Prediction accuracy adversarial examples. For MNIST and FashionMNIST, do
limit the e of adversrial examples. For CIFARIO and Restricted ImageNet, we limit the to be 0.
and 0.05 respectively (lxo difference, the maximum 1.0). Best performance for each data set for
each attack method is highlighted (except when the best performance is below random guess).
Model
clean
FGSM
PGD
DF((2)
DF((,))
C&W
ADef
S&P
V
0.989
0.013
0
0
0
0
0
VR
0.986
0.05
0.969
0.821
0
0.01
0
T
0.989
0.01
0.039
0.404
0.344
0.033
0.131
0.001
TR
0.983
0.138
0.53
0.547
0.465
0.162
0.772
0.109
MNIST
A 0.989
0.075
0.021
0.349
0.094
0.129
0.071
0
AR 0.988
0.223
0.017
0.314
0.088
0.163
0.044
0
L 0.951
0.136
0.022
0.466
0.072
0.094
0.12
0.001
LR 0.952
0.292
0.012
0.934
0.662
0.295
0.204
0.001
V 0.908
0.012
0.001
0.001
0.001
0
0
VR 0.885
0.009
0.075
0.758
0.635
0.005
0.413
0.002
T 0.918
0.04
0.151
0.275
0.271
0.014
0.157
0.003
Fashion MNIST
TR
0.873
0.055
0.166
0.785
0.741
0.086
0.554
0.003
A 0.869
0.065
0.038
0.141
0.101
0.015
0.081
0.035
AR 0.747
0.06
0.072
0.231
0.225
0.121
0.218
0.002
L 0.86
0.082
0.034
0.216
0.044
0.031
0.089
0.034
LR 0.752
0.176
0.103
0.573
0.448
0.354
0.335
0.012
V 0.85
0.02
0.02
0.02
0
O
0.64
0.85
VR
0.82
0.03
0.02
0.02
0
0.01
0.64
0.82
T
0.78
0.01
0.01
0.01
0
0
0.62
0.77
CIFAR1O
TR
0.81
0.03
0.03
0.03
0.01
0
0.74
0.81
0.1)
A 0.72
0.07
0.55
0.55
0.48
0.03
0.68
0.72
AR
0.72
0.09
0.58
0.58
0.57
0.02
0.72
0.72
L
0.66
0.12
0.56
0.56
0.55
0.03
0.65
0.66
LR
0.68
0.11
0.57
0.57
0.54
0.03
0.62
0.68
V 0.851
0.063
0.017
0.004
0.851
0.851
VR
0.855
0.047
0.001
0.025
0,001
0.006
0.855
0.855
Restricted ImageNet
A 0.404
0.081
0.022
0.33
0.03
0.189
0.404
0.404
AR 0.637
0.078
0.021
0.476
0.023
0.239
0.637
0.637
(c = 0.05)
I. 0.587
0.11
0.021
0.477
0.034
0.257
0.587
0.587
LR
0.591
0.096
0.02
0.468
0.036
0.254
0.591
0.591
will probably not be considered as the shape for most people. Also, if we apply our regularization with negative parameter (xH), the patches tend to behave in a more shattered manner.
4.2 STANDARD NUMERICAL EVALUATION
In Table I we report the prediction accuracy over the generated adversarial examples across the attack methods. For MNIST and FashionMNIST, we do not limit the of adversarial examples. In principle, when there is constraint, one should always be able find the adversarial example for any sample, however, in practice, many search attempts fail when the attack methods are set with the default hyperparameters in Foolbox. We consider these failures of searches (under default parameters) also measure of the robustness of models. For CIFAR10 10 and Restricted ImageNet, we and 0.05, respectively (the maximum pixel value is 1.0).
set the to
Overall, across most of the settings, our regularization helps achieve numerically the best adversar- ially robust models. Impressively, for MNIST and FashionMNIST., for some attack methods (e.g., both versions of DeepFool), our regularization only applied to the vanilla training loss, suggesting the importance the smooth regularization. Also, for these two datasets, the improvements of regularization mostly significant over the non-regularized counterparts. For CIFARIO and Restricted ImageNet, the performance gains are less significant but still observable. In the Appendix, report the accuracy and curves over fo.
improve the robustness significantly even when

---

Under review as a conference paper at ICLR 2020
Digit 0:
Digit 1:
Digit 2:
Digit
Digit 4:
Digit 5:
Digit 6:
Digit 7:
Digit 8:
Digit
T
VR
AR
LR
VR T TR AR L LR
Figure 3: Sample-independent interpretation of models trained over MNIST. I stands for the input.
T-shirt:
Coat:
Pullover:
Dress:
Trouser:
Shirt:
Sandal:
Bag:
Sneaker:
Boot:
VR TR
AR
L
LR
IV VR TR A AR L LR
Figure 4:
interpretation of models for FashionMNIST. I stands for the input.
lg, and foo distances, for more thorough comparisons. In general, the performances evaluated by the curves are consistent with results in Table
4.3 INSPECTING MODEL'S S PERCEPTION OF CLASS
We also leverage of the most classic model-inteeprretatinn methods, activation maximization (Erhan et al.] 2009). to further demonstrate the strength of our regularization. Concretely, we follow Simonyan al. 2013) Engstrom et al. 2019) maximize the logit of certain class that the most representative features of that class can be exaggerated in the input image. Specifically, with an input image of Gaussian noise, apply projected gradient descent 10000 iterations with learning rate 0.001 to update the input image. Notice that the interpretation is sample-independen.
Figure 3 depicts what the models consider be the digits. While V and T can barely be interpreted human, when our regularization is plugged in, the patterns appear to be observable, with impressive examples such as Digit and our regularization significantly helps in other cases, such as Digit 0, 1, 2, 4, and 8. Figure 4 shows similar story for FashionMNIST dataset: while might have the cleanest interpretation for the "sneaker" case, our regularization (especially AR) probably has the best interpretation in all other cases, with good examples such as "Trouser," "Dress." and "Boot." Interestingly, AR the only method that interprets "Bag" with a strap, and the average image of all training "Bag" samples in FaSHionMNIST
3. can also deliver interpretable decisions (e.g., Digit 3 and 5),
bag with a strap. Figure 5 shows the visualization of models trained on CIFAR10. While A seems have the best interpretation in "horse" case, AR and LR have equal or better interpretation in comparison with A in other cases. Impressively, only AR and LR understand "bird," and only AR understands "deer". Figure shows the visualization for Restricted ImageNet

---

Under review as a conference paper at ICLR 2020
plane:
mobile:
bird:
cat:
deer:
dog:
frog:
horse:
ship:
truck:
VR TR
AR
LR
T TR AR
Figure 5: Sample-independee interpretation of models trained over CIFARIO. I stands for the input.
Dog:
Frog:
Turtle:
Cat:
Bird:
Primate:
Fish:
Crab:
Insect:
A
AR
LR
1
A
AR
L
LR
1
A
AR
L
LR
Figure 6:
interpretation models trained over Restricted ImageNet. stands for the input. Results of simpler models not shown because they cannot be interpreted well.
(results of simpler models are not shown because they cannot be interpreted). AR is the only method that can describe the outline of the "bird" and "crab" classes, while these models seem to remain more or less the similar interpretation power for other labels.
Other results, such as visualization of targeted attack through saliency-based methods and selec- tive visualization of adversarial examples generated along with the experiments, are shown in the Appendix. Overall, the empirical evidence supports our intuition Section B] the regularization helps push the model to focus on the low-frequency components of the image and thus leads to more perceptually-aligne gradients.
CONCLUSION
Inspired by neuroscience literature emphasizing the connection between low-frequency components and shape recognition (Bar 2004 Awasthi et al., 2011), proposed smooth kernel regularization that forces the CNN to learn smooth convolutional kernels (kernels with small differences among adjacent weights) during training. As the relation between smoothness and low-frequency can be argued intuitively and supported by some known results in proved theorems (Titchmarsh 1948) BBracewlll 1986, Platonov| 2005), our regularization should help the model depend more on the low-frequency components of images. To verify the effectiveness of the regularization, we plug in the idea onto multiple training losses, including the vanilla loss, Trades loss (Zhang et al. 2019a), the adversarial training loss (Madry et al.] 2018), as well as variation of Logit Pairing loss (Kannan jet al. 2018). With seven different attack methods, we demonstrate the empirical strength our regularization with standard numerical evaluations. Further, also leverage the standard model interpretation methods to explain the decision of models, showing that our technique, like those demonstrated by Santurkar et al. (2019) tends to result in more perceptually- gradients.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Naveed Akhtar and Ajmal Mian. Threat of adversarial attacks on deep learning in computer vision:
A survey, IEEE Access, 6:1441014430, 2018. ISSN 2169-3536. doi: 10. 10.9100//ccss0000 2807385.
Rima Alaifari, Giovanni S. Alberti, and Tandri Gauksson. ADef: an iterative algorithm to construct adversarial deformations. In International Conference on Learning Representationss 2019.
Bhuvanesh Awasthi, Jason Friedman, and Mark A Williams. Faster, stronger, lateralized: low spatial frequency information supports face processing. Neuropsychologiaa, 49(13)335833333 2011.
Moshe Bar. Visual objects in context. Nature Reviews Neuroscience, 568):61,,0000
Ronald Newbold Bracewell. The Fourier transform and its applicatioon, volume 31999, McGraw- Hill New York, 1986.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium Security and Privacy (SP), pp. 39-57. IEEE, 2017.
Fabio Maria Carlucci, Antonio D'Innocente, Silvia Bueci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles, 2019.
Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopad- hyay. Adversarial attacks and defences: A survey, 2018.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples, In Proceedings of the 34th International Conference on Machine Ctarning-Vollume 70, pp. 854-865 JMLR. org, 2017.
Jeremy M. Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via ran- domized smoothing. In Proceedings of the 36th International Conference on Machine Learning, ICMI. 2019, 9-15 June 2019, Long Beach, California, USA, pp. 1310-1320, 2019.
Imre Csiszar and János Körner. Information theory: coding theorems for discrete memoryless sys- lems. Cambridge University Press, 2011.
Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Ankur Moitra, and Alistair Stewart. Robust estimators in high-dimensions without the computational intractability, SIAM Journal on Computing, 48(2):742 864, 2019.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry. Learning perceptually-aligned representations via adversarial robustness. arXiv preprint 1rXiv:1906.0099 00945. 2019.
Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of a deep network. University of Montreal, 1341(3):1, 2009.
Alhussein Fawzi, Hamza Fawzi, and Omar Fawzi. Adversarial vulnerability for any classifier. In Advances Neural Information Processing Systems 31: Anmual Conference on Neural Infor- mation Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada., pp- 1186-1195, 2018
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples (2014). In International Conference Learning Representutions, 2015.
Daniel Jakubovitz and Raja Giryes. Improving dnn robustness to adversarial attacks using jacobian regularization. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 514-529, 2018.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv: 1803.033,. 2018.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, Citeseer, 2009.

---

Under review as a conference paper at ICLR 2020
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. con/exdb/nmist// 1998.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Reprexentations, 2018.
Takeru Miyato, Shin ichi Maeda, Masanori Koyama, Ken Nakae, and Shin Ishii. Distributional smoothing with virtual adversarial training. 2015.
Seyed-Mohsen Moosavi-Dezfool Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pp. 2574-2582, 2016.
Seyed-Mohsen Moosavi-Dezfooli Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Ro- bustness via curvature regularization, and vice versa. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9078-9086, 2019.
Sergei Sergeevich Platonov. The fourier transform functions satisfying the lipschitz condition on rank 1 symmetric spaces. Siberian Mathematical Joumna, 46(6): 101-1088111 1118, 2005.
Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox benchmark the robustness of machine learning models, 2017.
Andrew Slavin Ross and Finale Doshi-Velee Improving the adversarial robustness and interpretabil- ity of deep neural networks by regularizing their input gradients. In Thirty-second AAAI confer- ence on artificial intelligence, 2018.
Shibani Santurkar, Dimitris Tsipras, Brandon Tran, Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Computer vision with single (robust) classifier. arXiv preprint
2019.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint arXiv:/312.6034 2013.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
Edward C Titchmarsh. Introduction the theory of fourier integrals. 1948.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. In International Conference Learning Representa- tions, 2019. URL TtLLttt/////ppeeeeee
net / Eorum? sidiiddyyy SyxAb--yxxx3cC
Haohan Wang, Songwei Ge, Eric P Xing, and Zachary C Lipton. Learning robust global repre- sentations by penalizing local predictive power. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, Vancouver, Canada., 2019a
Haohan Wang. Zexue He. and Eric P. Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019b.
Haohan Wang, Xindi Wu, Pengcheng Yin, and Eric P. Xing. High frequency component helps explain the generalization of convolutional neural networks. CoRR, abs/1905 13545, 2019c.
Tobias Weyand, Ilya Kostrikov, and James Philbin. Planet photo geolocation with convolutional neural networks. Lecture Notes in Computer Science, pp. 3755, 2016. ISSN 1611-3349. doi:
00.10077997-33331-44-688
3.
Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässsan Stockholm, Sweden, July 10-15, 2018, pp. 5283-5292, 2018.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint 1788.0777 2017.
10

---

Under review as a conference paper at ICLR 2020
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael Jor- dan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th International Conference Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, pp. 7472-7482, 2019a
Huan Zhang, Hongge Chen, Zhao Song, Duane Boning, inderjit dhillon, and Cho-Jui Hsieh. The limitations of adversarial training and the blind-spot attack. In International Conference on Learn- ing Representations, 2019b.

---

Under review as a conference paper at ICLR 2020
A MODEL AND HYPERPARAMETER CHOICES
For MNIST and FashionMNIST data set, the model is a simple two-layer architecture with two convolutional layers and fully connected layers. The Loo perturbation bound of PGD is set to 0.3/1.0 for MNIST and 0. 0/1. for FashionMNIST. For CIFARIO, the model is a ResNet18, and the Lo perturbation bound of PGD set to 0.03/1.0 (roughly 8/255). For Restricted ImageNet, the model is ResNet50, and the for perturbation bound of PGD is set to 0.005/1.0, then along the processing process, the pixel values of the images are divided by the standard deviation (0.2575), so is the perturbation bound. Also, for Restricted ImageNet, we continue with either the standard ResNet50 (for V and T losses) or the adversarially trained ResNet50 on Re- stricted ImageNet (Santurkar et al. 2219) (for A and L losses). With our hardware settings (NVIDIA 1080Ti), we cannot effectively train Trades loss over ResNet50.
12

---

Under review as a conference paper at ICLR 2020
B ACCURACY-EPSILON CURVES
The accuracy-epsilon curves are shown for lo. &2 and for bounds are shown Figure and Figure 9
Figure
VR
DeepFoolic...
SaltS/Pepper
20.75
0.22
0.0%
3.5
1.9
5.
0.5
30
6.3
13
6.
3.0 63
MNIST
FGSM
PGD
DeepFoollfy)
DeepFooli(..)
CSW
ADof
Salt&Pepper
1.00
0.59
0.0%
(b) FasSionMNIST
FGSM
Deepf colle.!
DeepF
Salt&Pepper
0.775
0.29
0.0
1.0
0.6
0.2
1.5
3.0
6.6
1.0
(c) CIFARIO
FGSM
PGD
DeepFooliia
DeepFoolde
C&W
ADef
Salt&Peepee
0.50
0.0%
(d) Restricted ImageNet
Figure 7: Accuracy over ln bound across different attack methods with models trained with the three data sets. Novel methods are denoted with solid lines.
13

---

Under review as a conference paper at ICLR 2020
VR
TR
AR
LR
FGSM
PGD
DeepFoolicy)
DeepFoolić_.)
C&W
ADef
Salt&Pepper
1.00
0.59
0.00
(a) MNIST
FGSM
PGD
DeepFoollf.
DeepFoolf
C&W
ADef
Salt&/Pipper
16
10
(b) FashionMNIST
DeepFoolkk,
DespFooll(..)
CSW
ADef
Salt&/Pepper
2..79
0.33
CIFAR10
1.00
FGSM
PGD
DeepFooliig!
DeepFoolit_
CSW
ADef
Salt&&Pepper
0.50
0.03
302
1000
(d) Restricted ImageNet
Figure 8: Accuracy over & bound across different attack methods with models trained with the three data sets. Novel methods are denoted with solid lines.
14

---

Under review as a conference paper at ICLR 2020
VR
TR
AR
LR
FGSM
PGD
DeepFoolicy)
DeepFooli(_.)
CSW
ADef
Salt&Pepper
1.00
0.59
0.09
(a) MNIST
FGSM
PGD
DeepFoollf,
DeepFool
C&W
ADef
Salt&/Popeer
5.5
1.0
0.5
8.5
3.0
(b) FashionMNIST
DeepFooll(,
DespFooll(..)
ADef
Salt&/Pepper
20.7
0.59
0.31
0.0%
CIFAR10
FGSM
PGD
DeepFooliig!
DeepFoolit_
CSW
ADef
Salt&&Pepper
1.00
0.59
0.09
(d) Restricted ImageNet
Figure 9: Accuracy over loo bound across different attack methods with models trained with the three data sets. Novel methods are denoted with solid lines.
15

---

Under review as a conference paper at ICLR 2020
C
TARGETED ATTACK
We also take advantage of the gradient to perform targeted attack, as shown in following figures. The titles of the columns describe the original class, and the titles of the rows describe the target classes
2
3
4
6
7
8
9
/
2
3
4
6
7
8
9
2
3
4
6
7
8
9
1
2
3
4
6
7
8
9
2
3
4
5
6
7
8
9
0
2
3
4
5
6
7
8
6
6
2
8
4
5
6
7
8
9
6
/
2
8
4
6
7
8
9
6
2
3
4
5
6
7
8
0
2
8
4
7
8
Figure 10: targeted attack visualization for V trained with MNIST. The titles of the columns describe original class, and the titles the rows describe the target classes.
16

---

Under review as a conference paper at ICLR 2020
2
2
3
4
2
3
4
7
8
2
3
4
6
7
8
2
8
4
6
7
8
6
6
/
2
3
4
5
6
7
8
2
8
4
6
7
2
3
4
7
8
2
8
4
6
7
8
0
2
8
4
7
8
Figure 11: targeted attack visualization for VR trained with MNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
17

---

Under review as a conference paper at ICLR 2020
/
2
3
4
6
7
8
2
3
4
6
7
8
9
0
2
3
4
6
6
7
8
9
/
2
3
4
6
7
8
6
2
8
4
5
6
7
8
9
6
/
2
3
4
5
6
7
8
9
2
3
4
5
6
7
8
9
0
/
2
8
4
6
7
8
9
6
/
2
3
4
5
6
7
8
9
0
/
2
8
4
5
6
7
8
Figure 12: targeted attack visualization for T trained with MNIST. The titles of the columns describe the original class, and the titles the rows describe the target classes.
18

---

Under review as a conference paper at ICLR 2020
2
Z
4
2
3
4
7
8
2
3
6
7
2
4
6
2
3
4
5
6
2
6
6
/
2
7
2
3
4
5
6
7
8
4
7
Figure 13: targeted attack visualization for TR trained with MNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
19

---

Under review as a conference paper at ICLR 2020
2
4
S
7
6
7
8
Figure 14: targeted attack visualization for A trained with MNIST. The titles of the columns describe the original class, and the titles the rows describe the target classes.
20

---

Under review as a conference paper at ICLR 2020
2
Figure 15: targeted attack visualization for AR trained with MNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
21

---

Under review as a conference paper at ICLR 2020
7
2
4
Exoricc
7
Z
Figure 16: targeted attack visualization for L trained with MNIST. The titles of the columns describe the original class, and the titles the rows describe the target classes.
22

---

Under review as a conference paper at ICLR 2020
7
2
3
3
4
2
2
7
7
Figure 17: targeted attack visualization for LR trained with MNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
23

---

Under review as a conference paper at ICLR 2020
T-shimmop
Trouser
Pullover
Driass
Cooe
Sandal
Shirt
Shacker
Eag
Antlee boot
Tousel
Sallveer
Eress
Strrt
Snewko
Aosk book
Figure 18: targeted attack visualization for V trained with FashionMNIST: The titles of the columns describe the original class, and the titles of the rows describe the target classes.
24

---

Under review as a conference paper at ICLR 2020
T-shiimop
Trouser
Pullover
Driass
Cooe
Sandal
Shirt
Shacker
Eag
Antlle boot
Sallverr
Eress
Strrt
Aosk book
Figure 19: targeted attack visualization for VR trained with FashionMNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
25

---

Under review as a conference paper at ICLR 2020
T-shimmop
Trouser
Pullovrr
Driass
Coos
Sandal
Shirt
Shackerr
Eag
Antlle boot
Tousee
Sallverr
Eress
CH
Jork book
Figure 20: targeted attack visualization for T trained with FashionMNIST: The titles of the columns describe the original class, and the titles of the rows describe the target classes.
26

---

Under review as a conference paper at ICLR 2020
T-shimmop
Trouser
Pullover
Driass
Coos
Sandal
Shirt
Shackorr
Eag
Antile boot
Tousee
Sallveer
Enees
Aock book
Figure 21: targeted attack visualization for TR trained with FaShionMNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
27

---

Under review as a conference paper at ICLR 2020
T-shimmop
Trouser
Pullovor
Drisss
Coor
Sandal
Shirt
Shecker
Eag
Antlee bcot
Trusee
Salever
OHH
BMG
Jock book
Figure 22: targeted attack visualization for A trained with FashionMNIST: The titles of the columns describe the original class, and the titles of the rows describe the target classes.
28

---

Under review as a conference paper at ICLR 2020
T-shimmop
Trouser
Pullovrr
Driass
Coor
Sandal
Shirt
Shacker
Eag
Antile bcot
Tousel
Sallver
Eress
Aosk book
Figure 23: targeted attack visualization for AR trained with FashionMNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
29

---

Under review as a conference paper at ICLR 2020
T-shimmop
Trouser
Pullovrr
Drisss
Cooe
Sandal
Shirt
Shackor
Eag
Antle boot
Tousee
Salover
Eress
OHH
Strr
BMG
Aock book
Figure 24: targeted attack visualization for L trained with FashionMNIST: The titles of the columns describe the original class, and the titles of the rows describe the target classes.
30

---

Under review as a conference paper at ICLR 2020
T-shimmop
Trouser
Pullover
Driass
Coor
Sandal
Shirt
Snacker
Eag
Antile boot
Sallveer
Enees
OH
Josk book
Figure 25: targeted attack visualization for LR trained with FaShionMNIST. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
31

---

Under review as a conference paper at ICLR 2020
Automobile
Desr
Home
Ship
Truck
Automantle
Deer
Thuck
Figure 26: targeted attack visualization for V trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
32

---

Under review as a conference paper at ICLR 2020
Automobile
Desr
Home
Ship
Truck
Automentle
Bim
Deer
09
Thuck
Figure 27: targeted attack visualization for VR trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
33

---

Under review as a conference paper at ICLR 2020
Automobile
Desr
Home
Ship
Truck
Automentle
Bim
Deer
Thuck
Figure 28: targeted attack visualization for T trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
34

---

Under review as a conference paper at ICLR 2020
Automobile
Desr
Home
Ship
Truck
Automentle
Bina
Deer
Thuck
Figure 29: targeted attack visualization for TR trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
35

---

Under review as a conference paper at ICLR 2020
Arrranw Automobile
Desc
Home
Ship
Truck
Automentle Arrieee
On
Oeer
Dog
Fro
Thuck
Figure 30: targeted attack visualization for A trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
36

---

Under review as a conference paper at ICLR 2020
Arrranw Automobile
Desr
Home
Ship
Truck
Auhomantte Arrieee
Bim
Deer
Drg
Thuck
Figure 31: targeted attack visualization for AR trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
37

---

Under review as a conference paper at ICLR 2020
Automobie
Desr
Home
Ship
Truck
Automantle Arrieee
Bin
Deer
09g
Thuck
Figure 32: targeted attack visualization for L trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
38

---

Under review as a conference paper at ICLR 2020
Automobie
Desr
Home
Ship
Truck
Automantte
Bino
OF
Dee
Dxg
Fro
Thuck
Figure 33: targeted attack visualization for LR trained with CIFAR10. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
39

---

Under review as a conference paper at ICLR 2020
dog
Cat
tuntle
cro
Insect
Figure 34: targeted attack visualization for A trained with Restricted ImageNet. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
40

---

Under review as a conference paper at ICLR 2020
dog
CaR
tuntl
cheepaeee
cro
Insect
Figure 35: targeted attack visualization for AR trained with Restricted ImageNet. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
41

---

Under review as a conference paper at ICLR 2020
dog
CaR
tuntle
cieppree
cro
Insect
Figure 36: targeted attack visualization for L trained with Restricted ImageNet. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
42

---

Under review as a conference paper at ICLR 2020
dog
CaR
tuntle
cieppree
cro
Insect
Figure 37: targeted attack visualization for LR trained with Restricted ImageNet. The titles of the columns describe the original class, and the titles of the rows describe the target classes.
43

---

Under review as a conference paper at ICLR 2020
D SELECTIVE ADVERSARIAL EXAMPLES
We visualize the generated adversarial examples to help us evaluate the models. We visualize the on-average most deceptive examples (the highest prediction confidence on the wrong class). We plot one example for each class of the data. For MNIST and FaShionMNIST, we focus on the visualization of adversarial examples generated by Adef attack because the attack more visually aligned with how human perceive the images.
44

---

Under review as a conference paper at ICLR 2020
3
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
S
S
S
S
S
S
5
S
6
6
4o
7
7
7
7
7
7
8
8
8
8
8
3
3
8
8
0
VR
T
TR
A
AR
L
LR
Figure 38: Adversarial Images generated by Adef attack
45

---

Under review as a conference paper at ICLR 2020
VR
T
TR
A
AR
L
LR
Figure 39: Adversarial Images generated by Adef attack
46


Reviews:
[{"review_id": "SkleCwjsYH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper summary:  This paper argues that reducing the reliance of neural networks on high-frequency components of images could help robustness against adversarial examples. To attain this goal, the authors propose a new regularization scheme that encourages convolutional kernels to be smoother. The authors augment standard loss functions with the proposed regularization scheme and study the effect on adversarial robustness, as well as perceptual-alignment of model gradients.\n\n\nComments: I will first discuss some high-level concerns I have with the paper, followed by more specific comments.\n\nMotivation and general idea:\nThe idea of improving model robustness by eliminating high-frequency components in the data is not new. This has in fact been the motivation behind several (later shown to be unsuccessful) defenses such as JPEG compression. In recent work, Yin et al. explore this phenomenon in depth, and importantly discuss how adversarial examples are by no means entirely a high-frequency phenomenon. Specifically, they show that while adversarial examples for natural models do tend to be biased towards higher-frequency components, this is not true for robust models. They argue that it is in general always possible to find adversarial examples in the lower end of the frequency spectrum (for instance, transfer attacks from robust models). Thus, based on evidence from prior work, it is unlikely that a defense based entirely on removing high-frequency components from the input data can be successful.\n\nEmpirical evaluation:\n\nThere are significant issues with the empirical evaluation of the proposed defense. In particular, in Table 1 (also Figures 7-9 in the Appendix):\n\n1. The numbers suggest that FGSM is more successful as an attack than PGD. The later is a multi-step version of the former, and hence should be strictly better (more successful in lowering model accuracy). This clearly highlights that the PGD attack is not being used correctly/not run with enough steps, and thus the numbers are not an accurate reflection of model robustness.\n\n2. At a higher-level, the numbers that the authors highlight in the table are the best performance over attacks. This is not the correct way to evaluate robustness, which must always be reported as the *worst-case performance* of the model and hence the lowest accuracy over attacks. If one takes this into consideration, it is clear that the proposed regularization does not really improve robustness.\n\n3. Furthermore, the authors state they use default parameters from Foolbox to evaluate their models. The issue with this is evident in the MNIST/FashionMNIST results where even with arbitrary eps, strong attacks like PGD are not able to break the model. This does not mean that the model is robust, it just means that the default hyperparameters from Foolbox cannot break the model. The authors need to re-run the attacks with different steps sizes/step counts for various attacks. The goal in evaluating model robustness should not be finding one attack (or one set of hyperparameters) which is not able to fool the model, but to show that *no attack* (for the given perturbation set) can lower model accuracy.\n\n4. The eps that are used for the CIFAR-10 and ImageNet experiments are extremely large, and not standard in the literature. In fact, based on my experience, it is probably not possible to be (too) robust to eps as large as 0.1 on CIFAR as this is large enough to visibly change the class even for a human (cf. Tsipras et al.). \n\nThus, I think the robustness evaluation in Table 1 is incorrect and violates some basic sanity checks (such as PGD being stronger than FGSM), and thus does not accurately reflect the model’s true performance. \n\n\nOther comments:\n\ni. It is unclear why the sensitivity of the proposed regularization scheme to the scale of w can be fixed by subtracting the norm of w (the exp loss will just be raised to the power alpha if you scale weights by alpha). Dividing by the norm would be the more correct way to fix the scale invariance and the authors should include results with this regularization, at least in the Appendix.\n\nii. In Figure 1, the kernels and activations that are visualized are after the first convolutional layer. The authors should perform a similar visualization after the second layer (or later layers in general) because it is somewhat obvious that you will get these properties at the first layer with the proposed regularization scheme (if the lambda is correctly tuned). The part that is unclear is how effective the regularization is over repeated applications of convolution coupled with non-linearities such as ReLU/pooling. \n\niii. While the visualizations in Figures 2-6 are interesting, it seems that many of the cases in which +R looks semantically meaningful, it also looks meaningful with the base loss itself. It thus is unclear whether the perceptual alignment is coming from the base loss, or from the added regularization. Moreover, the perceptual alignment of models should improve when their dependance on “human-meaningless” features reduces. Since high-frequency components are likely “human-meaningless” features, it is plausible that the proposed regularization scheme makes activation maximization/gradients more perceptually alignment. However, I think this is somewhat orthogonal to (and is not enough to tell us anything about) the robustness of the model itself, which is affected by its reliance on many kinds of “non-robust” features (Ilyas et al., 2019) of which high-frequency components might just be one example.\n\niv. In recent work by Wang et al., the authors conduct an interesting experiment to see how dependent the prediction of a given classifier is to high-frequency components in the data (vs low frequency components) (cf. Figure 1 in their paper). It would be interesting to see this experiment replicated with the proposed regularization scheme.\n\n\nOverall, I think there are significant issues with the paper, especially in the empirical evaluation section. The authors need to re-evaluate their models with the strongest possible form of the attack and demonstrate an improvement in worst-case performance to actually establish the merits of the proposed regularization scheme. Thus, I recommend rejection.\n\n\nReferences:\nYin, Dong, et al. \"A fourier perspective on model robustness in computer vision.\" arXiv preprint arXiv:1906.08988 (2019).\n\nTsipras, Dimitris, et al. \"Robustness may be at odds with accuracy.\" arXiv preprint arXiv:1805.12152 (2018).\n\nIlyas, Andrew, et al. \"Adversarial examples are not bugs, they are features.\" arXiv preprint arXiv:1905.02175 (2019).\n\nWang, Haohan, et al. \"High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks.\" arXiv preprint arXiv:1905.13545 (2019).\n\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "Hygk9pgCKH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a method for learning smoother convolutional kernels with the goal of improving robustness and human alignment. Specifically, they propose a regularizer penalizing large changes between consecutive pixels of the kernel with the intuition of penalizing the use of high-frequency input components. They evaluate the impact of their method on the adversarial robustness of various models and class visualization methods.\n\nAt a high level, the proposed idea is interesting. Reducing the reliance of classifiers on high-frequency patterns is a plausible way of improving human alignment. However, the experimental evidence presented is either unreliable or not sufficient to demonstrate the merit of the approach.\n\nMy biggest concern is the evaluation of adversarial robustness. The authors perform a number of off-the-shelf attacks using the foolbox library without accounting for fundamental differences between these attacks or basic principles of adversarial evaluation. Specifically, when evaluating the robust accuracy of a model, one needs to specify a concrete threat model (e.g., perturbations of L2-norm at most X), perform various attacks respecting this threat model, and report the number of  examples correctly classified against _all_ attacks (see https://arxiv.org/abs/1902.06705 for additional discussion and justification). Instead, the authors consider attacks with unbounded perturbation (for which robust accuracy does not make sense) and attacks using different perturbation constraints (ADef), while reporting improvements on a per-attack base. Hence, most of the columns of Table 1 are unreliable and cannot be taken into account when evaluating the models' robustness.\n\nEven ignoring these issues, taking into account the only column of Table 1 that is indicative, the PGD attack, we do not see sufficient evidence for the merit of the method. With the exception of a single outlier for MNIST TR (which I cannot interpret), the proposed regularizer only marginally affects the robustness of a classifier. In particular, no models become robust by adding the regularizer and the impact is limited for the case of already robust models.\n\nFurthermore, the evidence in favor of human alignment is relatively weak. While the method does have some effect for the case of simple models (MNIST and Fashion-MNIST), for the case of complex models (for which visualization is actually a challenging problem) the improvement is virtually non-existent. For Figures 5 and 6, the columns with R have essentially the same visual quality as the original columns. The only exception are the L and A columns for CIFAR10 bird and deer for which the original visualization fails. However, I find this confusing. There exist several works by now performing visualization using adversarial models and I have never encountered such a failure before. I wonder if there is some issue with the training of these particular models.\n\nOverall, while the high-level idea of the paper is interesting, the experimental evidence presented is either weak or unreliable. I will thus recommend rejection.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "BJgpiCsy5B", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1.\tPlease number the equations for better readability.\n2.\tThe usage of smooth kernel regularization, R(w) and logit pairing equation seems disjoint, what is the natural flow here?\n3.\tThe notation <X, Y> is misleading, do not use inner product notation to denote tuple!\n4.\tI do not see any justification of using KL\n5.\tThe contribution in terms of designing R(w) and the logit pair loss function is trivial.\n6.\tThe argmax to get x’ how practically you ensure the solutions are inside the closed ball? The authors mentioned about robustness without any justification. Why related to TV norm? The usage of Pinsker’s inequality to get upper bound is meaningful but that doesn’t prove the robustness. Please explain, also why not state this as a theorem. I suggest prove the robustness in a concrete manner, maybe using influence functions.\n7.\tWhat is the loss function l(.,.) supposed to be?\n8.\tIn R(w) the second part supposed to minimize the norm of convolution kernel I presume, then why there is a negative sign!\n9.\tThe title of Section 4.1 is pretty strange, you don’t have to say “… for sanity checking”.\n10.\tThe synthetic experiment is very immature and inconclusive. What can we get from Table 1? Also please justify the choices of hyperparameters used.\n11.\tAgain, looking at Fig. 4, I really can’t see the usefulness.\n12.\tSection 4.3 is meaningless and seems redundant. Why not have a single story rather than so many branching, the experiments are not convincing at all.\n13.\tIn Fig. 5, authors argued AR is better than A, I don’t see why, e.g., for horse A looks uch better than AR, same for insect. \n14.\tThe regularization seems not that useful, to me this work tries to justify using a regularization which by the choice of experiments is not well grounded.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a regularized for convolutional kernels that seeks to improve adversarial robustness of CNNs and produce more perceptually aligned gradients. While the topic studied by the paper is interesting, reviewers pointed out several deficiencies with the empirical evaluation that call into question the validity of the claims made by the authors. In particular:

1) Adversarial evaluation protocol: There are several red flags in the way the authors perform adversarial evaluation. The authors use a pre-defined adversarial attack toolbox (Foolbox) but are unable to produce successful attacks even for large perturbation radii - this suggests that the attack is not tuned properly. Further, the authors present results over the best case performance over several attacks, which is dubious since the goal of adversarial evaluation is to reveal the worst case performance of the model. 

2) Perceptual alignment: The claim of perceptually aligned gradients also does not seem sufficiently justified given the experimental results, since the improvement over the baseline is quite marginal. Here too, the authors report failure of a standard visualization technique that has been successfully used in prior work, calling into question the validity of these results.

The authors did not participate in the rebuttal phase and the reviewers maintained their scores after the initial reviews. 

Overall, given the significant flaws in the empirical evaluation, I recommend that the paper be rejected. I encourage the authors to rerun their experiments following the feedback from reviewers 1 and 3 and resubmit the paper with a more careful empirical evaluation.
Decision: Reject