Adversarial and Natural Perturbations for General Robustness
Authors: Sadaf Gulshad
Jan Hendrik Metzen
Arnold W.M. Smeulders
Venue:  (2021)
Keywords: Robustness
Adversarial Examples
Natural Perturbations
General Robustness

Abstract: In this paper we aim to explore the general robustness of neural network classifiers by utilizing adversarial as well as natural perturbations. Different from previous works which mainly focus on studying the robustness of neural networks against adversarial perturbations, we also evaluate their robustness on natural perturbations before and after robustification. After standardizing the comparison between adversarial and natural perturbations, we demonstrate that although adversarial training improves the performance of the networks against adversarial perturbations, it leads to drop in the performance for naturally perturbed samples besides clean samples. In contrast, natural perturbations like elastic deformations, occlusions and wave does not only improve the performance against natural perturbations, but also lead to improvement in the performance for the adversarial perturbations. Additionally they do not drop the accuracy on the clean images.

Under review as a conference paper at ICLR 2021
ADVERSARIAL AND NATURAL PERTURBATIONS FOR GENERAL ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper we aim explore the general robustness of neural network clas- sifiers by utilizing adversarial as well as natural perturbations. Different from previous works which mainly focus on studying the robustness of neural networks against adversarial perturbations, we also evaluate their robustness on natural per- turbations before and after robustification. After standardizing the comparison between adversarial and natural perturbations, we demonstrate that although ad- versarial training improves the performance of the networks against adversarial perturbations, it leads to drop in the performance for naturally perturbed samples besides clean samples. In contrast, natural perturbations like elastic deformations, occlusions and wave does not only improve the performance against natural per- turbations, but also lead to improvement in the performance for the adversarial perturbations. Additionally they do not drop the accuracy the clean images.
INTRODUCTION
A large body of work in computer vision and machine learning research focuses on studying the robustness of neural networks against adversarial perturbations (Kurakin et al. 2016 Goodfellow jet al. 2014 Carlini & Wagner 2017). Various defense based methods have also been proposed against these adversarial perturbations (Goodfellow et al.] 2014; Madry et al.] 2017 Zhang et al.) 2019b Song al., 2019). Concurrently, research also shows that deep neural networks are not even robust to small random perturbations e.g. Gaussian noise, small rotations and translations
(Dodge & Karam| 2017 Fawzi & Frossard 2013) Kanbak et al. (008).. There plenty research being performed in the domain of adversarial perturbations however, there is very little focus robustifying the networks against natural perturbations as we do here.
Furthermore, adversarial perturbations are difficult to be found in the real world, and naturally oc- curing perturbations are of different nature than these pixel based perturbations. Therefore, this paper consider natural perturbations of six different styles that noise, wave, saturation, and Gaussian blur. In this, "elastic" denotes a random sheer transformation applied to the image, "occlusion is large randomly located dot in the image and "wave" is ran- dom geometric distortion applied to the image. Additionally, there is no consensus about whether adversarial robustness helps against natural perturbations. Zhhang Zhu (201)) showed that adver- sarial training reduces texture bias. However, [Egstrom et al. (2019) demonstrated that to based robustness does not generalize to natural transformations like rotations and translations. Here we evaluate whether adversarial training helps against natural perturbations and vice versa.
elastic, occlusion, Gaussian
Besides the robustness of the neural networks against natural and adversarial perturbations there is an open debate in the literature about the trade-off between the robustness and the accuracy (Tsipras) al. 2018 Zhang al.] 9a; Su et al., 2018). Contrasting with adversarial training we found that networks partially trained with naturally perturbed images does not degrade the classification performance on the clean images. On the CIFAR- 10 dataset, we even found that partial training with naturally perturbed images improves the classification accuracy for clean images.
Given that deep neural networks are on par in performance with humans or they even surpass hu- mans clean images however, they fail to perform well on small natural perturbations (He et al. 2016 Dodge & Karam, 2017). [Hendrycks & Dietterich| (2019) introduced subset of Imagenet Deng al. (2009) called Imagenet-C with corruptions applied on images from Imagenet. Although

---

Under review as a conference paper at ICLR 2021
in Imagenet-C each corruption has five severity levels however they are not standardized for com- parison among them. We standardize the effect of perturbations on training data to a fixed drop in classification accuracy of the test set, through this we allow for fair comparison between differ- ent styles of training to retain robustness in the classifier against perturbations. We also normalize the performance of the network on different datasets to compare the robustness of the network for different datasets.
We conduct 320 experiments on five different datasets for adversarial and six different natural per- turbations. General robustness is the most desired case given as, the robustness against perturbations not seen during the training of the classifier. Hence, we evaluate the general robustness of networks by testing them on seen perturbations i.e. when the training and testing type of perturbations is the same, as well as on unseen perturbations i.e. when the training and testing type of perturbations are different. Among classifiers tested on the both seen and unseen perturbations, the natural perturba- tions of elastic, wave and occlusion come out on top compared to other natural perturbations as well as compared to adversarial perturbations. Our contributions are given as follows: i) We perform fair evaluation of robustness. ii) We show that, natural perturbation robust classifiers generalize to clean images. iii) We depict that, seen natural perturbations more robust than seen adversarial perturbations. iv) Our evaluation for general robustness shows natural elastic, wave and occlusion perturbations are best robust against unseen perturbations.
2 RELATED WORK
Robustness with Adversarial or Natural Perturbations. In Goodfellow et al. (2014) the robust- ness of neural networks was demonstrated by adding imperceptible i.e. adversarial perturbations in the input to the degree that it will misclassify the input into the wrong class. To solve the prob- lem Carlini & Wagner (017)) proposed adversarial training (AT) procedure that is by training the network on adversarially perturbed images networks can be robustified against these perturbations. In this work we employ a strong yet undefended attack "basic iterative method (BIM)" generate adversarial examples. "Projected gradient descent (PGD)" state of the art defense techmique for adversarial training to evaluate its effectiveness compared to other ways of robustification. Zhang et al. 2019a); T'sipras et al. (2018) questioned the generalization capability of adversarially trained neural networks on the clean images and showed that with the increase in adversarial robustness clean images drops. Therefore, we evaluate the performance of adversarially trained networks on clean, adversarial as well as natural perturbations, and compare them with networks trained with natural perturbations.
the accuracy of the networks
Hendrycks & Dietterich (2019) focused on testing the robustness of vanilla neural networks on 15 different natural perturbations with different perturbation levels. We observe that some of their perturbations are correlated c.g. Gaussian noise, shot noise and impulse noise (Laugros et al.) 2019). While, in our work train and test on six different natural perturbations covering the breadth of styles of natural perturbations. Furthermore, instead of selecting different perturbation levels randomly we standardize their effect by dropping the accuracy of the network fix level for fair comparison among them. Finally, rather than testing vanilla networks, we propose to robustify the networks with natural perturbations and test them for both adversarial and natural perturbations.
General Robustness with Adversarial and Natural Perturbations, Ford et al. (2019) established the close connections between adversarial robustness and natural perturbations robustness and sug- gested that adversarial and natural perturbations robustness should go hand in hand and networks should be robustified against both of them. In another similar line of work Kang et al. (2019); Engstrom et al. (2019). proposed natural perturbations based adversarial attacks and showed that testing with only one type of adversarial perturbations does not tell about the complete robustness of the network. We focus on determining the general robustness of neural network classifiers by testing them against unseen adversarial and natural perturbations.
Rusak et al. 2020 focus on robustification against natural corruptions besides adversarial pertur- bations. They utilize Gaussian and speckle noise and show that by augmenting the properly tuned training of a network with Gaussian noise makes it generalizable unseen natural perturbations. However, in this work we show that elastic, wave and occlusion perturbations surpass the robustness with Gaussian noise. Laugros et al. (2019) study the relationship between adversarial and natural
2

---

Under review as a conference paper at ICLR 2021
Elastic Trunsform ç*
Obstruction ç
Gaussian Noise (M
Wave Transform.
Saturation
Gaussian Blur
Figure 1: Randomly selected sample images of six different natural perturbations used in our exper- iments. Note that the perturbations for each image vary e.g. for another image the the occlusion will be another position in the image.
perturbations. However, they not study elastic and wave transforms. Furthermore, they generate adversarial examples by randomly selecting parameters but we select the parameters of both natural and adversarial perturbations by standardizing the effect of perturbations. So, their results contrast with ours. They also do not evaluate the performance of robustified networks on clean images.
METHOD
Given the Reh input image 2n, E 2 and the output class yn N, a standard classifier f(x,) yo predicts the class. In the real world, inputs of the classifiers may deviate from the learning set, whose members will be referred as clean images. As representatives of the perturbed images we consider sets of naturally occurring perturbations çh. and adversarial perturbations ÇAA for the purpose of enhancing the robustness
classifier.
Constructing Adversarial Perturbations. Adversarial examples satisfy two properties 1) the class for the perturbed image is different from the class predicted for clean image i.e. S(CA (x.)) + (xx,). 2) they are visually similar and their similarity is determined by the 4p- norm. While fulfilling these two properties we follow the standard procedure of the basic iterative method Kurakin et al. 2016) to introduce adversarial perturbations 5,7 (xx) J. in the images by finding the perturbation on with a small norm loo bounded by e such that, f(xn) ) + f(C (xn)). where SA = In + 8, and 5n I c. The equation to be optimized is given
CA = Fn AA (x = cA (xxt) (C (xt)) Vn. 0))
(2)
where, 6g is the step size at step &.
Constructing Natural Perturbations. For natural perturbations also restrict them satisfy two properties 1) the overall drop in the performance classifier is the same as the drop with the adversarial perturbations for comparison among them, they are visually similar enough to be correctly classified by humans. We consider a set of naturally occurring perturbations çt where t {F, O, N. W, S, B) denotes the type of perturbation operator. We construct images Çh (xr,)) by selecting perturbation operator from t. When tested on standard classifier, the perturbation will cause a drop in the performance. Selected samples of the six natural perturbations are shown in the Figure.
The first natural perturbation is Elastic deformation CF. Elastic deformation appears in small changes in the viewing angles. We introduce this perturbation by Ç T T(x,, ax', @ N(u, o o²)), where, x' E rand(-1, +1,,, +1), selects random number between -1 and +1, generated with uniform distribution and T is the affine transform. Occlusion is created with ço = min(r,, beedr) where, b is a matrix of zeros with Fe as its center and i, r being the thickness and radius of the circle
respectively. Gaussian noise is introduced using ( (xx) In x N(Hoo²) A Wave transform is

---

Under review as a conference paper at ICLR 2021
introduced in the images through Ç In b (In sin(2 where, - is shift operator. Saturation is added by using < (1 - a²xx' + az where, a E [0, 1]. x' is the black and white version of zn,- Gaussian blur {d is added by convolving two dimensional Gaussian function to the image.
Although these natural perturbations class agnostic however, they are image specific that is, the perturbation for each image is different. For elastic transform the intensity the transform is varied, in the occlusion the position of occlusion is randomly selected for each image, intensity of Gaussian noise is varied uniformly at random, wave is also scaled uniformly at random for each image, similarly saturation factor is also uniformly selected and finally, the intensity of Gaussian blur is uniformly randomly sampled for each image.
Fair Comparison. To permit the fair comparison between natural and adversarial perturbations, instead of selecting perturbations randomly at different levels of intensity to normalize the im- pact of study we propose robustification level which allows us to select the parameters of all perturbations such that, the performance drops to specific level for all the perturbations i.e. #of {f(,,, /11" (x.)))
#ol (1
a.
3.1 ROBUSTIHICATIO.
We consider Cross entropy loss as the standard training loss L. of a neural network with parameters / trained the training set S {(rn, Ba) E X, Vn Y).
Adversarial Perturbations Training. Next, considering the adversarial training method from Goodfellow et al., (2014) the network is trained with clean well with adversarial perturbed L. + L. Adversarial loss is given as
images. Hence, the total loss to optimize becomes =
min S
Σ
C(S(S, (z.n),
(CA (IN))
In spite of the fact that clean images are well represented in the training set this results in the drop of performance on clean images Zhang al. (2019a). Additionally, these perturbations are different from the naturally occurring perturbations and are rarely found in practice. Therefore, we utilize a simple but effective technique to train the network on naturally perturbed images. We argue that natural perturbations enhance the class boundary more precisely. Basically, two clean images may differ by an elastic deformation or occlusions and training a network on them therefore help the classifier to learn better.
Natural Perturbations Training. this section, we train our network with clean and naturally perturbed samples. We test these robustified networks clean, adversarial perturbed and naturally perturbed samples. The total loss to optimize is given as = C. LS. The loss for naturally perturbed images is
Σ
min 0
SS
3.2 IMPLEMENTATION DETAILS
Evaluation Metric. In order to evaluate the performance of the classifiers we consider drop in the accuracy as an evaluation metric defined as
AA 1==1([(( (f(((" = yv.) 1(f(x.) yn))
(5)
where, 1 is the indicator function.
Standard and Perturbed Image Classification. We perform clean image classification using Resnet-155 52 network pre-trained on Imagenet and fine tuned on the respective dataset. We construct adversarial images using BIM method with number of steps K 10 and values such that our de- sired drop in the accuracy is achieved as shown in Figure. [ each dataset. The similarity metric to determine similarity between clean and adversarial perturbed images Tao norm. We also construct naturally perturbed images using the method described in section 3. The parameters for natural per- turbations are set in such way that the same drop as adversarial is met each dataset Figure
4

---

Under review as a conference paper at ICLR 2021
Standardizing Comparison Among Perturba-
tions. We standardize the comparison among ad- versarial and natural perturbations by satisfying following two properties. 1) We select the param- of the perturbations such that there signif- icant drop in the accuracy to study the robustness and images are still visually recognizable by hu- mans. 2) for the fair comparison between adver- sarial and natural perturbations, the parameters of both types of perturbations are set such that the drop in the accuracy with natural perturbations
A Accuracy of Classifier
Standard tested on Adversarial
Standard tested Elastic
Standard tested on Occlusion
-20
Standard tested Gaussiar Noise
Standard tested on Wave
Standard tested Saturation
-25
Standard tested on Gaussiar Blur
same with adversarial perturbations for each
Standard tested Clean (Normalized)
dataset Figure. 2
-30
CUB
StanfordCe's
AWA
LAD
CIFAR-10
Adversarial Perturbations Training. We robus- tify the network using projected gradient descent method with the same values which lead to the drop in Figure. for each dataset and number of steps K taken as 10.
Figure Calibrating the drop in the accuracy.
Natural Perturbations Training. We robustify the network with natural perturbations introduced in the images with the method explained the section 3.1 and parameters which lead to drop in Figure
4 EXPERIMENTS AND RESULTS
Datasets. The five datasets of varying size and granularity used in our experiments are Large At- tribute Dataset (LAD) Zhao Zhao al. (2018), Animals with Attributes (Aw (AwA)Xian al. (2019), Stanford Cars dataset Krause et al. (013), CUB-birds (CUB) Welinder et al.] (2010) and CIFAR10 dataset Krizhevsky al. (2009). LAD contains 78017 total number images with 230 classes. We use (11702 train/ 9947 val 92884 test) for our experiments. AWA contains 37322 images with 50 classes. We use (10450 train/ 7524 val 9674 test). The CUB dataset consists of 11,788 images (5395 train / 599 val 5794 test) belonging 200 fine-grained categories of birds. Stanford Cars dataset contains (8 (8144 train, 8041 test) images with 196 fine grained categories of cars. CIFARIO dataset consists of 10 coarse grained categories with (50,000 train, 10,000 test) images.
4.1 STANDARD NETWORK CLASSIFICATION
Normalizing Accuracy. We start with evaluating the performance of a standard classifier on the clean images. A standard classifier shows the accuracy of 81.20% CUB, 86.48% stanford Cars, 94.79% on Aw 83.75% on LAD and 87.86% on CIFARI0 dataset. For fair comparison among the results of different datasets normalize the performance of the classifier each dataset and show it with black line (cross symbol) on 0 in Figure. 2
Calibrating the Drop in the Accuracy. Considering the accuracy of standard classifier the clean images as the baseline we drop the accuracy of the network with adversarial as well as natural perturbations 10%. The maximum variation among drops with all the perturbations on one dataset is of standard deviation 0.26 which is negligible as compared to 10. The drop in the accuracy of the network with adversarial as well as natural perturbations -10). Each black line shows the drop in the accuracy with different type of perturbation and each point shows one experiment. We can observe from the plot that the drop for each dataset and each perturbation is achieved with minor variations among them. Hence, we achieve equal drop for both adversarial and natural perturbations to perform the fair comparison among them.
shown in Figure 2 (black lines at
4.2 EVALUATING ROBUSTIFIED NETWORKS ON CLEAN IMAGES
We robustify the classifiers through adversarial training and natural perturbations training and trast their performance on clean images. Figure Balshows the performance for the clean images when the networks robustified with adversarial training. The yellow line with the cross symbol in the
5

---

Under review as a conference paper at ICLR 2021
A Accuracy of'a Classifier
AT tested en Clean
A Accuracy of Classifier
-5
-5
-10
-15
Elastic tested on Clean
Occlusion tested Clean
-20
20
Gaussian Noise tested Clean
Wave ceste Clean
25
25
Saturation tested
tested on
-30
-30
CUB
StanfordCars AWA
CIFAR-10
StanfordCars
LAD
CIFAR-10
(a) Evaluating adversarial training clean im-
(b) Evaluating natural perturbations training
ages.
clean images.
Figure 3: Comparing the performance of adversarial training with natural perturbations training on clean images.
plot shows the performance of the network robustified with adversarial training clean images. While Figure shows the performance when the network is robustified with natural perturbations and tested on clean images. Each line plot with different color shows a network robustified different natural perturbation and tested on clean images (cross symbol). We observe that, adver- sarial training leads drop in the accuracy on clean images for all the datasets except CIFARIO while robustification with natural perturbations does not lead to the drop in the performance on the clean images. The network robustified with Gaussian perturbations for CUB, AWA and LAAD dataset and the network robustified with Gaussian blur for LAD dataset does not completely recover the accuracy, however the drop less as compared to robustification with adversarial training. For coarse grained CIFARIO 10 dataset robustification with all the natural perturbations even leads to im- provement in the performance on clean images Hence, this shows that robustification with natural transforms does not deteriorate the performance of network on clean images while robustification with adversarial perturbations leads to drop in the accuracy for clean images on four out of five datasets.
4.3 EVALUATING ROBUSTIFEED NETWORKS ON SEEN PERTURBATIONS
We compare the performance of adversarially trained networks with naturally robustified networks on seen perturbations. The results for the adversarially trained network tested on adversarial pertur- bations are presented in Figure (yellow line with plus symbol). While, results for the naturally robustified networks tested on same type of natural perturbations shown in Figure 4b) Each line in the plot with different color shows network robustified different natural perturbation and tested on same kind of natural perturbation. By testing the performance of an adversarially trained network adversarial images, and the naturally robustified networks images contain- ing same type of natural perturbations we observe that although adversarial training helps against adversarial perturbations however, the improvement in the performance with natural perturbations is higher. We further notice that for CIFARIO dataset the drop in the performance due to natural perturbations completely recovered. Hence, these results show that, robustification with natural perturbations outperform robustification with adversarial perturbations on the seen test set.
4.4 EVALUATING ROBUSTIFIED NETWORKS ON UNSEEN PERTURBATIONS
Ineffectiveness of Adversarial Training Against Natural Perturbations. The results for adver- sarially trained network tested on unseen natural perturbations are shown in Figure 5a 5c 5e Each line in each subplot shows an adversarially trained network tested on a different natural perturbation with the symbol representing the type of the perturbation. We can clearly observe from the plots that adversarial training does not help against natural perturbations but causes a further drop in the performance. This drop sion on AWA and Gaussian on LAD dataset. This is contrast to the results presented in Laugros et al. (209)) where the authors showed that the performance of an adversarially trained network is
double e.g. against occlusion
the CUB, elastic and occlu-

---

Under review as a conference paper at ICLR 2021
5
5
A Accuracy of a Classifier
AT tested on Acversarial
A Accuracy of Classifier
0-
-5
-10-
-10
Elastic tested on Elastic
-15-
-15-
Occ usion tested on Occlusion
-20
20
Gaussian Noise tested on Gaussian Noise
Wave tested on Wave
-25
25
Saturation tested Saturation
Gaussian Blur tested on Gauss an Blur
-30
-30
CIFAR-10 (a) Evaluating adversarial training on the seen perturbations.
CUB StanfordCars AWA
LAD
CUB
StanfordCars
AWA
LAD
CIFAR-10
(b) Evaluating natural perturbations training on seen perturbations.
Figure 4: Comparing the performance of adversarial training with natural perturbations training on seen perturbations.
the same as standard network on natural perturbations. We argue that this difference is because we compare natural perturbations with adversarial after standardizing their effect. Thus, our results show that adversarial training does not generalize to natural perturbations but leads to further drop in the performance.
Effectiveness of Natural Perturbations Training Against Adversarial and Unseen Natural Per- turbations. Each subplot in Figure [b] [d 5f shows the results for network trained different type of natural perturbation and tested on unseen perturbations. Within one subplot each line plot shows testing different perturbation with color representing the training perturbation and sym- bol representing the test perturbation. For example, the first subplot is trained on elastic perturbation (red color) and tested on occlusion, Gaussian noise, wave, saturation and Gaussian blur with sym- bols square, star, triangle up, pentagon and triangle down respectively.
In each subplot of Figure 5b 5d 5f line with "plus" symbol shows the performance for training with the natural perturbations and tested on adversarial perturbations. We can clearly observe that robustification with natural perturbations generalizes adversarial perturbations, Furthermore, this shows a similar pattern among all the six natural perturbations training. On CUB and StanfordCars dataset it recovers all the drop, on AWA and LAD datasets it reduces the drop from 10% to around 2% ~ 3%, and on CIFAR10 10 dataset it even helps to improve the accuracy on adversarial images. Contrary to this in Laugros et al. (2019) the authors depicted that training with natural does not generalizes to adversarial. Our results show that, robustification with natural perturbations training transfers to adversarial perturbations.
Augmentation with elastic perturbations leads to improvement in the performance against all the unseen natural as well as adversarial perturbations except Gaussian perturbation. For CIFAR1O also leads to drop on wave perturbation besides Gaussian. Elastic and wave which look similar however, they do not perform well on each other on CIFAR1O which shows that they are not corre- lated on CIFARIO. Training with occlusion perturbations shows similar behavior elastic also enhances the accuracy on all the unseen perturbations except Gaussian and wave perturbations on the CIFAR 10 dataset. However, it significantly helps to improve the performance against elastic and saturation perturbations on the CIFAR 10 dataset.
Training with Gaussian perturbations provides minimum defense against unseen natural perturba- tions and shows worst performance on AWA dataset. On AWA dataset it leads to further drop in the performance from 10%. Augmentation with saturation does not lead to much drop on unseen test set however it does not improve much either. Robustification with the wave perturbations helps against all of the unseen perturbations except a little drop on Gaussian perturbation for CIFARIO I0 dataset. Finally, training with the Gaussian blur shows an average behavior, for some perturbations like elastic on CIFARIO it leads to the complete recovery in the drop whereas for some of them like Gaussian noise it leads to the further drop in the performance and for rest of the unseen perturbations it did not help. We also observe that, robustification with any of the natural perturbations transfers to adversarial however most of them fail perform well the Gaussian noise this implies that

---

Under review as a conference paper at ICLR 2021
Accuracy of Classifier
Accurncy of Classiffer
Accuracy a Classiffer
-3-
-10
-15
Elattic terted Adsartanil
Llattic teaned Ocs Lccision
-20
Elactic 103000 causs an Noda
Voise
Elactic ico0od Ware
-25
tester Elastic
Elastic rested Seturation
22
Occuzacn tictad Scturation
Occlussio
Cladtic
Oullzainn lodaal Geuosia
-30
CLB
SarffrCarr AMA
LAD
CRAR-1
StrffndCars ANA
LAO
OFAN-10
CUB
SarfordCars AMA
AD
OFAR
(a) Evaluating adversarial train-
(b) Evaluating natural perturbations training unseen perturba- tions.
ing on elastic and occlusion perturbations.
Accuracy ofa Classifeer
Accurccy of Classifier
A Accuracy of Classlier
-10
Wave teited on Adversati
Wave on Elastic
-20
25
tested Gaussion
Gounsian lelee tarted on Dedia
Gounsan tecteedddd Bastic
leveed Wive
20
Wize Saaaddinn
Wine Ousiuzion
Goussion
Counserr Male boced Celsilar
CUB
ManntordLars
AMA
LAD
GIA-JU
CU3
StenbreCCrs
AMA
LAG
DFAB-10
RanfeedCars
UAO CIFA-300
(c) Evaluating adversarial train-
(d) Evaluating natural perturbations training unseen perturba-
ing on the Gaussian noise wave perturbations.
tions.
A Accumicy ofa Classifeer
A Accemicy al'a Chassiier
A Accurncc of'a Classiiffer
-10
-20-
Saturalion tedied Eleetic
Scturotin tasted on Occission
Causs Our tested Adverania
Gauos in Dlr tested on Twutt:
tasted Occlation on
Seturab tested on Counsion Nobe
-25
Saturat on tested on Wave
texted Saturat
teated on Gaussian
tasted Gutsson
Oauss un Our tested Wave
CJB StanfordCars
LAD
CFAR10
Stari
CUB
SiarforeCars AMS
LAD
OTAR-20
(c) Evaluating adversarial train-
(f) Evaluating natural perturbations training unseen perturba-
ing on the saturation and Gaus-
tions.
sian blur perturbations.
Figure 5: Comparing the performance adversarial training on unseen perturbations with natural perturbations training unseen perturbations.
adversarial and Gaussian are not correlated. Thus, evaluation on unseen perturbations depict that augmentation with the six natural perturbations under consideration robustify the networks against adversarial perturbations. By comparing the subplots with each other we learn that "elastic", "occlusion" and the "wave" are the best performing ones.
CONCLUSION
In this work, focus general robustness and robustify networks with natural as well as adver- sarial perturbations while standardizing comparisons among them. We demonstrate that adversarial training leads the drop in the accuracy clean images while robustification with natural per- turbations does not degrade the performance on the clean images, even for CIFAR10 10 it leads to the improvement in the performance. We also showed that classifiers trained with natural perturba- tions show better improvement in the performance on seen perturbations than ad adversarial training on adversarial images. Finally, we contrasted the results of adversarially trained networks on unseen perturbations, with natural perturbations trained networks on unseen perturbations. We observed that all the natural perturbations being considered improved the accuracy on adversarial examples. "Elastic", "occlusion" and "wave" showed the best performance on unseen perturbations. In con- trast, adversarial training lead to a further drop in the accuracy on unseen perturbations. Thus, although general robustness against any arbitrary perturbation is hard prove, conclude that natural perturbations added to the training scheme provide always better robustness than adversarial training does to (almost) any of the unseen perturbations have provided.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). pp. -9-57. IEEE, 2017.
J. Deng. W. Dong, R. Socher, L. Li. Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Samuel Dodge and Lina Karam. A study and comparison of human and deep learning recogni- tion performance under visual distortions. In 2017 26th international conference on computer communication and networks (ICCCN), pp. 1-7. IEEE, 2017.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Ex- ploring the landscape of spatial robustness. In International Conference on Machine Learning, pp. 1802-1811, 1, 2019.
Alhussein Fawzi and Pascal Frossard. Manitest: Are classifiers really invariant? arXiv preprint arXiv: 2015.
Nic Ford, Justin Gilmer, Nicolas Carlini, and Dogus Cubuk. Adversarial examples are natural consequence of test error in noise. arXiv preprint arXiv:1901.10113, 2019.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. arXiv preprint 1903. 2019.
Can Kanbak, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Geometric robustness of deep networks: analysis and improvement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4441-4449, 2018.
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardr. Testing robustness against unforeseen adversaries. arXiv preprint arXiv: 1et.000080006 2019.
Jonathan Krause, Michael Stark, Jia Deng, and Fei-Fei. 3d object representations for fine- ane-grained categorization. In 4th International IEEE Workshop 3D Representation and Recognition (3dRR-13) Sydney, Australia, 2013.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv: 1607.0060..025:: 2016.
Alfred Laugros, Alice Caplier, and Matthieu Ospici. Are adversarial robustness and common pertur- bation robustness independant attributes? In Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 0-0, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks, arXiv preprint arXiv: 06083, 2017.
Evgenia Rusak, Lukas Schott, Roland Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. Increasing the robustness of dnns against image corruptions by playing the game of noise. arXiv preprint arXhv:2001.00057 2020.
Chubiao Song, Kun He, Jiadong Lin, Liwei Wang, and John E Hopcroft. Robust local features for improving the generalization of adversarial training. arXiv preprint arXiv: /tt1:::99991114444. 2019.

---

Under review as a conference paper at ICLR 2021
Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin- Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy?-a a comprehensive study on the robustness of 18 deep image classification models. In Proceedings of the European Conference Computer Vision (ECCV), pp. 631-648, 2018.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. arXiv preprint a8Xiv: 12152, 2018.
Welinder, Branson, Mita, C. Wah, Schroff, Belongie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-00 California Institute of Technology, 2010.
Y. Xian, C. H. Lampert, B. Schiele, and Z. Akata. Zero-shot learning comprehensive evalu- ation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2251-252 2265, 2019.
Hongyang Zhang. Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent Ghaoui, and Michael I Jordan. Theorctically principled trade-off between robustness and accuracy. arXiv preprint 191.0079, 2019a.
Huan Zhang, Hongge Chen, Zhao Song, Duane Boning, Inderjit Dhillon, and Cho-Jui Hsieh. The limitations of adversarial training and the blind-spot attack. arXiv preprint 1rXiv:1901.044 2019b.
Tianyuan Zhang and Zhanxing Zhu, Interpreting adversarially trained convolutional neural net- works. arXiv preprint arXiv: 1905 2019.
Bo Zhao, Yanwei Fu, Rui Liang, Jiahong Wu, Yonggang Wang, and Yizhou Wang. large-scale attribute dataset for zero-shot learning, arXiv preprint arXiv:1804.0433 2018.
10

---

Under review as a conference paper at ICLR 2021
Accuracy of'a Classifier
A Accuracy of Classifier
-5-
-10
15
-15
Elastic tested on Clean
Occlusion tested Clean
-20
-0
Gaussian Noise tested Clean
Wave ceste Clean
25
AT tested Clean, K=10
25
Saturation tested
tested on Clean, K-5
tested on
-30
-30
CUB
StanfordCars
AWA
LAD
CIFAR-10
StanfordCars
LAD
CIFAR-10
(a) Evaluating adversarial training clean im-
(b) Evaluating natural perturbations training
ages.
clean images.
Figure 6: Comparing the performance of adversarial training with natural perturbations training on clean images.
A APPENDIX
A.1 NATURAL. PERTURBATIONS TRAINING ALGORITHM
Algorithm Natural Perturbations-based Training for Robustification.
1: Given S = (xx, (xr E 1R2,,x E NJ. Learning rate n). A set of natural perturbations Ç*.
2: Initialize 0 randomly
3: for epoch = 1 to Nep do
4: for minibatch C [S]
5: =. =(==(((xxm) Un.
6:
epoch > delay then
7:
CS [(f(s) (zn)), Vn.
C t2+ 2
8:
9:
end if
10:
Update 0 with SGD.
11: 0 == V LS
12: end for
13:
end for
A.2 EVALUATION FOR VARYING PARAMETERS OF ADVERSARIAL PERTURBATIONS.
In this section we present the results for adversarial perturbations generated with K 5 and adjusted such that the same drop of 10% is retained for the fair comparison among perturbations.
Evaluating Robustified Networks on Clean Images. In Figure 6a we observe that by varying the number of steps required to generate adversarial examples from K = 10 to K 5 and the perturbation size while keeping drop same, the performance of the adversarially trained network does not vary significantly on clean images. By comparing the plot for K = 5 in the Figure with the plots the Figure/bb) we learn that, training with natural perturbations provides better recovery in the drop of performance
the clean images better than the network adversarially trained with K = 5. Hence, our results show that, by varying the number of steps and perturbation level for generating adversarial examples while maintaining the drop, the behavior of an adversarially trained network on the clean images does not change significantly.
Evaluating Robustified Networks Seen Perturbations. Figure 7a shows the results for two ad- versarially trained networks on adversarial examples with different parameter configurations while keeping the drop to 10%. We observe that with the change in the parameters of adversarial training

---

Under review as a conference paper at ICLR 2021
5
5
A Accuracy of Classifier
A Accuracy of Classifier
-5
-10-
-10
-15
-15-
Elastic tested on Elastic
Occ usion tested on Occlusion
-20
20
Gaussian Noise tested on Gaussian Noise
Wave tested on Wave
25
tested on Adversarial. K=10
25
Saturation tested Saturation
tested on Adversarial. K=5
Gaussian Blur tested Gauss Blur
30
-30
CUB StanfordCars
AWA
LAD
CIFAR-10
CUB
StanfordCars
AWA
LAD
CIFAR-10
(a) Evaluating adversarial training on the seen perturbations.
(b) Evaluating natural perturbations training on seen perturbations.
Figure 7: Comparing the performance of adversarial training with natural perturbations training on seen perturbations.
the performance on the adversarial examples does not vary significantly. Only for CUB dataset the network with number of steps K 10 performed better than K 5. The contrast between the plots in Figure 7a and Figure depicts that training with natural perturbations transfers to natural better than adversarial training adversarial perturbations. Therefore, learn that by varying the number of steps and perturbation level for generating adversarial examples while maintaining the drop, the behavior of an adversarially trained network on the adversarial images does not change significantly.
Evaluating Robustified Networks Unseen Perturbations. Figure two adversarially trained networks with different parameter configurations on unseen perturbations. Figure (right) shows the plots for networks trained on natural perturbations and tested on unseen perturbations. Each subplot on the Figure (right) shows network trained on different type of natural perturbation and tested on the unseen perturbations.
(left) shows the results for
By contrasting yellow line plots (for K 10) with brown line plots (for K = 5) in Figures 8a 8c 8e we observe that the performance of adversarially trained networks with K = 5 and K unseen natural perturbations does not vary significantly. We notice the difference in performance only on CUB dataset among two networks. An adversarially trained network with K for CUB dataset shows better recovery against elastic perturbations than K 10 network. However, it shows worst performance against Gaussian noise, wave and saturation on the CUB dataset.
on
The line plots in each subplot in Figures. 8b 8d with symbols "plus" and "hexagon" show the performance of naturally trained networks on adversarial perturbations with K 10 and K respectively. By contrasting their performances in each subplot in Figure & (right) observe that the recovery with the natural perturbations is almost the same except some differences on the CUB dataset. We observe that for the CUB dataset all the natural perturbations trained networks recovered the drop against adversarial examples with K = 10 steps better than K = 5. Thus, our analysis on unseen perturbations show that with the change the parameters of adversarial perturbations while keeping the drop same overall there is no significant change in the performance of networks both adversarially trained as well as natural perturbations trained. However, for fine grained CUB dataset we observed that an adversarially trained network with K 10 is better at unseen robustification than K = 5. On the other hand, natural perturbations trained networks are better at recovery against K 10 adversarial perturbations on the CUB dataset.
12

---

Under review as a conference paper at ICLR 2021
Accuracy of Clussitier
Accursey of Classifferr
Classifeer
lested Bartic C-5
Al tested Occlusion on Ki-5
-15
Occisiin Advessrid,
Dablt las an Alhenattal Kea Betted Cossiton
te52s Adkorcaao Keld
20-
-0
Causuaa
-5
Seturntion
Ocousen rested in Gosotion Bar
baaed Counslan
-10
G3
SteffrnCars M6A
LAO
Startordaars
LAD
1390110
(a) Evaluating adversarial train-
(b) Evaluating natural perturbations training on unseen perturbal ba-
ing the elastic and occlusion perturbations.
tions.
Accuracy of Clussiter
XT Histed un Wevee X-10
Geseviinn
Accuracy of Classifee
Accurney of' Clossiferr
AT unsted Gazzian Naiaw, K-5
K-5
Weoe baltd en Adversani K-10
-15
-15
Wioe listed on Adversaww, K-5
Wave bisted on Elasic
-20-
25
20
Wive bested Satur ston
Wave bisted on Coussion Ricke
Wore booled in Ezciankn
-20
25
Danis en Notee botted on Dccissie
Days Netse Elsiic
Dazaa en
Wave tested on Couesion
Dazza Nobs
Birr
CUB
StanbrrCars AOA
LAO
QHARCO
-30
0LB
StanfbrsCars
JMA
LAD
CIAR-10
-10
GJR
Parferstars
LAO
CPA8-10
(c) Evaluating adversarial train- ing the Gaussian noise and perturbations.
(d) Evaluating natural perturbations training on unseen perturba-
tions.
Avcuracy of CCussiffer
Accuracy of Classifeer
Clossifier
-10
-35
Seturation
Saturstion tested on Aleert sal Ke5
Coutdat B treted on Advertataa, 8-5
Gonniat lasted Advessrtid, E-10
20
Insted Savedan Bizr K-10
Insted on Seturation 19
20
Selaretonn boted Blrstt
Occiaton
GundaT B tested on Eusti:
25.
tested Saturation
baated Saturaiinn
Basssian Ke5
Selsratien
Councet D nasted on Mase
CUB
SantbroCars JiA
LAD
0FA-10
018
Sastackaas
AMA
140
StavbrdCaas
Mil
LAO
CHA611
(c) Evaluating adversarial train-
(f) Evaluating natural perturbations training on unseen perturba- tions.
ing the saturation and Gaus- sian blur perturbations.
Figure 8: Comparing the performance of adversarial training unseen perturbations with natural perturbations training on unseen perturbations.
13


Reviews:
[{"review_id": "s9OSlXigbc5", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "#### Summary\n\nIn this paper, the authors evaluate the performance of classifiers trained and then later tested on both adversarially generated perturbations as well as more natural perturbations.  By considering six different natural perturbations, they show empirically that natural perturbations can improve performance against clean and adversarially-perturbed images.  They also show that adversarial training does not improve performance on unseen natural transformations.\n\n#### Strengths\n\n- This paper studies an interesting problem.  Natural perturbations have gained increasing interest from the robustness community recently.  While there are many works that study the trade-offs (e.g. in the test accuracy of clean and adversarialy perturbed data) with respect to adversarial perturbations, these trade-offs are much less well understood for more \"natural\" notions of robustness.\n\n- The performance normalization step here is interesting, and in my opinion is an useful way to compare notions of robustness across domains.  As this problem receives more attention, I believe that this will be a useful metric for comparing results for the community.\n\n- The result that (in some cases) adversarial training hurts natural robustness and natural robustness helps adversarial robustness is interesting and noteworthy.  A more in-depth analysis of these phenomena and whether they hold for broader classes of natural transformations would be quite interesting.\n\n#### Weaknesses\n\n- Although the title advertises that this paper considers \"natural\" perturbations of data, it is arguable as to whether these transformations are representative of transformations that are more likely to be \"found in the real world\" (quoted from the Introduction).  Although it seems true that adversarial perturbations may not occur naturally, I'm not entirely convinced that these transformations are any more likely to be encountered.  Moreover, based on Figure 1, the elastic, wave, gaussian noise, and gaussian blur transforms all look quite similar to one another.  A more convincing argument could be made here if a more diverse set of realistic perturbations was considered  (e.g. changing the weather conditions in iamges).\n\n- The notation throughout this paper is at times confusing.  I bullet the instances where I felt the notation was unclear:\n\n   + it seems unintuitive as to why $\\zeta_n^A$ and $\\zeta_n^t$ need to depend on $n$.\n   + The authors write that each image $x_n$ is an element of $\\mathbb{R}^2$ -- I assume this is a typo.  \n   + The symbol $\\circledast$ is undefined -- is this a pointwise convolution?  \n   + The constant $\\alpha$ is not defined in the definition of $\\zeta^O$, and the notation $b^{x_c, t,r}$ is confusing -- I'm still unsure of how $\\zeta^O = \\min(x_n, b^{x_c,t,r})$ would be applied in practice.  What is the size of $b$?  \n   + The notation $x_n^{N(\\mu,\\sigma^2)}$ is confusing -- I assume this means that $x_n \\sim N(\\mu,\\sigma^2)$?  \n   + The \"shift operator\" in the definition of the wave transformation is also undefined.  \n   + In the definition of the accuracy drop, the symbol $\\alpha$ is reused; previously, it was also used to denote parts of $\\zeta_S$ and $\\zeta^E$.  To this end, the definition of $\\alpha$ here is confusing -- shouldn't it be defined in terms of the cardinality of these sets?\n   + Why is the cross entropy loss indexed by lower-case $s$, which seems to be undefined?\n\n- The figures are quite hard to parse.  In particular, although the message of Fig. 2 is clear from context, the symbols all overlap, which when we look at later figures makes hard to interpret.  Also, it's unclear why all the lines are connected, as each data point on the line simply corresponds to a different dataset.\n\n- In some sense, the fact that natural transformations improve clean accuracy is already known.  The training scheme that is introduced for training using natural perturbations is simply data-augmentation.  And many works have previously shown that data-augmentation (e.g. [1], [2], [3]) improves clean accuracy.  \n\n- I'm a bit surprised that adversarial training does not decrease the performance on clean images (e.g. Fig 3a) for CIFAR-10.  Unless I have misunderstood, this is at odds with other works (e.g. [4]) that claim that robustness is at odds with accuracy.\n\n- I'm not sure how to interpret Fig 5.  In particular, it seems that in some cases, training with data augmentation improves performance and in others, performance suffers when evaluating on unseen transformations.  This makes the claim of the paper feel somewhat weaker, given that the message emphasized in the intro only holds for a subset of the combinations of training/testing transformations.\n\n#### Final thoughts\n\nWhile I think that this paper studies an interesting problem, and has some interesting ideas such as normalizing the accuracy drop to create a fairer comparison, I feel that the experiments don't fully support the claimed conclusion.  Specifically, the phenomena of interest, such as that data augmentation improves performance on unseen natural transformations and on adversarially perturbed inputs, seems to not hold in general; on the contrary, it appears that a non-negligible amount of the time, it also hurts performance.  Further, the presentation is hard to follow at times due to grammatical mistakes and undefined notation.  Also, the natural transformations under consideration are all quite similar -- it would be very interesting to see if these kinds of results would hold with respect to much more severe changes in images, such as adding snow or changing the background colors.  For these reasons, I am leaning toward suggesting rejection for this paper. \n\n#### References\n\n[1] https://link.springer.com/article/10.1186/s40537-019-0197-0\n[2] https://ieeexplore.ieee.org/document/8388338\n[3] https://arxiv.org/pdf/1708.04896.pdf\n[4] https://arxiv.org/pdf/1805.12152.pdf", "extra_scores": {}, "extra_text": {}}, {"review_id": "PvrknKRASrB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper summary\nThis paper studies the degradation in performance of deep learning models under both adversarial as well as natural perturbations or contamination. It proposes to study and compare their effects, as well as their potential use for defenses across different scenarios.\n\nReview Summary\nTo this reviewer, the questions that these papers asks are somewhat interesting as the understanding of the robustness of machine learning models in general is an important questions. However, I believe the reported results are naturally expected and the paper is hard to read.\n\nDetail Review\n\nStrengths:\n- This paper studies a relevant problem\n- There is a large empirical study of different training and testing settings.\n\nWeaknesses\n- The driving question of this paper seems to be whether natural perturbations hurt generalization as much as adversarial examples. The authors answer this question in the negative, but this is naturally expected: the perturbations that the authors study yield images that are likely in the distribution of natural images. As a result, rather than perturbations, they could well be considered as data augmentation. \n- On section 3 (introducing the updates in Eq (1&2)) the authors write \"the equation to be optimized is given as\". However, what follows is an equation, and not any optimization problem to be optimized. \n- The authors stress that natural perturbations are designed with care, so that they induce a control loss in accuracy while being similar enough to the original samples so that they are recognizable by humans. How is this verified? The occlusions are drawn at random, so could it not be the case that the main object in the image is occluded?\n- The expression for the perturbed samples with Gaussian noise makes no sense to me. I believe this should read $\\zeta(x_n) = x_n + v$, $v\\sim N(\\mu,\\sigma^2)$.\n- It is not always clear if the reported results are over training or testing samples. For example, at the end of Sec 4.3, the authors comment \"robustification with natural perturbations outperform robustification with adversarial perturbation on the seen test set\". Some other times, the authors refer to \"the unseen test set\". What does it mean that the test set is 'seen' or 'unseen'? \n- To make comparisons fair, the authors calibrate the energy of all perturbations. However, the definition of this quantity $\\alpha$ simply measures whether there exist a perturbation on $f(x_n)$ or not, regardless of whether $f(x_n)$ was a correct prediction. In other words, are the authors assuming a perfect model?\n- The plots are a bit hard to interpret, as one needs to distinguish between overlapping symbols of the same color (see e.g. Fig 5.b).\n- In defining the evaluation metric, $\\Delta \\mathcal A$, the authors must have meant the average of the reported quantity over the samples.\n- There authors refer to Carlini & Wagner (2017) as proposing adversarial training, in the beginning of section 2. I believe this is incorrect, and that they must have meant to cite Goodfellow et al. (2014) or Madry et al (2017)\n- The paper is hard to read and English usage should be improved. I read the paper a few times, and I'm hoping I understood the authors claims and results correctly (which I believe I did).", "extra_scores": {}, "extra_text": {}}, {"review_id": "CvfxhWk84E", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the effect of “robustification” (i.e., adversarial training or data augmentation) of models on the accuracy to seen and unseen perturbations. The authors propose a technique to “standardize” the robustification process across different perturbations. They evaluate their approach on several datasets, highlighting how standardization yields different insights compared to prior work.\n\nPros\n- Standardization is an interesting idea\n\nCons\n- Unclear contributions, and generally poor clarity and presentation\n- No justification of standardization technique\n- Experiments do not study dependence on standardization level\n\nIt is unclear exactly what contributions this paper is making. First, as the authors acknowledge, there have been several recent works studying robustness to natural vs. adversarial perturbations. For completeness, one the authors miss is the following:\n\nTaori et al., Measuring Robustness to Natural Distribution Shifts in Image Classification. On arXiv.\n\nIn more detail, the authors study “robustification” of the DNNs  on adversarial vs. natural examples. For natural examples, this is really just standard data augmentation, whereas for adversarial examples it is adversarial training.\n\nIt appears that the main difference compared to prior work, Laugros et al 2019, is that the authors “standardize” the robustification by capping the permitted decrease in robustness. This is an interesting concept, though the authors need to do a significantly better job explaining and justifying it. For instance, the authors claim that their results differ from Laugros et al 2019 due to standardization, but it is not clear why standardization would cause these differences.\n\nMore generally, it is not really justified why standardization should be important. Intuitively, it makes sense to me that different kinds of perturbations may not be directly comparable due to different levels of shifts. However, there is no reason why standardization is the right way to normalize them. For instance, an alternative would be to tune the level of the shift based on the drop in accuracy of the *unrobustified* network. Why should the authors’ standardization technique better?\n\nEqually importantly, the authors do not study the choice of different standardization levels on their results. Given the apparent importance of standardization to the authors’ contributions, they should justify that their results are not dependent on specific hyperparrameter choices (or at least study how the hyperparameter of standardization level affects their results).\n\nAs written, it is not clear what meaningful insights the authors are studying. For example, they find that “robustification with natural perturbations outperform robustification with adversarial perturbations on the seen test set”, but this fact is unsurprising given that natural perturbations are (presumably) naturally present in the training data to some degree (though, depending on the transformation, the term “natural” is debatable).\n\nFinally, the figures are not well explained very well, making it even harder to understand exactly what the authors are trying to convey. For instance, the authors should define give precise definitions of “\\Delta Accuracy of a Classifier”, and preferably give a reminder in the captions. In addition, not all the curves in the figures are labeled. Also, it is unclear why the authors are using a line plot, which is confusing). \n\nIn summary, I think this paper has some interesting ideas, but it is far from ready for publication and the author need to spend significant effort clarifying their contributions and improving the clarify of their paper.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers indicated a number of concerns (which I agree with) which have not been addressed by the authors as they have not provided any response.  Indeed, the paper would be significantly improved once these issues are addressed. 
Decision: Reject