Pareto Adversarial Robustness: Balancing Spatial Robustness and Sensitivity-based Robustness
Authors: Ke Sun
Mingjie Li
Zhouchen Lin
Venue:  (2021)

Abstract: Adversarial robustness, mainly including sensitivity-based robustness and spatial robustness, plays an integral part in the robust generalization. In this paper, we endeavor to design strategies to achieve comprehensive adversarial robustness. To hit this target, firstly we investigate the less-studied spatial robustness and then integrate existing spatial robustness methods by incorporating both local and global spatial vulnerability into one spatial attack design. Based on this exploration, we further present a comprehensive relationship between natural accuracy, sensitivity-based and different spatial robustness, supported by the strong evidence from the perspective of representation. More importantly, in order to balance these mutual impact within different robustness into one unified framework, we incorporate the Pareto criterion into the adversarial robustness analysis, yielding a novel strategy towards comprehensive robustness called \textit{Pareto Adversarial Training}. The resulting Pareto front, the set of optimal solutions, provides the set of optimal balance among natural accuracy and different adversarial robustness, shedding light on solutions towards comprehensive robustness in the future. To the best of our knowledge, we are the first to consider comprehensive robustness via the multi-objective optimization.

Under review as a conference paper at ICLR 2021
PARETO ADVERSARIAL ROBUSTNESS: BALANCING SPATIAL ROBUSTNESS AND SENSITIVITY-BASED Ro- BUSTNESS
Anonymous authors Paper under double-bling review
ABSTRACT
Adversarial robustness, mainly including sensitivity-based robustness and spa- tial robustness, plays an integral part the robust generalization. In this paper, we endeavor to design strategies to achieve comprehensive adversarial robust- ness. To hit this target, firstly we investigate the less-studied spatial robustness and then integrate existing spatial robustness methods by incorporating both lo- cal and global spatial vulnerability into one spatial attack design. Based on this exploration, further present comprehensive relationship between natural ac- curacy, sensitivity-basee and different spatial robustness, supported by the strong evidence from the perspective of representation. More importantly, in order to bal- ance these mutual impact within different robustness into one unified framework, we incorporate the Pareto criterion into the adversarial robustness analysis, yield- ing novel strategy towards comprehensive robustness called Pareto Adversarial Training. The resulting Pareto front, the set optimal solutions, provides the set of optimal balance among natural accuracy and different adversarial robustness, shedding light on solutions towards comprehensive robustness in the future. To the best of our knowledge, are the first to consider comprehensive robustness via the multi-objective optimization.
INTRODUCTION
Robust generalization can serve as an extension of tradition generalization, i.c., Empirical Risk Minimization in the case of i.i.d. data (Vapnik & Chervonenkis, 2015), where the test environments might differ slightly or dramatically from the training environment (Krueger et al., (020)) Improving the robustness of deep neural networks has been one of the crucial research topics, with various different threads of research, including adversarial robustness (Goodfellow et al., 2014 Szegedy jet al. 2013). non-adversarial robustness (Hendrycks Dietterich|
et al. 2019), Bayesian deep learning (Neal 2012. Gal 2016) and causality (Arjovsky et al.) 20199). In this paper, focus on the adversarial robustness where adversarial examples are carefully manipulated by human to drastically fool the machine learning models, e.g., deep neural networks, posing a serious threat especially on safety-critical applications. Currently, adversarial training (Goodfellow et al.) 2014]] Madry et al. [2017] Ding al. 20188) is regarded as one promising and widely accepted strategy to address this issue.
However, similar to Out-of-Distribution (OoD) robustness, one crucial issue is that adversarial bustness also has many aspects (Hendrycks et al. 2020), mainly including sensitivity-basee robust- ness (Tramer et al. 2020). i.c. robustness against pixel-wise perturbations (normally within the constraints of an mations. Firstly, y ball), and spatial robustness, i.c.. robustness against multiple spatial transfor- the computer vision and graphics literature. there are two main factors that
determine the appearance of pictured object (Xiao et al.] 2018 [Szeliski] 2010): (1) the lighting and materials, and (2) geometry. Most previous adversarial robustness focus on the (1) factor (Xiao) et al. 2018) 8) based pixel-wise perturbations, e.g., Projected Gradient Descent (PGD) attacks, suming the underlying geometry stays the same after the adversarial perturbation. The other rising research branch tackled with the second factor, such as based (Xiao et al.) 2018) and Rotation- Translation (RT)- based attacks (Engstrom et al., 2017]] 2019). Secondly, by explicitly exploring the human perception, /hharif et al. 2018) pointed out that sensitivity-basee based robustness, i.e., ly-distance

---

Under review as a conference paper at ICLR 2021
measured robustness, is not sufficient to adversarial robustness in order to maintain the perceptual similarity. This is owing to the fact that although spatial attacks or geometric transformations also result in small perceptual differences, they yield large I, distances.
In order to head towards the comprehensive adversarial robustness, we find that the crucial issue to investigate the aforementioned whole part of adversarial robustness is the relationships among accuracy, sensitivity-basee robustness and spatial robustness. Prior to our work, a clear trade-off between sensitivity-baser robustness and accuracy has been revealed by series of works (Zhang et al. 2019; Tippas et al. 2018 Raghunathan et al. 2020). Besides, recent work (Tramèr & Boneh, Kamath al. 2020) exhibited that there seems to exist an obscure trade-off between Rotation- Translation and sensitivity- based robustness. However, this conclusion lacks considering Flow- based attacks (Xiao et al., 2018; Zhang & Wang 2019), another non-negligible part in the spatial robustness evaluation, making the previous conclusion less comprehensive or reliable. As such, the comprehensive relationships among all the quantities mentioned above are still unclear and remain to be further explored. More importantly, new robust strategy that can harmonize all the considered correlations needed, in order to achieve optimal balance within the comprehensive robustness.
2019
In this paper, in order to design new approach towards comprehensive robustness, we firstly ex- plore the two main branches in the spatial robustness, i.c. flow-based spatial attack (Xiao et al.) 2018) and Rotation-Trannn Translation (RT) attack (Engstrom al. 2019). By investigating the different impacts of these two attacks on the spatial sensitivity, we propose an integrated differentiable spatial attack framework, considering both local and global spatial vulnerability. Based on that, we present a comprehensive relationship among accuracy, sensitivity- tial robustness. Especially we show that the trade-off between sensitivity-based and RT robustness is fundamental trade-off as opposed to the highly interwoven correlation between sensitivity-basee and Flow-based spatial robustness. We further provide strong evidence based on their different saliency maps from the perspectives of shape-bias, sparse or dense representation. Lastly, to balance these different kinds of mutual impacts within a unified adversarial training framework, we introduce the Pareto criterion (Kim & De Weck 2005 2006 Zeleny, 2012) in the multi-objective optimization, thus developing an optimal balance between the interplay of natural accuracy and different adversar- ial robustness. By additionally incorporating the two-moment term capturing the interaction between losses of accuracy and different robustness, we finally propose a bi-level optimization framework called Pareto Adversarial Training. The resulting Pareto front provides the set of optimal solutions that balance perfectly all the considered relationships, outperforming other existing strategies. Our contributions are summarized as follows:
robustness and two branches of spa-
We propose an integrated spatial attack framework that incorporates both local and global spatial vulnerability based on Flow-based and RT attacks, paving the way towards the com- prehensive spatial robustness analysis the future.
We present comprehensive relationships within accuracy, sensitivity- based, different spatial robustness, supported by strong evidence from the perspective of representation.
We incorporate the Pareto criterion into adversarial robustness analysis, and are the first attempt to consider multiple adversarial robustness via the multi-objective optimization.
2 TOWARDS COMPREHENSIVE SPATIAL ROBUSTNESS
2.1 MOTIVATION
In order to better investigate the relationships between accuracy and different kinds of adversarial robustness, we need firstly provide a fine-grained understanding of spatial robustness, which has been less studied as opposed to sensitivity-baat
robustness. We summarize two major branches among a flurry of related work about spatial robustness (Engstrom et al. 2017 2019; Xiao et al., 2018| Zhang & Wang 2019 Tramer& & Boneh 2019 Kamath et al. 2020): (1) Flow- based Attacks, and (2) Rotation- Translation (RT) Attacks. Specifically, we find that the former mainly focuses on the local spatial vulnerability while the latter tends to capture the global spatial sensitivity. Our
motivation is to firstly shed light on the fundamental difference between these two approaches, and then propose integrated spatial robustness evaluation metric.
2

---

Under review as a conference paper at ICLR 2021
MNIST
CIFAR-10
Caltech-256
Benign Examples
2
0
4
1
4
PGD Attack
2.6
6
9
7
3
Flow-based Attack
2_
1100
6 4 5
4
L
1
14
RT Attack
6
4
4
Integrated Spatial Attack
Figure Visualization of Flow-based, RT and Our Integrated Spatial adversarial examples on MNIST, CIFAR-10 and Caltech-256. More discussions can refer to Appendix A.
2.2 INTEGRATED SPATIAL ATTACK: COMBING LOCAL AND GLOBAL SPATIAL SENSITIVITY
Local Spatial Robustness: Flow-based Attacks The most representative Flow-based Attack is Spatial Transformed Attack (Xiao et al. 2018), in which a differentiable flow vector wF = (Ap, Ae) defined in the 2D coordinate (g.v) v) to craft adversarial spatial transformation. The vanilla targeted Flow-based attack (Xiao et al. 2018 (N 0) follows the optimization manner.
WF arg min max (z(ww)) [o + T (wo wp 1et
where fo(x) - (f (x:) is the classifier in the K-classification problem. Twp is Flow- based adversarial example parameterized by flow vector wy. Lilow measures the local smoothness of spatial transformation further balanced by т.
Interestingly, in empirical study shown in the left part of Figure I find that Flow-based attack tends to yield local permutations among pixels in some specific regions irrespective of the option of т, rather than the global spatial transformation based on their shapes. We analyze that this phenomenon is owing to two factors: 1) Local permutations, especially in regions where colors of pixels change dramatically, are already sufficiently sensitive manipulate, demonstrated by our empirical results shown above. 2) The optimization manner does not incorporate any sort of shape transformation information, e.g., parameter equation of rotation, as opposed to vanilla Rotation- Translation attack, which we present in the following. Thus, Flow-based attacks tend to capture the local spatial vulnerability. Further, for the need to design the integrated spatial attack, we transform Eq into its untargeted version under cross entropy loss with flow vector bounded by ep-ball:
w arg max L (Fwp. v) s.t. || EF
where CC (2, y) = log E, exp (f()) fx)). One difference compared with Eq. is that we replace local smoothness term Lyoow with our familiar 4p constraint. Moreover, vanilla Flow- based attack (Xiao al. 2018) follows the max operation suggested in (Carlini & Wagner 2017) However we leverage cross entropy loss instead in pursuit a uniform optimization form in our integrated spatial attack. Proposition reveals the correlation between the two loss, indicating that the smooth approximation version of mar operation in Eq. denoted as Cg has a parallel updating direction with Cross Entropy loss regarding Wy. Proof can be found in Appendix [A.2]
Proposition For a fixed (Iwe, y)) and 0 consider L (x. y) = log Eizy exp (f)(z)) - (x)) the smooth version loss of Eq. without local smoothness term, then we have
V we Lg (Awr y) r(E 8) wp Co L5 (xwps y), where r(Zur- v) Eiet exp (fō(Tur)) (3)
Ex cxp (f((zwr))
Global Spatial Robustness:
(RT)-based Attacks The original Rotation-
Translation attack (Engstrom al. 2017 2019) applies parameter equations constraint the 2D coordinate, thus capturing the global spatial information:
u' cos 0 -sin 0
8u
+
(4)
sin 0
cos
5u
3

---

Under review as a conference paper at ICLR 2021
To design generic spatial transformation matrix that simultaneously consider rotation, trans- lation, cropping and scaling, we re- re-parameterize the transform matrix äs a generic 6-dimensional affine transformation matrix, inspired by (Jaderberg et al. 2015):
-
1
0 0
0
+
w
gT
11
w
u 1
(5)
RT
where we denote Aeer as the generic 6-dimensional affine transformation matrix, in which each war indicates the increment on different spatial aspects. For example, (w RT. wT) determines the translation. Finally, the optimization form of the resulting generic and differentiable RT-based attack bounded by cur-ball exhibited as:
w WiT
arg max Cy (IWAe 3) s.t. |RRTI| S ERT.
(6)
wAT
Integrated Spatial Robustness The key to achieve this goal to design an integrated parameter- ized sampling grid Twar.wr (G) that can warp the regular grid with both affine and flow transfor- mation, where G is the generated grid. We show our integrated approach as follows:
T Tugr.wr wy (G) A Aurr
V
WF
ade
Tuggr.wy
(G) xr.
Then sample new addv by T TWRT we (G) via differentiable bilinear interpolation (Jaderberg al, 2015). Then the loss function of the differentiable integrated spatial attack can be presented as:
w arg max Lg (ar + Yau v), s.l. ||=]|| S
where [wy, Warl" and The is the crafted integrated spatial perturbation. Note that The itself does not necessarily satisfy the lp constraint directly. For the implementation, we follow the PGD attacks. We consider the
procedure (Madry et al. (2017), common practice in sensitivity-basse infinity norm w and different learning rates for the two sorts of spatial robustness:
W c+1
++1
w
+
aF aRr
clip, (sign( in ())))
w nT
w
RT
(9)
x I+1
= Tuttt (G) o x
where denote r0+1 [m [ and E From Figure we observe that our Integrated Spatial Attack can construct both local and global spatial transformations on images. Then, visualize the loss surface under this Integrated Spatial Attack leveraging "filter normalization" (Li et al.) 2018)) as illustrated in Figure 2 It is worth noting that the highly non- concave loss landscape with respect to only rotation and translation raised by (Engstrom al. 2019) has been largely alleviated by considering both local and global spatial vulnerability. verifying the efficiency of our Integrated Spatial Attack.
Figure 2: Loss landscape of Integrated Spatial Attack on CIFAR-10. (Left) A distant view of loss landscape w.r.t w before the optimization Eq. g (Middle) close view before the optimization that shows highly convex surface near the initialization point. (Right) The loss landscape around the maxima w after the optimization Eq. g More explanation can refer to Appendix A.3
4

---

Under review as a conference paper at ICLR 2021
3 RELATIONSHIPS BETWEEN SENSITIVITY AND SPATIAL ROBUSTNESS
3.1 RELATIONSHIPS
Based on the analysis above, next we focus on investigating the relationships between the strength of one specific robustness and other types of robustness. Firstly, empirically explore these relation- ships through conducting thorough experiments on MNIST, CIFAR-10 and Caltech-200 256 datasets. By adversarially training multiple PGD (sensitivity- robust models with different iteration steps, we further test their Flow-based and RT-based spatial robustness via methods proposed above.
MNIST
CIFAR-10
Caltech-256
Robust Text Accurncy (Sputial Robustoce 94
100
Robus Tost Accurazy (8patial Robusnes
Thx
Robat (S Robustece
50
27
80
50
E7
40
40
15 20 PGD Models
PGD Matek Robuizon)
PGD Modds Relostnas)
Figure 3: Relationship between sensitivity (PGD) robustness and two spatial robustness on three datasets. The X-axis represents the PGD-trained models under different PGD iterations while the Y- axis represents the robust accuracy on test data perturbed by Flow Attack (red) and RT Attack (blue).
As shown Figure 5. turns out that Flow-based spatial robustness (red) measured by its robust test accuracy presents steady ascending tendency across three datasets the PGD robustness increases, while the trend of RT-hased spatial robustness fluctuates conversely. It is worth noting that we test the accuracy on correctly classified data for the considered model for fair comparison. The trade-off' between sensitivity-based and RT-based spatial robustness is consistent with previous conclusion (Kamath et al., 2020; Tramer & Boneh, 2019), but it does not (even on the contrary) apply to Flow-based spatial robustness that delicately measures the local spatial sensitivity of an image. We provide the strong evidence from the perspective of representation in the next subsection.
3.2 EXPLANATION FROM THE VIEWPOINT OF SHAPE-BIA -BAS REPRESENTATION
We go first with our brief conclusion: the sensitivity-bass robustness corresponds to the shape-bias representation (Shi et al., 2020) Zhang & 7hu 2019), indicating that sensitivity- based robust models rely more on global shape when making decisions rather than local tex- ture. By contrast, the spatial robustness is associated with different representation strate- gies, serving as significant supplement toward the comprehensive robust representation. To demonstrate this conclusion, we visualize the saliency maps of naturally trained, PGD. Flow- based and RT adversarially trained models on some randomly selected images on Caltech-25k exhibited in Figure ! Specifically, visualizing the saliency maps aims at assigning a sensitivity value, sometimes also called "attribution", to show the sensitivity of the output to each pixel of input image. Following (Shi et al. 2020 Zhang & Zhu 2019), we leverage SmoothGrad ((x):
Natural
PCD AT
Flow AT
RTAT
Figure Saliency map of four types of train- ing models on some randomly selected images on Caltech-256.
(Smilkov et al. 2017) to calculate saliency map
5

---

Under review as a conference paper at ICLR 2021
0S (zi)
S(x) =
(10)
Oz;
Figure 4 manifests that PGD trained models tend to learn scarce and shape- biase- representation among all pixels of an image, while two types of spatially adversarially trained models suggest converse representation. In particular, the resulting representation from the Flow-based training model has the tendency towards a shape-biased one as places extreme values on the pixels around the shape of objects, e.g., the edge between the horse and the background shown in Flow AT in Figure 4 On the contrary, RT-based models have less reliance on the shape of objects, and at the meantime, the saliency values tend to be dense, scattering around more pixels of an image. Quantitatively, we calculate the distance saliency maps from different models across all test data on Caltech-256 dataset, and then compute their skewness shown in Figure 5
Modian Skewness Natuall Models
SkowiSND -Sael
SkoWiSILe Sta))
stoweSp -Suu)
Medium of Sk Skewnees vs PGD AT Model.
showlSyue Soppl
-22
$046
Madd
Madd
Midell
Made:
Mould
Modi
Modd
56661
3iddl
Figure 5: Median of skewness of saliency maps difference among robust models across all test data compared with other models. The first three sub-pictures are compared with the naturally trained model while the last one is compared with the PGD trained model.
Specifically, compute the pixel-wise distance between the saliency map of a robust model and that from a considered model, and then we calculate the median of skewness of the saliency map difference among all test data. Note that if two saliency maps have difference, then the difference values will be normal distribution with skewness 0. A negative skewness indicates that the original saliency map (representation) tends to be sparse compared with a considered model. We plot the tendency of skewness as the strength of some specific robustness increases shown in Figure 5 Based on the observations Figure 5) we summarize the following conclusions:
1) Based on the first and forth sub-picture picturrs, both PGD and Flow-based robust models tend to learn sparse and shape-biased representation compared with the natural model, but the Flow-based trained model is less sparse or shape-biased in comparison with the PGD trained one. 2) On the contrary, RT-based robust models have the trend to learn dense representation, which also intuitive as the RT trained model is expected memorize broader pixel locations to cope with potential rotation and transformation in the test data. The fundamental representation discrepancy of RT-based and sensitivity- robustness provides deep insights to explain why the trade-off of these two robust- ness occurs. In the Appendix A.4, we provide a sketch map that better illustrates their relationships.
PARETO ADVERSARIAL ROBUSTNESS WITH PARETO FRONT
4.1 MOTIVATION
Pareto Optimization. Based on the aforementioned analysis on the relationships between natural accuracy and different kinds of adversarial robustness, natural question how design training strategy that can perfectly balance their mutual impacts, which mainly sources from their different representation manners. In particular, in most cases their relationships reveal trade-off ones, except when the sensitivity robustness increases, Flow-based spatial robustness is enhanced. To better ad- dress these competing optimization objectives, we introduce Pareto optimization (Kim & De Week, 2005 Lin et al. 2019), and the resulting Pareto front, the set of Pareto optimal solutions, can offer valuable trade-off information between objectives. We provide more background information about Pareto optimization in Appendix A.5

---

Under review as a conference paper at ICLR 2021
Limitation of Existing Strategies. Given perturbation sets S = 1,.... m. and its corresponding adversarial risk Rud Sf: := (r.y))D [max,es, [(f( + r), y)], our goal is to find fo that can achieve the uniform risk minimization across all S; well as the minimal risk
Comparison with Single PGD
Max AT Robout Accuracy Dillornco?%)
-10
in the natural data. 1) Average adversarial training (Ave AT) (Tramèr & Boneh 2019), i... Rave (f: == B(aa))D m 2i=1 maxres. [[f(x + r). regards each adversarial robustness as the equal status. It may yield unsatisfactory solutions when the strength
-2
-30
-40-
-50
PSD Roisot, Accaaay Dillosanee
Floe Robust Assurcy Diftfrnnce
of different attacks mixed in training are not bal- anced, which we demonstrate in our experiments. 2) Max adversarial training (Max AT) (Tramer & Boneh, 2019, Maini et al. 2019), i.e., RRmx((::) := (<,,))> max; { { [[ + r), may overfit to specific type of adversarial robustness its adversarial attack used for training is too strong. Figure ( demon- strates that as the strength of PGD attack used in Max AT increases, the comprehensive robustness of Max AT degenerates to a single PGD adversarial training, owing to the fact that the PGD loss tends dominate among all losses. Appendix provides more details about Figure 6 and also introduces Proposition 2 to illuminate that Max AT also closely linked with specific weights of Ave AT.
660
Rebust Accuncy Difierezee
0.10
0.15
0.20
0.25
0.50
8 in PGD attacks Used for Max AT
Figure 6: The difference between the model trained by the PGD method and Max AT with different parameter for the PGD attack in the adversarial train- ing.
4.2 PARETO ADVERSARIAL ROBUSTNESS AND PARETO ADVERSARIAL TRAINING
The key to Pareto adversarial robustness is to find the optimal combination (trade-off in most cases) between natural accuracy, sensitivity-bat based and spatial robustness. Specifically, we hope to compute optimal a in the following formula:
min Le - Cro an LPGD + 022 Crow + C.3CRT, 0,a
(11)
where a = (as. 01, 022 az)) and Cg is the cross entropy loss based on the integrated framework we previously analyzed. Lnat; LPGD, Lrow and Let represent the natural loss, the PGD adversarial loss, the Flow-based and the RT- adversarial loss, respectively. Note that we additionally introduce natural loss guarantee a high-level natural accuracy (Raghunathan et al., 2020). However, direct joint minimization over Eq. 11 will degenerate to the trivial solution and the introduction of vali- dation dataset to tune a, e-g., DARTS (Liu al., 2018), is also computationally expensive for the adversarial training with multiple iterations. To avoid these, our approach is to introduce the Pareto criteria to choose optimal a to balance the mutual impacts between different adversarial robustness. Specifically, based on Eq 11 we additionally introduce the two-moment term regarding all losses into a bi-level optimization framework, in order to compute the optimal combination a during the whole training process. We name this bi-level optimization approach as Pareto Adversarial Training, and the lower-level optimization regarding a can be formulated as follows:
3 3
min Σ
Σ 1=0.==0
Ea az,,,)² s.t. E, rr, Σ a, a= a;
2
0, Vi 1,
(12) where Lo, L1, L2, La represent Luat LPGD, Lytow: SeT respectively for simplicity, sharing the same model parameter 0. r indicates the expectation of one-moment over all robust losses, i.e., spatial and sensirivity-based losses, which reflects the strength of comprehensive robustness we require after solving this quadratic optimization. In particular, given the fixed (C.) following the up- dating of 0 based on Eq. I1 the larger we require will push the resulting a, i 1, 2,2,3 3 larger as well, thus putting more weights on the robust losses while the whole process of Pareto Adver- sarial Training. For the understanding of the two-moment objective function, firstly we regard all losses as random variables with its stochasticity arising from the mini- batch sampling from data. The weighted quadratic difference is to measure the trade-off within natural accuracy and various robustness, and then the minimization is to alleviate this mutual trade-off under certain constraints. In addition, we leverage sliding windows technique to compute the expectation and CVXOPT tool to
7

---

Under review as a conference paper at ICLR 2021
Sacrificed Clean Accuracy (%)
MNIST
CIF CIFAR-I 10
Caltech-256
3.0
2.5
PCD
2.5
bpatad Al
Max
ATEAT
1.3
KODAT
1.5
SUGLAT
1.5
Avcht
Pacto hrel
0.3
Mare AT
1.0
-0.5
0.5
40
60
80
100
120
140
160
60
70
80 90
100
110
120
130
70
80
90
100
110
120
Score
Robustness Score
Figure 7: The Pareto front between the robustness score and sacrificed clean accuracy on MNIST, CIFAR-10 and Caltech-256. The vertical axis is the decrease of the natural accuracy compared with the naturally trained model and has been under the log transformation along two directions.
solve this quadratic optimization within each mini-batch training. Overall, for the upper level opti- mization in our bi-level Pareto adversarial training method, we leverage our familiar SGD method to update in Eq. [I with a calculated from the lower level problem. In the lower level procedure, solve the quadratic optimization regarding a to obtain the optimal combination among natural loss, sensitivity-basse -ased and spatial adversarial loss. We provide the proof about the quadratic formulation in Eq. 12 and our algorithm description in Appendix A.7 A.7
4.3 PARETO FRONT IN EMPIRICAL STUDY
By adjusting the upper bound of expected adversarial robustness loss, i.e., we can evenly generate Pareto optimal solutions where the obtained models will have different levels of robustness under op- timal combinations. The set of all Pareto optimal solutions then forms the Pareto front. Concretely, we train deep neural networks under different adversarial training strategies, and then evaluate their robustness by PGD. Flow-based and RT attacks, which proposed previously, under different it- cration steps. After equally averaging robust accuracy for each category of these attacks, we then compute the difference of robust accuracy between different training strategies and standard train- ing, attaining Robustness Score to evaluate the comprehensive robustness of all adversarial training strategies. Finally, we plot the Robustness Score and sacrificed clean accuracy of all methods across three datasets in Figure] 7 Experimental details can be found in Appendix A.8
The Pareto criterion (Appendix(A.5) exhibited in Figure 7 be interpreted that Pareto Adversarial Training can achieve the best comprehensive robustness compared with other training strategies, given a certain level of sacrificed clean accuracy we can tolerate. By adjusting the different levels of expected comprchensive robustness r in Pareto Adversarial Training, we can develop the set of Pareto optimal solutions, i.e., the Pareto front. manifests that all other methods are above our Pareto front, thus lacking effectiveness compared with our proposal. Overall, our proposed Pareto Adversarial Training develops an optimal (Pareto) criterion, by which we can maintain the optimal balance among the mutual impacts of natural accuracy and different robustness, based on the deep understanding of their relationships.
5 DISCUSSION AND CONCLUSION
The essential purpose of our work is to design novel approach towards comprehensive adversarial robustness. achieve this goal, we firstly analyze the two main branches of spatial robustness and then integrate them into one framework. Based on that, we further investigate the thorough relationships between sensitivity-based and two distinct spatial robustness from the perspective of representation. More importantly, having understanding the mutual impacts of different kinds of adversarial robustness, we introduce Pareto criterion into adversarial training framework, yielding the Pareto Adversarial Training. The resulting Pareto front provides optimal performance under the Pareto Criterion over existing baselines. In the future, we hope to apply Pareto analysis into more general Out-of-Distributio generalization settings.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907 0007000993,. 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Gavin Weiguang Ding. Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Max-margin ad- versarial (mma) training: Direct input space margin maximization through adversarial training. International Conference Learning Representations, ICLR 2020, 2018.
Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and translation suffice: Fooling cnns with simple transformations. arXiv preprint arXiv:1712.02799, 1(2):3, 2017.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Ex- ploring the landscape of spatial robustness. In International Conference on Machine Learning, pp. 1802-1811. 2019.
Yarin Gal. Uncertainty in deep learning. University of Cambridge, 1(3), 2016.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations, 2014.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. International Conference on Learning Representations, 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang. Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness; A critical analysis of -utto--distriuution generalization. arXiv preprint arXiv:2006. /6241, 2020.
Max Jaderberg, Karen Simonyan, Andrew Zisserman, et Spatial transformer networks. Ad- vances in neural information processing systems, pp. 2017-2025, 2015.
Sandesh Kamath, Amit Deshpande, and KV Subrahmanyam. Invariance vs. robustness of neural networks. arXiv preprint urXiv:2002.1 1318, 2020.
Yong Kim and OL De Week. Adaptive weighted sum method for multiobjective optimization: method for pareto front generation. Structural and multidisciplinary optimization, 31(2): 105-116, 2006.
Yong Kim and Oliver L De Weck. Adaptive weighted-sum method for bi-objective optimization: Pareto front generation. Structural and multidisciplinary optimization, 29(2): (99())149911)) 149-1114911... 158, 2005.
David Krueger, Ethan Caballero, Joem--Henik Jacobsen, Amy Zhang, Jonathan Binas, Remi Le Priol, and Aaron Courville. Out-of-distributton generalization via risk extrapolation (rex). arXiv preprint arXiv:2003.000 00688, 2020.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land- scape of neural nets. In Advances in Neural Information Processing Systems, pp. 6389-6399, 2018.
Xi Lin, Hui- Hui-Ling Zhen, Zhenhua Li, Qing-Fu Zhang, and Sam Kwong. Pareto multi-task learning. In Advances in Neural Information Processing Systems, pp. 12060-12070, 2019.
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806, 09055, 2018
Alcksander Madry, Alcksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant adversarial attacks. International Conference Learning Representations, ICLR 2018, 2017.
Pratyush Maini, Eric Wong, and J Zico Kolter. Adversarial robustness against the union multiple perturbation models. International Conference Machine Learning, 2019.

---

Under review as a conference paper at ICLR 2021
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang. Understand- ing and mitigating the tradeoff between robustness and accuracy. International Conference on Machine Learning, 2020.
Mahmood Sharif, Lujo Bauer, and Michael K Reiter. On the suitability of ip-norms for creating and preventing adversarial examples. In Proceedings the the IEEE Conference on Computer Vision and
Pattern Recognition Workshops, pp. 1605-1613, 2018.
Baifeng Shi, Dinghuai Zhang, Qi Dai, Zhanxing Zhu, Yadong Mu, and Jingdong Wang. Informative dropout for robust representation learning: A shape-bias perspective. International Conference Machine Learning. 2020.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint (0d6:/700.0338 2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint urXiv:1312.0 2013.
Richard Szeliski. Computer vision:
algorithms and applications. Springer Science Business
Media, 2010.
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems, pp. 5866-5876, 2019.
Florian Tramèr, Jens Behrmann, Nicholas Carlini, Nicolas Papernot, and Jörn-Henrik Jacobsen. Fundamental tradeoffs between invariance and sensitivity to adversarial perturbations. ICML, 2020.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry- Robustness may be at odds with accuracy. International Conference on Learning Representations, ICLR 2019, 2018.
Vladimir N Vapnik and A Ya Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. In Measures of complexity, pp. 11-30. Springer, 2015.
Chaowei Xiao, Jun Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially trans- formed adversarial examples. International Conference on Learning Representations, ICLR 2018, 2018.
Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier per- spective on model robustness in computer vision. In Advances in Neural Information Processing Systems, pp. 13276-13286, 2019.
Milan Zeleny. Multiple criteria decision making Kyoto 1975, volume 123. Springer Science & Business Media, 2012.
Haichao Zhang and Jianyu Wang. Joint adversarial training: Incorporating both spatial and pixel 2019.
attacks. arXiv preprint
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference Machine Learning, 2019.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural net- works. International Conference on Machine Learning, 2019.
10


Reviews:
[{"review_id": "zffltdlUk0x", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper first provides explanations to the inherent tradeoff between rotation adversarial attack and sensitivity attacks/spatial transform attacks, through their differences in saliency maps. Further, the authors proposed to utilize pareto training to find the best tradeoff among the four dimensions: natural accuracy, robustness against sensitivity/rotation/spatial transformation attacks. Experimental results show the proposed pareto adversarial training achieves better tradeoff between clean accuracy and adversarial robustness averaged across three types of attacks.\n\nPros:\n1. The idea of using pareto training to tradeoff between different accuracies/robustness is interesting. This reminds me of a very recent paper [1], where the authors considered a relevant (but different) problem: How to achieve in-situ tradeoff between different attacks during inference time.\n2. Using shape-bias and saliency maps to explain and demonstrate the inherent difference between rotation and sensitivity attacks is a good point.\n\nCons:\n1. Limited novelty. \nThe tradeoff between rotation and sensitivity attacks has already been discussed in many previous works (e.g., [2]); the rotation and spatial translation attacks are defined in previous works [3,4] and the combination of the two is kind of trivial; the authors directly apply pareto training [5] in the adversarial tradeoff task, without any modification to fit the new problem.\n2. Missing detailed experimental results.\nOnly the average robustness scores are provided in Figure 7. Please also consider providing robustness scores on each type of corruptions, so that we see which ones have larger improvements after pareto adversarial training and which ones do not.\n\n[1] Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free. NeurIPS, 2020.\n[2] Fundamental tradeoffs between invariance and sensitivity to adversarial perturbations. ICML, 2020.\n[3] A rotation and a translation suffice: Fooling cnns with simple transformations.\n[4] Spatially transformed adversarial examples. ICLR, 2018.\n[5] Pareto Multi-Task Learning. NeurIPS, 2019.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "tfi71eNGLNc", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work aims to study so called “comprehensive” robustness by considering natural accuracy, sensitivity-based robustness and spatial robustness simultaneously. The authors claim that they propose a “Pareto” Adversarial Training strategy to balance the mutual impact within different robustness and the resulting solutions provide the set of optimal balance among accuracy and different adversarial robustness.\n\nPros:\n- The paper investigates comprehensive robustness. For me, the problem itself is interesting.\n- This paper provides empirical evidence to show that there exists representation discrepancy for different notions of adversarial robustness. This shape-bias representation viewpoint is appealing.\n- The motivation for their training strategy incorporating “Pareto criteria” is good, despite that their method has a few questionable inaccuracies summarized below. \n\nCons:\n- The proposed “Pareto” Adversarial Training strategy is not well justified. First, the authors misuse Pareto. Pareto optimality (or front) is introduced to describe the trade-off between conflicting objectives from the multi-objective optimization. However, the proposed method is actually solving a series of bi-level optimization problems by varying the hyperparameter $r$. Since each problem is solved independently, it is not a multi-objective optimization, and the solutions obtained are also not Pareto optimal (or front). Second, in the lower-level formulation (12), the authors propose to use weighted quadratic difference to measure the trade-off. However, it is not clear why this makes sense. Is it because the two-moment objective function results in a quadratic programming problem which is easy to solve. In Section 3.2, the authors claim that there exists representation discrepancy for different adversarial robustness. Isn’t it more reasonable to consider the representation discrepancy as a measure? \n- To integrate flow-based and RT-based attacks, the authors propose to use equation (2) instead of the original Flow-based attack equation (1). To justify this, the authors then provide Proposition 1 which proves that the “smooth approximation” of max operation in equation (1) has a parallel updating direction with cross entropy loss in equation (2). However, equation (1) is to minimize the max operation over $\\omega_F$, whereas equation (2) maximizes the cross entropy loss over $\\omega_F$. Therefore, equation (2) is different from equation (1). Moreover, the integrated spatial attack proposed in Section 2 seems not to appear anymore in their next analysis and training strategy. The readers might think that Section 2 is a little unnecessary as it is unrelated to other parts of the paper.\n- In section 4.1, the authors claim that Max AT is closely linked with specific weights of Ave AT by providing Proposition 2. However, I have some concerns about the proof of Proposition 2. First, it assumes that KKT conditions hold. This assumption is strong in the sense that KKT conditions imply zero duality gap which usually does not hold for nonconvex problems. Second, even if KKT conditions hold, we can not derive that $R^*_{max}$ is a first-order stationary point of $\\sum \\lambda_iR^{S_i}$ because the Lagrangian function might be non-convex with resepct to $f$.\n\nMinor comments:\n- It would be better if the authors can give the definition of $x_{\\omega_F}$ in (1).\n- In equation (9), $clip_\\epsilon$ should be used after the update of $w_F^t$.\n- In equation (10), what does $n$ represent? The number of training data or the dimension of input?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "aHUIzYGTJ1L", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper first studies the tradeoffs between two forms of spatial robustness, including robustness against Flow-based spatial attack and Rotation-Translation (RT) attack. In particular, it proposes an approach to account for both local and global spatial transformations in an integrated framework. In addition, the paper investigates the relationship between the sensitivity-based (lp-norm based) and spatial robustness, and proposes a training method called ‘Pareto Adversarial Training’ to find optimal combination between natural accuracy, sensitivity-based and spatial robustness.\n\n\nPros:\n\n+ Investigating the spatial robustness along with sensitivity based robustness seems to be a natural step towards comprehensive robustness, which is well motivated. Existing works on spatial robustness are well-explained. \n\n+ Studying the saliency maps of different adversarially-trained models is interesting. Figure 4 visually looks good.\n \n+ Introducing the Pareto optimization to account for different robustness objectives is well-motivated.\n\n\n\nCons: \n\n- The experiments are the major weakness of this paper. In particular, Figure 3 is really hard to parse: (1) legends overlap with the presented curves; (2) Why is the spatial test accuracy against Flow Attack higher than clean accuracy for Caltech-256? (3) How do you set all the hyper-parameters for adversarial training and the attack strength, such as \\epsilon_F in Equation 2 and \\epsilon_RT in Equation 6?  (4) X-axis label says PGD Trained Models (PGD Robustness), whereas the X-axis tick represents the number of PGD attack iterations. \n\n- Figure 7 is not explained with enough details. The X-axis label ‘Robustness Score’ is not defined throughout the paper. The meaning of each abbreviated term appeared in the legend needs clarification. Instead of a single Robustness Score, I would recommend to include the sensitivity-based robustness, Flow-based spatial robustness and RT spatial robustness individually for each models.\n\n\nOther Questions and Comments:\n\n1. From Equation 11 to Equation 12, it is unclear to me why minimizing the two-moment term regarding all losses leads to the optimal \\alpha parameter. In addition, why do the constraints in Equation 12 not appear in Equation 11?\n\n2. The presentation of the experimental sections needs to be improved in order to reach the acceptance bar of ICLR. Specifically, the experimental settings of each presented figure should be presented with enough details. Evaluation metrics of the experiments should be defined clearly in the corresponding section. Discussions on the results should be thorough enough such that each figure can be understood clearly.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper aims to address the robustness issues by considering natural accuracy, sensitivity-based robustness and spatial robustness at the same. However, the reviewers pointed out that many things, like the expriment, the presentation, the algorithm, are not clear. In addition, the technique part is weak and below the bar of ICLR.
Decision: Reject