Improving Local Effectiveness for Global Robustness Training
Authors: JINGYUE LU
M. Pawan Kumar
Venue:  (2021)

Abstract: Despite its increasing popularity, deep neural networks are easily fooled. To alleviate this deficiency, researchers are actively developing new training strategies, which encourage models that are robust to small input perturbations. Several successful robust training methods have been proposed. However, many of them rely on strong adversaries, which can be prohibitively expensive to generate when the input dimension is high and the model structure is complicated. We adopt a new perspective on robustness and propose a novel training algorithm that allows a more effective use of adversaries. Our method improves the model robustness at each local ball centered around an adversary and then, by combining these local balls through a global term, achieves overall robustness. We demonstrate that, by maximizing the use of adversaries via focusing on local balls, we achieve high robust accuracy with weak adversaries. Specifically, our method reaches a similar robust accuracy level to the state of the art approaches trained on strong adversaries on MNIST, CIFAR-10 and CIFAR-100. As a result, the overall training time is reduced. Furthermore, when trained with strong adversaries, our method matches with the current state of the art on MNIST and outperforms them on CIFAR-10 and CIFAR-100.

Under review as a conference paper at ICLR 2021
IMPROVING LOCAL EPFECTIVENESS FOR GLOBAL ROBUST TRAINING
Anonymous authors Paper under double-bling review
ABSTRACT
Despite its popularity, deep neural networks are easily fooled. To alleviate this deficiency, researchers are actively developing new training strategies, which encourage models that are robust to small input perturbations. Several successful robust training methods have been proposed. However, many of them rely on strong adversaries, which can be prohibitively expensive to generate when the input dimension is high and the model structure is complicated. We adopt a new perspective robustness and propose novel training algorithm that allows a more effective use of adversaries. Our method improves the model robustness at each local ball centered around an adversary and then, by combining these local balls through global term, achieves overall robustness. We demonstrate that, by maximizing the use of adversaries via focusing on local balls, achieve high robust accuracy with weak adversaries. Specifically, our method reaches a similar robust accuracy level the state of the art approaches trained on strong adversaries on MNIST, CIFAR-10 and CIFAR-100, 100, As a result, the overall training time is reduced. Furthermore, when trained with strong adversaries, our method matches with the current state of the art on MNIST and outperforms them on CIFAR-10 and CIFAR-100.
INTRODUCTION
With the proliferation of deep neural networks (DNN) in areas including computer vision, natural language processing and speech recognition, there has been a growing concern over their safety. For example, (Szegedy al. 2013) demonstrated that naturally trained DNNs are fact fragile. By adding to each data a perturbation that is carefully designed but imperceptible to humans, DNNs previously reaching almost 100% accuracy performance could hardly make a correct prediction any more. This could cause serious issues in areas such as autonomous navigation or personalised medicine, where an incorrect decision can endanger life. To tackle these issues, training DNNs that robust to small perturbations has become active area of research in machine learning.
Various algorithms have been proposed (Papernot et al., 2016 K Kannan et al.] 2018 Zhang et al. 2019b Qin al. 2019 Moosavi-Dezfoolii et al. | 2020 Madry al. 2018 Ding al.] (020). Among them, adversarial training (ADV) (Madry et al.) (018) and TRADES (Zhang et al. 2019b) are two of the most frequently used training methods so far. Although developed upon different ideas, both methods require using strong adversarial attacks, generally computed through several steps of projected gradient descent. Such attacks can quickly become prohibitive when model complexity and input dimensions increase, thereby limiting their applicability. Since the cost of finding strong adversaries mainly due to the high number of gradient steps performed, one potential approach to alleviate the problem is to use cheap but weak adversaries. Weak adversaries are obtained using fewer gradient steps, and in the extreme case with a single gradient step. Based on this idea, |Wong et al.) (2020) argue that by using random initialization and larger step-size, adversarial training with weak adversaries found via one gradient step is sufficient to achieve satisfactory level of robustness. We term this method one-step ADV from now While one-step ADV does indeed exhibit robustness, there is still a noticeable gap when compared with its multi-step counterpart.
In this paper, further bridge the gap by proposing new robust training algorithm: Adversarial Training via LocAl Stability (ATLAS). Local stability, in our context, implies stability of prediction and is the same local robustness. Specifically, make the following contributions:

---

Under review as a conference paper at ICLR 2021
We adopt a new perspective on robust accuracy and introduce framework for construct- ing robust training losses that allow more effective use of adversaries. The framework consists of local component and global component. The local component maximizes the effectiveness of an given adversary by improving the network's 's robustness on both the adversary and points around it. In other words, the local component attempts increase the radius of a ball centered at the adversary on which the network is being robust. The global component combines all local balls in a regularized way achieve the desired overall robust performance.
Based on the framework and guided by the need of fast robust training, we propose our novel robust training algorithm ATLAS.
We show that ATLAS makes a more effective use of weak adversaries by favourably comparing it against one-step ADV on three datasets: MNIST, CIFAR-10 and CIFAR-100.
Although one-step ATLAS is more expensive than its other one-step counterparts, ATLAS still allows efficient robust training. We show that, with a one-step weak adversary, ATLAS manages to achieve comparable levels of robust accuracy to multi-step state of the art methods on all datasets.
Finally, we show that when strong adversaries are used, ATLAS matches with the current state of the art on MNIST and outperforms them on CIFAR-10 and CIFAR-100.
RELATED WORKS
Robust training aims to learn a network such that is able to give the same correct output even when the input is slightly perturbed. Existing robust training algorithms be divided into two categories: natural image based methods and adversaries based methods. Within the first category, the common form of loss natural loss term plus regularizer computed at natural images. We briefly mention some of these methods. Moosavi-Deziten
et al.] (020)) observed empirically that reducing the curvature of the loss function and the decision boundary could lead robust models. The authors thus propose regularizer based on the Hessian of the loss function. Closely related is the regularizer, introduced in (Jakubovitz Giryes 2018 Hoffman al. 2019 Ross Doshi- Velez 2018), that penalizes the Frobenius norm of the Jacobian of the loss function. Jacobian regularizer can also be seen as way of reducing the curvature of the decision boundary (Jakubovitz Giryes) 2018). Although calculating the norm is computationally expensive, fast estimator with empirically high accuracy has been developed in Hoffman et al. (2019).
We focus on adversary based robust training, as they generally perform better in terms of robust accuracy. Under this category, an early fundamental work is the Fast Gradient Sign Method (FGSM) by Goodrellow et al. (2015). Adversarial Training (ADV) (Madry et al. 2018) is multi-step variant of FGSM. Rather than using one step FGSM, ADV employs multi-step projected gradient descent (PGD) (Kurakin et al. 2016) with smaller step-sizes generate perturbed inputs. These modifications have allowed ADV to become one of the most effective robust training methods so far (Athalye) al. 2018). Another frequently used robust training method is TRADES (Zhang al.) 2019b). Motivated by its theoretical analyses of the trade-off between natural accuracy and robust accuracy in binary classification example, TRADES encourages model robustness by adding to the natural loss a regularizer involving adversaries to push away the decision boundary.
Recently, Qin et al. (2019) suggest that robust model can be learned through promoting linearity in the vicinity of the training examples. They designed a local linearity regularizer (LLR), in which adversaries are used maximize the penalty for non-linearity, Applying LLR also allows efficient robust training. We note that the underlying idea of LLR is complementary to ATLAS. In addition, several works have suggested to adopt input dependent treatments. These works include incorporating the fact of whether the given input is correctly classified (Wang et al. 2020 and using adaptive perturbation for different inputs (Ding et al. 2019).
One major drawback of adversary based methods (Madry et al. 2018 Zhang et al. 2019b Qin et al. 2019; Wang et al. 2020 Ding et al. 2019) is that most of them rely on strong adversaries, computed via expensive PGD. When the input dimension is high and the model structure is complicated, finding adversaries can be too expensive for these methods to work effectively. Several works have researched possible ways to speed up the process. Algorithmically, Zhang al. (2019a) cut down the total
2

---

Under review as a conference paper at ICLR 2021
number of full forward and backward passes for generating multi-step PGD adversaries while Zhang et al. (2020) introduce a parameter to allow early stop PGD. [Shafahi et al. (2019) reduce adversary overhead by combining weight update and imput perturbation update within single back propagation and use a single step FGSM adversary. Wong et al.] (220) argue that the main reason one-step FGSM, generally regarded initialization used. As a result, Wong et al. (2020) proposed that with random initialization and larger step size, weak adversaries generated by FGSM could lead models with a high level of robust accuracy. In this study, we adopt similar viewpoint of accelerating robust training through the use of weak adversaries.
robust, is effective in (Shafahi et al., 2019) is due to the non-zero
PRELIMINARIES AND NOTATIONS
We consider classification tasks. Let æ be image from the data distribution X' R and Va be its correct label from classes C (1....,C) C}. We denote a neural network, parameterized by 0, (a:; f) X C The function outputs logits and the predicted class label is chosen to be the element with the largest scalar value. Given a supervised dataset, the parameters 0 are obtained by minimizing loss function over the dataset. In natural training, a common loss choice is the cross-entropy loss, denoted as 8CE_ To compute eCE, we need prediction probabilities p(æ:; 0), which element-wise.
exp(f:(<=:0)) evaluated pe ; 0) Σ, (x:0)
Given tolerance level and a distance measure norm Ey. we say the network f is robust to adversaries at given point a X, if satisfies
arg max fa(;' 0) = arg max fi(2;; 0), IEC
Vad s.t ||æx
æ11p S.
(1)
iEC
We equivalently use ball (x (a' ||== - x01p S e) to represent the allowed perturbation. If f robust on B,, we call B, a robust ball.
Evaluating the true robustness of f on X' is challenging. In practice, we replace with a test set and evaluate the robustness of f by measuring the percentage of the test set satisfying the condition in equation ( To check whether the condition is met, various attack strategies are applied. Commonly used attacks are PGD based. To be specific, PGD performs the following gradient step during each iteration + 1,
E++1 = (x)) + sgn( VL(((( 0)),
<<E,
(2)
and repeats the iteration several times. Here, sgn denotes the sign function. On the other hand, FGSM uses single gradient step
= sgn (=;((((;())))))
Finally, we introduce the following notation to facilitate the discussion. We first assume that the neural network f contains ReLU activation for the sake of clarity. This implies that the neural network is piecewise linear. As a consequence, at each x E X, it is easy to find a weight matrix W e RCXN and constant vector b0 E R such that [a;; 0) W b". To simplify the notation, we define
: W b" (+++1))
T" x
[1]
R (M++1).
(4)
As result, at each point æ, we have f(x; WT Before delve in to the details of the algorithm ATLAS, we first introduce new framework for designing robust training losses.
4 THE GENERAL FRAMEWORK
Unlike the optimization perspective taken by ADV and the regularization viewpoint adopted by TRADES, our new loss framework is motivated from geometric standpoint. We start by briefly mentioning its closely related regularization type approaches. Generally, in such approaches, we start from natural image By including a loss term for regularization type losses ensure the model gives the correct prediction Ya at æ. Assume the network makes the correct prediction at the natural image æ. It follows that, there exists a robust ball B, (x), where 7 2 0 is the maximum radius achievable. Previous use of adversary regularization term encourages the radius y of the ball (x)) centered æ, to increase.
3

---

Under review as a conference paper at ICLR 2021
We take different direction in new loss framework. Instead of focusing centered at æ, we focus on local balls centered at various points E (x) and hopefully, by combining them in regularized way, achieve the desired robustness 7 c. To facilitate the illustration, we introduce our loss framework through a binary classification problem. Let there be two classes C = ((1,02 02). For arbitrary point X, the correct class label y/r o1 Since are interested in network's 's performance in classifying labels, we compute logits difference as
global ball
(æ: 0) fz 0) d R.
where [1, [1] is row vector. For binary classification problem with cross-entropy loss, the loss decreases monotonically with the increase of the value dW To achieve the desired robustness, we require d x' > 0 for all x' E B. (zx). When cross entropy loss & considered, we need ((x') log(0.5) for all x' (z). In Figure ! show potential loss curve B, (x) by fixing the value of a on all dimensions apart from one. Our loss framework consists local component and global component. Both components update the parameters of the network model.
Loss Original
Figure I: Original loss curve on (x) by fixing the value of all dimen- sions apart from one. Points with the loss above the decision boundary log 0.5 are adversarial points and marked as red.
Decision boundary (og()
-E
4.1 LOCAL ROBUSTNESS
For the local component, at each given local point x' (2r), the goal is to attain a robust ball (x"()) with large radius. We maximize the use of adversaries by treating them as local center points a'. other words, given an adversary x' E B. (ær), we need to satisfy two requirements: predicts the required label ya æ' and the radius 7 of (xy(x") should be enlarged. When compared with ADV, the additional second requirement allows us to achieve improved robustness performance when weak adversaries used.
model
The first requirement can be easily met by applying a standard loss term at x' with the label Yae For the second requirement, to push away the decision boundary. various methods including Jacobian regularizers (Jakubovitz & Giryes] [2018] Ross & Doshi Velez 2018), MMA (Ding et al. 2019)) and
MMR (Croce Hein 2019) have been introduced. An expected loss after introducing the local component is shown in Figure
Loss after a local step
ofter a global step
Desrease loss
Decrease the gop
Increase local roqustness
x
8 (Weok adverssryy)
(Weok adversory)
Loss curve after a local step. The dashed line is the original loss curve. Provided with a weak adversary s', the local step aims decrease the loss at x' while increase local robustness so the local robust ball centered at x' has large radius.
Figure 2:
Figure 3: Loss curve after a global step. The dashed line is the loss curve after local step. To further smooth the loss curve and to remove adversaries between and x', we penalize the difference p(z; 0) - p(æ': 0). As result, the gap between ((x) and ((x') is decreased.
4

---

Under review as a conference paper at ICLR 2021
4.2 GLOBAL REGULARIZATION
In terms of the global component, we combine local balls together in a controlled way to further improve robustness. We first make the following observation.
Proposition Consider ball (a( at an arbitrary point x: E E. For any xx E (a)), we define Wa and x" accordingly, as in equation (4). Let u be a constant such that, for all possible W the
following is satisfied [[ ||5 I u. Assume x² and x² are arbilrary chosen points in B. Then for any
2n = 1- + (1-1) x²,
(5)
we have
1 dW F. 2 (dW x² div
2h2) - L,
(6)
where L = 22 + 1 1 22) is constant.
We now consider the combination of local robust balls. At each loss computation, we have two points: the natural image and the adversary x'. Assume the model makes the correct prediction at the natural image and, after the local component loss, at æ', so we have two non-empty local robust balls B, (a) and (ar'). We further assume these two balls are disjoint for the sake of simplicity. The same underlying idea should work for more general cases. The goal is that, with the global regularization step, points that are misclassified even after the local step can be correctly predicted.
To do so, given that local robust balls (x)) and B,,((x) are disjoint, we assume there exists an adversary point x= E Bx(x) satisfying
x
(1 17") x',
where 1/" E (0, 1) is constant depending on (x)) and (xy) Then direct application of P gives
dW x² 2 (dw' n + - x') u L
(8)
d(W ा. x¹) dw x' u
where u and L. are defined accordingly.
To make x" no longer an adversary (that is, to encourage dW x' 0) after global combination, we should make the lower bound on the right as high as possible. Since both d and dW x' have the same max value theoretically, we can increase their sum by first increasing the value of dW E' and then decrease the distance between dWi= and dWi=' ø' We recall that during the local robustness step, have introduced cross-entropy loss x', which directly maximizes the value of dW ø'. In addition, enlarging local robust balls often leads to decreased value of Given that dw '' and u are already optimized, a sound global step is minimise the distance between d and dlia'' ø' We note that, for the distance to be zero, it is sufficient to have the relative differences
fz 0) and f2(2; 0) - (x; 0) to be equal rather than requiring an equality between logits fa; and f(x'; (d'))) 0). To account for this fact, we use prediction probabilities instead. The global regularization step requires minimizing the prediction probability difference p(æ;; 0) - p(x:; ; 0). As the result, we are likely to turn x² into a correctly predicted point after the global step, illustrated in Figure
When the loss surface considered, the global component loss encourages the loss surface to be smoothened and flattend on (x), so the global component is helpful even when such adversaries x" exist. As suggested in (Zhang et al., 2019b] Cisse et al. 2017 Ross Doshi- Velez 2018 Moosavi-Deztooli al. 2017). these properties of the loss surface are desired and possibly
indispensable for models to be robust.
Overall, various losses can be constructed under this framework. Depending the problem at hand, one can choose an appropriate regularizer for increasing local robust balls while selecting a suitable metric for penalizing the prediction probability difference.
5

---

Under review as a conference paper at ICLR 2021
5 ATLAS
Guided by the goal of effective fast robust training, we propose a new training algorithm ATLAS, which stands for Adversarial Training via LocAl Stability. We mention that local stability, in context, means stability of prediction, so it is the same as local robustness. ATLAS based on the framework. We introduce and explain our choices for local and global components.
5.1 ATLAS-LOCAL
Many existing approaches for increasing robust ball radius are computationally expensive. In this work, we adopt the standard Jacobian approach to meet our local component requirements based on the fact that an efficient estimation algorithm for Jacobian has been developed in Hoffman et al. (2019). We briefly introduce how Jacobian can be used to increase local robustness.
Given a local point æ', let the correct class label be hyper-surface separating classes and consists of points x" satisfying
Ve. Then for any d' * c, the boundary
fa 0) for (a= 0) 0
(9)
Applying the standard formula for computing the distance between a point and a hyper-plane) we get the first order approximation distance de of x' the boundary hyper-surface in equation under the l2 norm as
fo (x'; -
de V (xx;; V (2(';)))
(10)
Since the above equation holds true for any c', conclude the model is robust l2 norm ball centered at x' with radius := min.' de. To maximize the radius d, we borrow the following
proposition from Jakubovitz & Giryes (2018), which introduced Jacobian regularizer to the natural loss, to provide a lower bound for d.
Proposition Assume the model is making the correct prediction c= yu for x' and the distance metric is measured via 22 norm. The first order approximation of the minimum perturbation d that is required to find an udversary example is lower bounded by
d [2|J(=')|1
min [f((a'; fe fe (x':0)), d:0
(11)
where J(x') = Var (at'; ; 0) is the Jacobian matrix computed at x' and || ||r is the Frobenius norm.
For larger distance d, need to both increase the value of [so( 0) fer (ar:: 0)] and decrease the Frobenius norm of [zx). The first term can be easily taken care of by using a cross-entropy loss on x' while for the second term, we can include cheap approximation of [|]((x)) [[] denoted by
[J[x'))] [[pppror as penalty term in our loss. Using the idea of random projection, Hoffman et al. (2019) shows theoretically and empirically that [|J((z) | [apprer can be estimated with high quality by
one backward pass regardless the total number of classes C.
To combine the above analyses, for the local robustness component, we introduce the following loss,
E (x'))( CCE (x') + [| ppppror
(12)
where a is a positive scalar. It is worth noting that our local loss shown in equation 12 is different from that of (Jakubovitz & Giryes| 2018 Hoffman et al.) 2019 Ross & Doshi-Veler 2018 as the loss is evaluated at an adversary x instead of the image æ. In the fast robust training setting, x' is weak adversary obtained through one-step FGSM.
5.2 ATLAS-GLOBAL
To penalize the prediction probability difference, a natural and frequently used choice is Kull- back-Leibler (KL) distance. We thus formulate the global loss as
Gelobal (x') = =B. -KL(p(x's [ p(); 0)).
(13)
where 3 is positive constant.
in this case, the point should be x' and the hyper-plane is tangent the boundary hyper-surface.

---

Under review as a conference paper at ICLR 2021
Although generally ( 0))= (L(p(z, 0)lp((/ .0) due to the asymmetric nature of KL distance, the same underlying nature of both terms means it is sufficient to use one direction for the penalty. In our loss, have use the prediction distribution at x' to be the base distribution. There are two reasons for this choice: firstly, we want it to be consistent with the fact that the local component loss is computed on x'; and secondly, we hope it could mitigate a possible over-fitting issue. If a model over-fits at the original image ' causing an element (æ; 0) of p(æ; 0) to be close to O, the distance between pi (x;; 0) and p. (a'; 0) would not be penalized effectively. In extreme cases, the difference will not be penalized at all if p: (: 0) = O. Using p(='; ; 0) as the base might alleviate this issue, as x' keeps changing during each epoch. Under the same reasoning, we point out TRADES relies on finding strong adversaries. Since it uses the natural image x to compute cross-entropy loss and as the base distribution in the KL regularizer, TRADES cannot use adversaries effectively when they are weak. This is consistent with what we see in the experiments section that is, TRADES with one step FGSM is largely outperformed by other methods on challenging datasets. Furthermore, in Appendix C.I. we show that when putting more emphasis on adversaries by replacing with x' in TRADES, improved robust accuracy can be achieved.
5.3 FINAL Loss
Integrating the above analyses, we propose the following as the final loss for ATLAS. Given > 0 and 3 > O, the loss is formulated as
1
ATTIAS
CE (x')
+ a [J('))l Ippprr
3KKL(p(a': 0)),
a' E B,r)). (14)
X
ØEX
local
global
We mention that although our analyses are carried out in L2 norm, our results are easily generalizable to other norms. To ensure the model makes correct predictions at natural images E X' and for effective patch combination, we adopt an adaptive value scheme for the pre-determined perturbation epsilon. Specifically, we start by setting c = 0 and gradually increases its value to the required number over epochs during the initial stage of training. Assume the final value of E is v, to be consistent with the magnitude of e, we replace a and / as a and & 3 accordingly.
6 EXPERIMENTS
We evaluate the performance of ATLAS on three datasets: MNIST, CIFAR-10 and CIFAR-100. We consider the cases of training robustness models with weak adversaries generated by FGSM and with strong adversaries by multi-step PGD. To be consistent with the general experimental setting, we adopt 1z0 norm and use the perturbation value c = 0.3 for MNIST and = 8/ 8/255 for CIFAR- 10 and CIFAR-100. MNIST is trained on 4-ayer CNN, which consists of 2 convolutional layers followed [Zaooruyko
by 2 fully connected layers. CIFAR- and CIFAR-100 are trained on & Komodakis 2016). To encourage efficient robust training, instead of a fixed learning rate scheduler that decreases the learning rate at pre-specified epoch numbers, we randomly choose subset of the data to be the validation set and then use its robust accuracy to guide the learning rate adjustment and to terminate the training process. For MNIST, 20-step PGD is applied for computing robust accuracy on the validation set while for C0 and CIFAR- 10-step PGD is used. In addition, we gradually increase epsilon value from 0 to 0.3 or 8/255 for the first 15 epochs. We use the SGD) optmizer for the training. More training related details can be found in Appendix B.
Methods We compare against ADV and TRADES. We term methods that use weak FGSM ad- versaries as one-step methods. Following the advice from Wong et al. (2020), we apply FGSM, combined with random initialization and a step size of 1 1,25c to generate adversarial examples. One- step ADV is an implementation of (Wong et al.) a 2022)) To be consistent with their multi-step variants, we use cross- gross-entropy loss for computing the gradient in FGSM for ADV and ATLAS while employ KL. distance for TRADES. Multi-step methods are trained on strong adversaries generated by PGD: 20 steps for the MNIST and 10 steps for CIFAR- 10 and CIFAR-100. We also include the Jacobian penalty loss (Jakubovitz & Giryes 2018 Hoffman et al. 2019 Ross & Doshi Velez 2018) due to its close relation to our local loss component. We refer to it as zero-step JAC to emphasize the fact it does not require adversaries. For each method, a range of parameters are tested and those that give both high clean accuracy and high validation robust accuracy parameter choices are provided in Appendix F. Due to the limited space, the ablation study on the effect of each component of ATLAS: the local part (ATLAS-I, by setting B 0) and the global part
chosen. Results for all tested
7

---

Under review as a conference paper at ICLR 2021
(ATLAS-g, by setting 0) is left to Appendix C and the same is for detailed comparison between TRADES and ATLAS-g. In addition, we have include in Appendix D.1 an experiment to demonstrate ATLAS's 's effective use of weak adversaries. We compare ATLAS against loss which always treats the natural image æ, instead of a weak adversary x', as the center point for the local component. In the words, the loss takes the form of combining TRADES and zero-step JAC.
Robustness Evaluation Extensive robustness evaluations are conducted in order to avoid a false sense of security. Black box attack generates adversaries on surrogate models with 1000 PGD steps. In terms of white box attacks, apart from an initial robustness evaluation with PGD-20, we further introduce three strong white attacks: PGD-1000, Untargeted (Un-T) attack (Carlini & Wagner 2017) and Multi-targeted (Multi-T) attack (Gowal et al.] 2019). Untargeted attack uses the loss Ju (r; 0) - fa(ar; 0), where is the correct class and u = arg ma (z))). We allow u to change during each gradient step. Regarding Multi-targeted attack, we perform attack with the loss fa 8) fo ax; 0) for all i E C such that * c. On the big Wide-ResNet model, we run 10 restarts with 100 steps for Untargeted attack and restarts with 20 steps for Multi-target attack. We reduce the number of restarts and steps for the Multi-target attack to account for the fact that each incorrect class needs to be tested. Due to the large number of classes, Multi- target is not performed on CIFAR-100 On the other hand, we perform 20 restarts with 50 steps for both attacks on the small CNN model. For each method, attack results are reported for a model at the epoch that gives the best validation robust accuracy during the training. On the challenging datasets CIFAR-10 and CIFAR- 100, two more robust evaluations are carried out to obtain a more comprehensive understanding of models' robustness performance. Specifically, we apply an hard-label attack, RayS (Chen & Gu 2020) and an ensemble of diverse parameter-free attacks, AutoAttack (Croce & Hein 2020). Models' robustness performance under these two evaluations are reported in Appendix D.2.
We summarize attack results in Table []] We mention that with learning rate scheduler, which is guided by the robust accuracy on the validation set, all models achieved their best performance around 30 epochs, except 12 epochs for zero-step JAC on MNIST. In terms of robustness performance, slight performance improvements for all methods and narrower performance gaps will be observed if all models are trained for 70 epochs with a fixed learning rate scheduler on MNIST. On the other hand, on CIFAR-10 10 and CIFAR-100 we found all methods perform better with the guided learning rate scheduler. Since we are interested in efficient robust learning, we use the guided learning rate scheduler for all experiments.
Evaluations on CIFAR-10 We study the challenging CIFAR-10 first. For zero-step Jac, we observe that to achieve roughly 30% robust accuracy, nominal accuracy is dropped to below 65%. Directly including a Jacobian penalty at natural images could thus lead to over-regularization issues. This may also be the reason why the method is used as a post-procsssing technique in (lakubovizz & Ciryes] 2018). On the other hand, by using local adversaries instead of natural images, ATLAS-I manages to obtain robust accuracy without sacrificing the nominal accuracy too much, which is demonstrated in the Appendix. When TRADES is considered, we find that its effectiveness relies on strong adversaries: robust accuracy increases sharply from one-step case multi step case. Since TRADES computes loss at natural images and use them as the base distribution in the KL penalty, weak adversaries are not effectively used.
In addition, we constantly find for one-step TRADES that, after a certain number of epochs, the percentage of the model making incorrect decisions on weak adversaries falls sharply, leading to sudden drop of more than 10% on validation robust accuracy. This phenomena is referred to as catastrophic overfitting (Wong et al., 2020 Andriushchenko & Flammarion 2020). We observe
that when this behaviour occurs, the approximated value J(x') [I'']l [[pprr increases steeply, which is
consistent with what has been observed in AAnrrushchhnko & Flammarion (2020). When dealing
with catastrophic overfitting is the main concern, Andriushchenko & Flammarion (2020) argue the key is to increase local linearity and they introduce a regularizer to maximize gradient alignment for points within the perturbation ball. In our case, the issue could be similarly resolved by increasing the coefficient a to encourage local linearity. However, since the goal of this project is to achieve high robust accuracy fast, we do not restrict ourselves to models without catastrophic overfitting only, which are likely to compensate robust accuracy for stability. Instead, we use early termination and consider a wider range of models. We mention that the same phenomenon is observed on one-step ADV and on one-step ATLAS when is small but less frequently. More discussions on catastrophic overfitting can be found in Appendix E.
8

---

Under review as a conference paper at ICLR 2021
Table I: Robustness performance for MNIST on small CNN and for CIFAR-10 10 and CIFAR-100 on Wide-Resnet 28-8. The higher the better.
White hox attack
Time
Model
Clean accuracy
Black hox attack
PGD-20
PGD-1000
Un-T attack
Multi-T attack
per hatch
attack
attack
zero-step JAC
98.34%
89.84%
58.09%
2.48%
28.83%
28. 72%
0.012s
one-step ADV
99.49%
96.26 %
96.29%
83.25%
91.02%
90.62%
0.009s
one-step TRADES
99.40%
96.21%
96.28%
83.05%
90.66%
90.02%
0.016s
MNIST
one-step ATLAS
99.47%
96.36%
96.84%
89.56%
92.44%
92.04%
0.018s
twenty-step ADV
99.48%
96.40%
97.16%
92.32%
93.01% %
92.87%
0.077s
twenty-step TRADES
99.49%
96.21%
97.01%
90.51%
92.49%
92.21%
0.097s
twenty-step ATLAS
99.44%
96.45%
97.19%
92.20%
93.03%
92.74%
0.086s
zero-step JAC
63.24%
60.45%
34.40%
34.34%
31.13%
30.32%
0.47s
one-step ADV
86.24%
82.53%
45.73%
44.98%
45. 45.14%
42.97%
0.25s
one-step TRADES
86.84%
82.63%
40.03%
39.41%
39.31%
38.14%
0.55s
CIFAR-10
one-step ATLAS
84.52%
81, 81.06%
49.01%
48.59%
48.54%
46.55%
0.73s
ten-step ADV
84.77%
82.53%
50.60%
50.22%
49, 49.74%
47.96%
1,38s
ten-step TRADES
84.01%
82.63%
52.70%
52.45%
50.29%
49.66%
2.02s
ten-step ATLAS
82.94%
.066
53.45%
53. 13%
51.51%
50.16%
1.88s
zero-step JAC
47.96%
44.67%
15.39%
15. 18%
14.35%
0.47s
one-step ADV
48.88%
45.59%
19 19.89%
19.72%
19.04%
0.25s
CIFAR- CIFAR-100 100
one-step TRADES
62.58%
57.64%
16.93%
16.21%
14.83%
0.53s
one-step ATLAS
60.98%
57.04%
26.30%
25.94%
26.28%
0,73s
ten-step ADV
60.72%
58.15%
27.71%
27.43%
26.83%
1.37s
ten-step TRADES
59.76%
57.78%
26.27%
26. 10%
23.86%
2.00s
ten-step ATLAS
59.62%
57.83%
29.12%
28.89%
27.77%
1.87s
Overall, one-step ATLAS outperforms other one-step methods in all strong white box attacks. Even when compared with methods trained with 10 steps PGD, the performance gap between one-step ATLAS and multi-step ADV under the strongest Multi-T attack below 1.5%. Although adding local and global penalties result in extra computational time for one-step ATLAS (0.73s) than one-step ADV (0.25s), the fact that ATLAS achieves comparable high robust accuracy to that of multi-step ADV (1.38s) with weak adversaries still allows a roughly 50% speed-up in training. For a fair comparison, we also evaluate an ADV model with PGD steps. The 5-step ADV models requires slightly more time (0.75s per batch) but is less robust than one-step ATLAAS by reaching 44.93% accuracy under the Multi-T attack. When trained on strong adversaries, multi-step ATLAS wins over the other two nulti-step methods in all strong white box attacks.
Evaluations on CIFAR-100 and MNIST Similar performances are observed on the CIFAR- The key fact to notice here is that the per batch training cost required for CIFAR- 1100 is similar to that of CIFAR-10. Since we apply an efficient approximation to evaluate ]| || the increase total class number does not result in rise of total computational cost. Again, evaluate ADV models with PGD steps, which requires 0.75s per batch and reaches 25.21% 2 robust accuracy under Un-T attack. On MNIST, one-step ATLAS performs the best as well. Since MNIST is a simple dataset, all multi-step methods obtain high robust accuracy on the small CNN model. For the same reason, the unstable sudden drop of validation robust accuracy is not observed on any one-step method.
7 DISCUSSION
We have adopted a different perspective and proposed new framework for constructing losses that allow more effective use of adversaries. Specifically, based on the framework, we introduce a novel algorithm ATLAS for fast robust training. ATLAS can be used as initialisation technique for other complicated tasks. For instance, model trained with ATLAS can be employed as the starting point for layer-wise adversarial training in improving certified robustness. Apart from fast robust training, believe other losses could be constructed from the framework to accommodate different problems. We leave the explorations of potential applications to various robust training setting future research.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. Advances in Neural Information Processing Systems, 33, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, 2018.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 2017..
Jinghui Chen and Quanquan Gu. Rays: A ray searching method for hard-label adversarial attack. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1739-1747, 2020.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In Proceedings of the 34th International Conference on Machine Learning-Volie Volume 70, pp. 854-863. JMLR. org, 2017.
Croce and M. Hein. Provable robustness against adversarial 4p lp-perturbations for p 2 1. International Conference Learning Representutions, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks, arXiv preprint MxXiv:2003.01600 2020.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In International Conference on Learning Representationss 2019.
G.W. Ding, Y. Sharma, K. Y.C.C. Lui, and R. Huang. Max-margin adversarial (mma) training: Direct input space margin maximization through adversarial training. International Conference on Learning Representations, 2020.
IJ Goodfellow, Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. Interna- tional Conference on Learning Representations, 2015.
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy Mann, and Pushmeet Kohli. An alternative surrogate loss for pgd-based adversarial testing. arXiv preprint drXiv:1910.09338, 2019.
Hoffman, D.A. Roberts, and S Yaida. Robust learning with jacobian regularization. https://arxis.or//aa///00 027099 2019.
Daniel Jakubovitz and Raja Giryes. Improving dnn robustness to adversarial attacks using jacobian regularization. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 514- 529, 2018.
H. Kannan, A. Kurakin, and I. Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06373, 2018.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv:1607.0255: 2016.
A. Madry, A. Makelov, L.. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. Analysis of universal adversarial perturbations. arXiv preprint arXiv: 100..005559544 2017.
S.M. Aidosavi-Dezfoolh A. Fawzi, J. Uesato, and P. Frossard. Robustness via curvature regulariza- tion, and vice versa. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
10

---

Under review as a conference paper at ICLR 2021
N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami. Distillation defense to adversarial perturbations against deep neural networks. IEEE Symposium on Security and Privacy (SP) (pp. 582-597), 2016.
C. Qin, Martens, Gowal, D. Krishnan, K. Dvijotham, A. Fawzi, S. De, R. Stanforth, and P. Kohli. Adversarial robustness through local linearization. In Advances in Neural Information Processing Systems, 2019.
Andrew Slavin Ross and Finale Doshi Velez. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In Thirty-second AAAI conference on artificial intelligence, 2018.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, pp. 3353-3364, 2019.
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1316.199, 2013.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference Learning Representations, 2020.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. The International Conference on Learning Representations, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.077 2016.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Painless adversarial training using maximal principle. In Advances in Neural Information Processing Systems, 2019a.
H. Zhang, Y. Yu, J. Jiao, E.P. Xing, L.E. Ghaoui, and M.I. Jordan. Theoretically principled trade-off 2019b.
between robustness and accuracy, arXiv preprint
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning, 2020.

---

Under review as a conference paper at ICLR 2021
A. PROOF FOR PROPOSITION 2
Proposition 1. Consider ball (x(x) at an arbitrary point x E X. For any x. E Ba()) we define
W= and x" accordingly, as in equation (4) Let u be constant such that, for all possible W= the following is satisfied | ||r I Assume x² and x² are arbitrary chosen points in (x)x). Then for any
Bn = + 11) xx²
(5)
we have
dW F
1
ddi x² dii divx=(=2=
L.
(6)
2 where I. = 12(22 + || #² 2
is constant.
Proof. The proof is straightforward. To simplify the notation, we define = dw x s² = dW= 8² and 8n, == dW Also, let / be the lower bound dW x² 2 for any x' E (x)). Such exits because B, (a) is a closed and bounded ball and / continuous. We first note that
1 - 1| - |s - s, S 1s,, - L].
Since s - 1 and 8n 12 1220 O, we can remove the absolute sign and have
1st s,l S Sq
(15)
The term 1s - -sy1 can be upper bounded as
|s snl |dw x² di/
< (dWWaa -dw dd ) + Id/ (x! - #.,)l
(16)
2u ||||2| + u01 1)]| x 1|2
We further denote = =2ulle 2u]|d]] [d]] 1 and u||d||2/| x 1|2. By replace 1st lyl with its
upper bounds equation (16). follows that
m (1 -o-n) n) S 8m.
(17)
Similarly for s², we have
1/0 S 8n
(18)
Finally, combining equation
17 and equation
18)
we get
1
1
(s
+
S Sn
(19)
2
the desired inequality.
B. EXPERIMENTAL DETAILS
We give more details about training and attacks in the following.
Training We rely on the robust accuracy on a validation dataset guide the training process. To be more specific, we first randomly select 5000 images from the training dataset to form a validation set before training starts. During training, after each epoch, we run multi-step PGD attack evaluate the model's robust accuracy the validation set. If the validation robust accuracy does not improve for a fixed number of consecutive epochs (we refer to the number as plateau epoch number), we decrease the learning rate by five. If the robust accuracy does not improve for ten consecutive epochs, we terminate the training process.
MNIST We use SGD optimizer with starting learning rate of 0.01 on MNIST. Batch size is set to be 128. After each epoch, we apply a 20-step PGD attack with step size 0.01 to determine validation robust accuracy. Similarly. when generating strong adversaries for multi-step methods, 20-step PGD attack with step size 0.01 is used. For numbers reported Table zero-step JAC trained with OJAC 0.5; one-step and multi-step TRADES trained with STRADES 1; one-step ATLAS is
12

---

Under review as a conference paper at ICLR 2021
trained with le 05 and 3 = 0.3 while multi-step ATLAS is trained with 5e 06 and 3 = 0.2.
CIFAR-10 and CIFAR-100 On both CIFAR-10 10 and CIFAR-100, employ the SGD optimizer with a starting learning rate of 0.1, a momentum 0.9 and a weight decay of 2e - 4. A batch size of 64 is used. In terms of computing validation robust accuracy and generating strong adversaries in multi-step cases, we apply 10-step PGD attack with step size 0.007. On the CIFAR-10 dataset, the following parameters are used for the numbers reported in Table zero-step JAC is trained with OJAC = 0.5; one-step TRADES is trained with STRADES = 2: multi-step TRADES is trained with STRADES 4: one-step ATLAS is trained with a 0.0002 and = and multi-step ATLAS is trained with a = 0.0001 and 3 = 5.0. On the CIFAR- 10 dataset, we used ajcC = 0.5 for zero-step for one-step step TRADES; STRADES
JAC: STRADES
for multi-step TRADES; a = 0.0002 and B = 5.0 for one-step ATLAS and a 0.0001 and , = 10.0 for multi-step ATLAS.
Attacks Random initialization is applied all attacks. Untargeted and Multi-targeted attacks are implemented by following the descriptions given in (Qin et al.) 2019). For these two attacks, we consider an attack is successful if an adversary is found after a gradient update at any point during the optimization procedure.
B. PLATEAU EPOCH NUMBER AND FGSM STEP-SIZE
Due to the simplicity of the MNIST dataset and similar behaviour on both CIFAR-10 and CIFAR-100 we determine the plateau epoch number and the FGSM step-size by experimenting with one-step ADV on CIFAR-10.
One step ADV
Clean-pit2
0.9
Robust-pit2
Clean-pité
0.8
Robust-pib4
Accuracy
0.7
0.6
0.5
0.4
0.3
0.2
1.1
1.2
1.3
1.4
15
16
1.7
1.8
1.9
FGD stop (multiples
Figure 4: Clean and robust accuracy for one-step ADV at different step sizes
We use strongest Multi-T attack for evaluating model's robust accuracy. In Figure 4 the green lines show the results for plateau epochs to be 2 (plt2) while red lines are for plateau epochs to be 4 (plt4). In terms of step sizes, we lested 3 possible values: 1.25c (the suggested value by |Wong et al. (2020), 1.5c and 1.8c. It is clear to see that with plt2, models perform better in both nominal accuracy and robust accuracy at all three step sizes. We thus use plateau epoch number to adjust the learning rate in all our experiments for comparable results. When step size is considered, both 1.25c and 1.5c give satisfactory results. Since other methods show more stable performances with the step size 1.25c, we use for generating weak adversaries in experiments.
ATLAS: ABLATION STUDIES
We perform ablation studies for ATLAS. We consider the one-step setting. Recall that ATLAS consists of two components: local component and a global component. We test the effect of the local component by setting 3 = 0 to get ATLAS-I:
LATLASH-
Σ sEN
a' E B.))..
(20)
13

---

Under review as a conference paper at ICLR 2021
and the global component by setting
0 to get ATLAS-g:
& =
Σ FEN
C (x)' + KL(p(='; 0)lp((; 0)), a' (.))).)
(21)
[
We apply the strongest Multi-T attack on C1FAR-10 and Un-T attack on CIFAR- 100. Results are summarised in Table 2
When CIFAR-10 is considered, both ATLAS-I and ATLAS-- alone are effective in improving the model's robust accuracy. Combining the local and global components (one-step ATLAS) lead to a further robustness improvement. In terms of time per batch, apart from the cross-entropy loss term at the adversary in both ATLAS-I and ATLAS-g, ATLAS-1 requires computing the gradient of a Jacobian approximation term and is computationally more expensive than ATLAS-g, which calculates the gradient of a KL penalty term instead. Furthermore, the fact that ATLAS-g is computationally cheaper than TRADES is because uses cross-entropy loss to compute gradients for FGSM while TRADES uses KI. distance. Similar performance is observed CIFAR-100.
Table 2: Robustness performance for CIFAR-10 10 and CIFAR-100 on Wide-Resnet 28-8. The higher the better.
Model
Clean accuracy
Un-T attack
Multi-T attack
Time per batch
onc-step ADV
86.24%
42.97%
0.25s
CIFAR- 10
one-step TRADES
86.84%
38. 14%
0.55s
one-step ATLAS-1
85.24%
44.46%
0.60s
one-step ATLAS-g
82.07%
44.43%
0.38s
one-step ATLAS
84.52%
46.55%
0.73s
one-step ADV
48.88%
19.04%
0.25s
CIFAR- 100
one-step TRADES
62.58%
14.83%
0.53s
one-step ATLAS-I
58.46%
24.73%
0.60s
one-step ATLAS-g
51.88%
19.71%
0.37s
one-step ATLAS
60.98%
26.28%
0.73s
We show model's performance when trained with ATLAS-I and ATLAS- g at different parameter values. Results for CIFAR-10 are summarised in Figure 5 and 6 while results for CIFAR-100 are summarised Figure 7 and g For CIFAR-10, we see that the clean accuracy decreases with the increase of value for ATLAS-I but robust accuracy retains the similar level. In terms of ATLAS-g g, the same trend is observed and when the value is large, both clean and robust accuracy decline. On CIFAR-100. there is big drop in robust accuracy for ATLAS-I when rises while robust. accuracy for ATLAS-g is relatively insensitive to the change of 3 value.
ATLAS-L (CIFAR-10) 10)
ATLAS-G (CIFAR-10)
ATLAS-G (GFAR-10)
Clar-Acc Robust-ACC
Acc Robust-Ac
0.9
0.9
0.5
0.7
0.5
0.5
0.4
0.4
0.3
8.000
0.001
0.003 ATLAS alstra value
0.002
0.004
0.3
10
10
20
30
40
ATL4S Bota value
ATLAS Bota value
Figure 5: Clean and robust accu- racy for ATLAS-1 at different on CIFAR-10
Figure 6: Clean and robust accuracy for ATLAS-I at different 3 on CIFAR-10
14

---

Under review as a conference paper at ICLR 2021
(C)FA--100)
ATLAS-G (CFFAR-100)
ATLAS-G (CIFAR-100)
Clan-Acc Robust Acc
0.8
Robust-ACc
0.8
0.8
0.7
0.6
3.6
0.5
0.5
0.4
0.3
0.2
0.2
3000
3.0
ATLAS
ATLAS
ATLAS
Figure 7:
Performance of different a on
Figure 8: Clean and robust accuracy for ATLAS-I at different on CIFAR-100
ATLAS-1
CIFAR-100
C.I ATLAS-G vs. TRADES
ATLAS-g and TRADES take similar forms. The only differences between these two methods are: firstly, ATLAS-g computes cross-entropy loss at the adversary while TRADES at the natural image; secondly, ATLAS-g g uses cross-entropy loss to find gradient in FGSM while TRADES employs KL distance.
On CIFAR-10, clear that ATLAS-g outperforms TRADES on both clean and robust accuracy for all tested / values. This observation supports the fact that ATLAS-ga g, by computing the loss at weak adversaries, allows more effective use of them. Maximizing the use of weak adversaries important in fast robust training setting. In terms of CIFAR-100, ATLAS-g outperforms TRADES on robust accuracy. However, TRADES achieves higher clean accuracy on small values. The regularization effect of ATLAS-g could be higher than TRADES.
ATLAS-G TRADES (CIFAR-10)
ATLAS-G vs TRADES (CIFAR-100)
ATLAS Robust-Acc-ATLAS
0.9
0.9
0.8
Clean-Art-Trades
0.8
Robust
0.7
0.5
0.6
0.4
0.5
0.
0.3
0.0
ATLAS Bala valuee
ATLAS Beta value
Figure 9: ATLAS-g vs. TRADES on CIFAR-10
Figure 10: ATLAS-g VS. TRADES on CIFAR-100
D ADDITIONAL EXPERIMENTS
We compare ATLAS against another loss of similar form. Furthermore, perform extra attacks to obtain more comprehensive evaluations of all tested models. Due to the simplicity of MNIST, we focus on the challenging datasets CIFAR-10 and CIFAR-100 only.
D.1 ADDITIONAL LOSS
We study an additional loss to demonstrate ATLAS's effective use of weak adversaries. We recall that within our framework, the key to maximize the use of weak adversaries to treat them as center points of local balls in the local component, We thus consider the following loss
1
Σ (z) + og [+( + Bi [L(pp<;; ())p(( 0)), x' E'E (x))).
TruueeJae
|x|
#EN
local
global
15

---

Under review as a conference paper at ICLR 2021
We term the loss TradesJac, as it can also been zero-step JAC loss. Adversaries are computed by using the gradient of the KL-distance. The main difference between TradesJac and ATLAS is in the local component, where TradesJac uses natural images as center points for local balls instead of weak adversaries.
combination of the TRADES loss and the
We train models using TradesJac with various sets of values for oi and 3g and reported results for the model with the best validation robust accuracy. Results can be found 3 On both CIFAR-10 10 and CIFAR-100, methods for easy comparison. is clear to see that ATLAS outperforms TradesJac in all attacks, confirming that ATLAS is able to employ weak adversaries more effectively in achieving robust accuracy.
have Orjj 0.0002 and Bij 0.5. We have also include other one-step
Table Robustness performance for one-step methods on CIFAR-IO and CIFAR-100 All models are trained with Wide-Resnet 28-8. The higher the better.
White box attack
Time
Model
Clean accuracy
Black box atlack
PGD-20
NGD-1000
Un-T
Multi-T atlack
per batch
attack
atlack
atlack
CIFAR-1O0 100 CIFAR-10
one-step ADV
86.24%
82.53%
45.73%
44.98%
45.14%
42.97%
0.25s
one-step TRADES
86.84%
82.63%
40.03%
39.41%
39.31%
38. 18%
0.55s
one-step ATLAS
84.52%
81.06%
49.01%
48.59%
48.54%
46.55%
0.73s
one-step TradesJac
84.38%
81.54%
46.04%
45.75%
44.60%
43.59%
1.14s
one-step ADV
48.88%
45.59%
19.89%
19.72%
19.04%
0.25s
one-step TRADES
62.58%
57.64%
16.93%
16.21%
14.83%
0.53s
one-step ATLAS
60.98%
57.04%
26.30%
25.94%
26.28%
0.73s
one-step TradesJac
58.15%
54.56%
21.22%
20.89%
19.78%
1.15s
D.2 ADDITIONAL ATTACKS
We perform two additional robustness evaluations on CIFAR-10 and CIFAR-100. The first one is hard-label attack called RayS (Chen & Gu 2020). For both datasets, we run RayS with 40000 queries on 1000 randomly selected images from test set. The second one is ensemble of parameter- attacks, named as AutoAttack (Croce & Hein 2020). AutoAttack consists of two variants of PGD attack, an attack focused on gradient-masker version of AutoAttack for all evaluations. Results for CIFAR-10 10 and CIFAR- can be found in Table and Table 5 respectiivl, ATLAS gives the most robust performance among methods in both weak and strong adversaries cases.
and a black-box attack. We have used the standard
Table 4: Robustness performance for CIFAR-10 on Wide-Resnet 28-8. The higher the better.
Table 5: Robustness performance for CIFAR-I00 on Wide-Resnet 28-8. The higher the better.
Model
RayS attack
Auto attack
Model
RayS attack
Auto attack
zero-step JAC
33.50%
30.03%
zero-step JAC
16.70%
11.85%
one-step ADV
50.20%
42.49%
one-step ADV
19.90%
17.17%
one-step TRADES
45.10%
37.74%
CIFAR-IM 100
one-step TRADES
19.50%
12.75%
CIFAR-10
one-step ATLAS
52.70%
46.16%
one-step ATLAS
28.60%
23.42%
one-step TradesJac
50.20%
43.16%
one-step Tradeslac
22.60%
17.99%
ten-step ADV
54.40%
47.56%
ten-step ADV
29. 29.80%
24.64%
ten-step TRADES
54.00%
49.40%
ten-step TRADES
28.20%
22.78%
ten-step ATLAS
53.20%
49.82%
ten-step ATLAS
30.60%
25.77%
16

---

Under review as a conference paper at ICLR 2021
E CATASTROPHIC OVERHITING
We give more details catastrophic overfitting behaviour in our experiments. We first show typical training plots for both cases, The upper row Figure [1 contains plots for model when catastrophic overfitting is avoided and the bottom row show plots for model when catastrophic overfitting happens. It is clear to see that catastrophic overfitting is highly associated with a sharp increase of Jacobian value, which is consistent with the observations made in (Andriushchenko & Flammarion 2020). In our experiments with roughly 30 epochs, we find that catastrophic overfitting always occurs for one-step TRADES and ATLAS-- but less frequently for one-step ADV and one-step ATLASS. For ATLAS-1, the catastrophic overfitting can be effectively avoided by increasing the value for a as it directly penalizes large Jacobian value. The same strategy can be applied for the full ATLAS method. We find that, for a stable training performance, the choice of a depends on the value of B. That is large / value requires a large a value.
Finally, we mention that since the goal is to achieve high robust accuracy, we use early termination and consider wider range of models regardless whether they experience catastrophic overfitting in the end.
Jacoblan approx
Val, Robust Acc
Test Test_Clean
0.5
0.55
6.6
0.2
46
0.4
070
34
0.2
0.5
20
0.1
6:5
0.0
636
Epoch
Epach
Epoch
(a)
(b)
(c)
Jarobtan approx
Val Robust Acc
Tesd Clean Arc
0.5
1.90
1900
1004
254
0.2
0.
Epoch
Epoch
Epoch
(d)
(c)
(f)
Figure 11: We show typical training plots for a model on CIFAR-10 when catastrophic overfitting is avoided in the upper row and when catastrophic overfitting happens in the bottom row. Upper row plots are for the ATLAS model with are for the ATLAS model with a = O and B = 8.. The figures on the left shows the value of [|J((a)) || approx throughout the training. Catastrophic overfitting is associated with steep increase of the Jacobian value. The figures in the middle gives the trend for robust accuracy on the validation dataset. Validation robust accuracy drops sharply with the sudden increase the Jacobian value. Finally, on the right, we have plots for clean accuracy the test set after each epoch. A further increase to almost 90% clean accuracy is experienced the catastrophic overfitting case.
0. 0.0002 and 3 5.0 while the bottom row plots
F PARAMETER CHOICES: ONE-STEP CASE
We show clean accuracy and robust accuracy for each method at various parameter choices.
17

---

Under review as a conference paper at ICLR 2021
F. ZERO-STEP JAC
Although zero-step Jac does not require adversaries, we used epochs for adjusting learning rate via validation robust accuracy be consistent. Model's performance over range of Onn values is shown in Figure 12 for CIFAR-10 and for CIFAR-100. There is a large trade-off between clean and robust accuracy for small Oracc values and then both clean and robust accuracy decrease with the increase of O'jac
Zero step Jac (CIFAR-10)
Zero step |ac (CIFAR-100)
AleanAcc Robust Acc
Clean-Acc Robust Acc
0.8
0.7
0.5
0.3
6.3
0.2
6.00
Jacobian
Jacobian
Figure 12: Clean and robust accuracy for zero-
Figure 13:
Clean and robust accuracy for zero-
step JAC at different Ogac on CIFAR-10
step JAC at different OJac on CIFAR-100
F.2 ONE-STEP TRADES
(CIFAR-10) and 15 (CIFAR-100) 100), we show one-step TRADES at various iTRADES values. It is easy to see that increasing the value of ÍTRADES mainly hurts the clean accuracy without improving robust accuracy.
In Figure 14
TRADES (CIFAR-10)
TRADES (CIFAR- 100)
0.9
Robestabc
Rebusi-Acc
0.8
Accurcy
0.6
0.5
0.4
0.4
0.3
6.2
0
2
6
10
12
10
12
Trides Betb
Trases Beta
Figure 14: Clean and robust accuracy for one-
Figure 15: Clean and robust accuracy for one-
step TRADES at different STRADES on CIFAR-
step TRADES at different PTRADES on CIFAR- 100
10
F.3 ONE-STEP ATLAS
In Figure 16 give clean and robust accuracy for models trained different and / values CIFAR-10. 10. In Figure [7] we give clean and robust accuracy for models trained at different a and B
values CIFAR- 100. slight trade-off between clean and robust accuracy can be observed.
18

---

Under review as a conference paper at ICLR 2021
ATLAS: alpha=0. 0001 (CIFAR 10)
ATLAS: alpha=0 0002 (CIFAR 10)
(CIFAR-10)
0.5
0.5
0.5
0.5
0.4
0.3
0.3
10
(a)
(b)
(c)
ATLAS: alppha0.000111111.00001000000010 (CIFAR-10)
ATLAS: alpha-0.002 (CIFAR-10)
ATLAS: alpha -0 0.00 (CIFAR-10)
Crarstre
0.8
0.3
Robustdtt
0.6
0.7
0.7
0.7
0.4
0.4
0.4
Value
(d)
(c)
Figure 16: Clean and robust accuracy for one-step ATLAS at different a, on CIFAR-10
beta-5 (CFAR-100)
beta-10 (CIFAR-100)
beta-20 (CIFAR-100)
ATLAS: betam ICII AR-1001
ALCUFOCY
ACOUDCT
0.2
0.2
01 3,0005 0.0010 6,0015 00020
00000
0.0015
0.0019 3.0015 0.0200
E10000
0.0005
0,0000
0.0015 9,0020
0,0000
9,0015 0,0019 0.0155 0,0020
Apha
ApraVVauu
Alpla
(a)
(b)
(c)
(d)
Figure Clean and robust accuracy for one-step ATLAS different o, 3 on CIFAR-100
19


Reviews:
[{"review_id": "fZjF5Wtz7QP", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary:**\nThe paper proposes a new way to combine existing techniques for improving adversarial robustness: adversarial training, Jacobian regularization and TRADES consistency loss. The obtained results on three datasets are better than the baselines which rely on adversarial training, Jacobian regularization or TRADES separately.\n\n**Pros:**\n- Good empirical results.\n- Extensive empirical evaluation on MNIST, CIFAR-10, CIFAR-100.\n- Ablation study for the components of the method.\n\n**Cons:**\n- My main concern is the novelty of the proposed approach. The paper proposes to use adversarial training together with the TRADES-loss (with a reversed KL-divergence) and a variant of approximate gradient penalization at an adversarial point. All these approaches existed before but now just all combined together.\n- Additional concern is that the method requires 2 additional hyperparameters alpha and beta compared to usual adversarial training that doesn’t lead to any additional hyperparameters. I think this concern is especially relevant since the method is proposed to be useful for fast adversarial training (i.e. with one-step adversarial examples).\n- Another concern is the clarity of the presentation. It was really hard to grasp what are the sets $P_x$, $P_x’$, $S_\\theta(x)$, $S_{\\theta^1}(x)$, $S_{\\theta^2}(x)$ and relations between them. Maybe it would be better to clarify it with a picture / diagram. Currently, the theoretical part doesn’t seem to be clear or convincing enough to me.\n- It would be more insightful if one can provide a clear discussion on how and why the proposed method mitigates the catastrophic overfitting problem (similarly to these recent papers [Li et al. (2020)](https://arxiv.org/pdf/2006.03089.pdf) and [Andriushchenko et al. (2020)](https://arxiv.org/pdf/2007.02617.pdf) which focus on overcoming this problem). There are some mentions \n\n**Minor suggestions**\n- For me, it seems to be a bit misleading to call a subset of the input space a patch given that in the literature, patches are mostly referred to perturbations on the image plane (e.g. as in [Adversarial Patch paper](https://arxiv.org/abs/1712.09665)).\n- Contributions 2 and 3 at the top of page 2 are nearly duplicates.\n- “we **take care** of local balls at various points” -- imprecise (what it means to take care in this context?) and a bit informal language.\n- Equation (6): in the denominator it should be f_c’ instead of f_c in the second term.\n- Page 5: “as we evaluate it at an adversary x’ instead of the image x” -- it’s not yet clear what “an adversary x’” means, in particular with which method this point is obtained, is it epsilon-bounded (I suppose yes but this becomes clear only much later in the text) or not. Would be good to clarify this.\n- Page 7: “validation set robust accuracy guided learning rate scheduler” -- consider splitting this phrase which is too complicated.\n- Page 7: “natural images will **prevail** when weak adversaries are used” -- meaning is unclear.\n\n**Score:**\n5/10 because of the concerns about the novelty and clarity of the provided justifications of the method.", "extra_scores": {}, "extra_text": {}}, {"review_id": "WyLrJRFjTfY", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors introduce a novel technique (called LEAP) that improves model robustness at local patches (around an adversarial example) and combines \"local patches\" to get global robustness. Their approach works better than other techniques when using a \"weak\" adversary.\n\nOverall, the paper is well structured. However, it is a bit confusing at points, and sometimes not clearly motivated. The experiments seem fair and demonstrate that the proposed approach is better than other state-of-the-art techniques.\n\n1) I find the notion of local patches difficult to understand, and the paper seems to go through many hoops to justify a rather simple idea (simple ideas are good). This paper reminds me a lot of [1] and the loss presented could be justified through other means.\n2) Talking of [1] (although this is not necessary due to short amount of time between NeurIPS acceptance and ICLR submission), I'd appreciate a discussion on the differences.\n3) When mentioning PGD, cite [2] (for BIM). When mentioning FGSM, cite [3].\n4) When W^x and b^x are initially introduced, it is unclear what they are representing. Is that a first order Taylor expansion around x?\n5) A few sentences seem useless and sometimes the language is hand-wavy: e.g., \"it is highly likely the volume of patch P_x is non-zero\".\n6) The whole paragraph before section 4.1 does not seem to be useful for the rest of the paper.\n7) It is unclear in Sec 4.1 whether proposition 1 is part of Hoffman et al. (2019).\n8) It took me a while to understand the A(J(x)) notation. A() seems to be a function that takes J(x) as input. Consider directly using the Frobenius norm |J(x')|_F here and mention that it can be approximated.\n9) Re-ordering the final loss a bit, we have L_leap = l(x') + \\beta * KL(...) + \\alpha * |J(x')|_F. The first two terms are reminiscent of TRADES (except that they are applied to x') and the last term acts as a regularization term (similar to one used in [1] or [4]). Could the experiments also include an alternative loss:  l(x) + \\beta * KL(...) + \\alpha * |J(x)|_F (combining TRADES and zero-step JAC).\n10) When mentioning Untargeted attack with margin loss, cite [5]. When mentioning Multi-targeted, cite [6].\n11) The authors seem to use beta = 4 for TRADES. However, TRADES seem to work better with beta = 6. In particular, training a WRN-28-10 using TRADES with 10 steps of PGD on CIFAR-10 should reach 51-52% robust accuracy, the authors get 49.66% for a WRN-28-8. Can the authors explain the gap?\n13) The authors should also compare with \"Adversarial training for free!\" [7]\n\nOther comments:\n\nA) Qin et al. [4] is lumped with techniques that requires a strong adversary. However, Qin et al. show that only 2 PGD steps are necessary.\nB) Table 1 could include the number of steps rather than the vague \"multi-step\" wording.\nC) The Discussion seems more like a Conclusion.\n\n[1] https://arxiv.org/pdf/2007.02617: Understanding and Improving Fast Adversarial Training\n[2] https://arxiv.org/pdf/1607.02533: Adversarial examples in the physical world\n[3] https://arxiv.org/pdf/1412.6572: Explaining and Harnessing Adversarial Examples\n[4] https://arxiv.org/pdf/1907.02610: Adversarial Robustness through Local Linearization\n[5] https://arxiv.org/pdf/1705.07263: Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods\n[6] https://arxiv.org/pdf/1910.09338: An Alternative Surrogate Loss for PGD-based Adversarial Testing\n[7] https://arxiv.org/pdf/1904.12843: Adversarial Training for Free!", "extra_scores": {}, "extra_text": {}}, {"review_id": "Paz1tUqajyI", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a new adversarial training scheme, LEAP, to obtain models robust against $\\ell_\\infty$-bounded adversarial examples. The loss used as the objective minimized during training involves both local and global (wrt the input space) properties of the network. Experiments suggest improved performance compared to single and multi-step standard adversarial training and TRADES.\n\nPros\n1. The proposed method achieves in the reported experiments better results than existing methods, for both single and multi-step adversarial training.\n2. The authors provide detailed ablation studies in the appendix.\n\nCons\n1. The final loss used looks like a straightforward combination of TRADES and a regularization term on the norm of the Jacobian matrix at the adversarial points. As mentioned in Sec. C.1, as differences from TRADES, first LEAP uses the adversarial images rather than the clean ones when computing the cross entropy loss, and second it computes the adversarial examples maximizing the cross entropy rather than the KL distance. Both seem minor modifications: the first change should lead in principle to lower clean accuracy (in Table 1 LEAP models have 1-2% lower clean accuracy than TRADES ones on CIFAR-10 and CIFAR-100) and worse trade-off clean vs robust accuracy, while the second one is not discussed in the main paper. Moreover, the regularization of the Jacobian matrix is, as acknowledge in the text, not novel.\n2. Although the experimental results seem to favor LEAP, I have some concerns about the hyperparameters used in PGD at training and test time. For the multi-step adversarial training on MNIST, with $\\epsilon=0.3$, 20 steps of PGD with step size 0.01 are used (Sec. B), meaning that the $\\ell_\\infty$-ball cannot be fully explored by the attack. Conversely, on CIFAR-10 and CIFAR-100 a step size of 0.07 > 17/255 is used with and $\\ell_\\infty$-ball of radius 8/255, which is quite large with a budget of 10 steps. For testing the robustness of the models, the MultiTargeted attack is used: it is a strong attack, but it is given a budget of only 20 iterations, which might be insufficient for the attack to be effective. Increasing the budget for the attacks or using some standard evaluation like that from (Croce & Hein, 2020) would strengthen the results.\n3. When evaluating LEAP on cheap adversarial training, considering that the computational cost is roughly 3x that of one-step ADV, also the method from (Andriushchenko & Flammarion, 2020) should be included in the comparison, as it should have a runtime similar to the proposed scheme and it is shown to outperform one-step ADV.\n\nOverall, I think the novelty is very limited, which combined with some concerns about the experiments makes me lean towards giving a negative score.\n\nCroce & Hein, \"Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks\"\\\nAndriushchenko & Flammarion, \"Understanding and Improving Fast Adversarial Training\"\n\n---\nUpdate after rebuttal\n\nI thank the authors for the response. I think that the revised version improved clarity. The overall impression I have of the paper is still similar to the initial one.\n\nThe final proposed loss is reasonable, but just the combination of two existing ones with minor modifications. About this, even in the revised version the authors seem not to discuss and justify (at least in the main part) how the adversarial point $x'$ in computed at training time, which is different from TRADES according to Section C.1.\n\nAbout the experimental part, I thank the authors for adding the new experiments in Section D. However, the baseline in Table 4 seems a bit weak, at least for 10 steps ADV. For reference, the baseline WRN-28-10 in (Gowal et al., 2020) has robustness under AutoAttack + MultiTargeted close to 51% (I see that here a WRN-28-8 is used, but I wouldn't expect such difference).\nMoreover, although a minor concern, the authors didn't address the question about the step size used on MNIST.\n\nThen, I keep my initial score.\n\nGowal et al., \"Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples\"", "extra_scores": {}, "extra_text": {}}, {"review_id": "74SssrnzRnE", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe authors developed a novel robust training algorithm LEAP to focus on the effective use of adversaries. The proposed method improves the model robustness at each local patch and combines these patches through a global term, achieves overall robustness. The authors showed by maximizing the use of adversaries, they achieved high robust accuracy with weak adversaries. Furthermore, when trained with strong adversaries, the proposed method matches with the current state of the art on MNIST and outperforms them on CIFAR-10 and CIFAR-100.\n\n\nComments:\n\n1 . The authors’ main idea is to promote local patch robustness combined with global robustness. While the local patch robustness is further related to the Jacobian norm, the global robustness is exactly the same as TRADES’ regularization term. My major concern is that the proposed method still does not have a very convincing intuition that why the local patch robustness term or combining the two terms helps. In particular, the LEAP_g algorithm is actually very close to TRADES, why it could achieve better robustness. The authors might want to add more explanations/demonstrative experiments to show that.\n\n\n2 . In eq (5), what does it mean by decision boundary = 0? Also in eq (6) what is standard computation? I assume the authors refer to first-order Tylor expansion? The authors need to reorganize the presentation of this part to make everything clear. And equation (5)/(6) is actually useless since finally, the authors only rely on Proposition 1 to promote the local patch robustness term. For the approximation of the Jacobian norm, the authors might want to briefly introduce some details on how the approximation is done.\n\n3 . The following work also performs robust training directly on x’ instead of x,\n\n\"Improving adversarial robustness requires revisiting misclassified examples.\" ICLR (2019).\n\nThe authors might also want to comment on it.\n\n4 . Notice that even adversarial training based algorithms could cause obfuscated gradient problem, therefore, it might be a good idea to further evaluate model robustness via totally gradient-free methods, such as hard-label attacks \n\n“RayS: A Ray Searching Method for Hard-label Adversarial Attack” KDD (2020)\n\nIn order to make the experimental results more convincing.\n\n\n5 . In section 4.1, “Since l∞ norm is equivalent to l2 norm”. It is fine to only present L2 norm analysis but this statement is not appropriate and may cause confusion. In cases where the data dimension N is large, the difference between the two settings could also be drastically different.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a new method of employing some existing techniques to improve robustness, which was verified through experiments. According to the reviewers’ comments and the authors’ responses to these comments, the reviewers generally appreciate the authors’ effort in properly improving and clarifying the proposed method. However, their major concerns still rely on the novelty of this paper, which is identified as a combination of some existing techniques. In addition, the proposed method at its current status still contains some un-convincing points. Hence, the paper is recommended rejected.
Decision: Reject