Scaleable input gradient regularization for adversarial robustness
Authors: Chris Finlay
Adam M Oberman
Venue:  (2020)
Keywords: adversarial robustness
gradient regularization
robust certification
robustness bounds

Abstract: In this work we revisit gradient regularization for adversarial robustness with some new ingredients.  First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization.  Second, we implement a scaleable version of input gradient regularization which avoids double backpropagation: adversarially robust ImageNet models are trained in 33 hours on four consumer grade GPUs.  Finally, we show experimentally and through theoretical certification that input gradient regularization is competitive with adversarial training. Moreover we demonstrate that gradient regularization does not lead to gradient obfuscation or gradient masking.

Under review as a conference paper at ICLR 2020
SCALEABLE INPUT GRADIENT REGULARIZATION FOR ADVERSARIAL ROBUSTNESS
Anonymous authors Paper under double-bling review
ABSTRACT
In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement scaleable version of input gradient regu- larization which avoids double backpropagation: adversarially robust ImageNet models are trained in 33 hours four consumer grade GPUs. Finally, we show ex- perimentally and through theoretical certification that input gradient regularization is competitive with adversarial training. Moreover demonstrate that gradient regularization does not lead to gradient obfuscation or gradient masking.
INTRODUCTION
Neural networks vulnerable to adversarial attacks. These small (imperceptible to the human eye) perturbations of an image which cause a network to misclassify the image (Biggio et al. 2013 (Szegedy et al. 2013) Goodfellow al. 2014). The threat posed by adversarial attacks must be addressed before these methods can be deployed in eeror-sensitive and security-based applications (Potember 2017).
Building adversarially robust models is an optimization problem with two objectives: (i) maintain test accuracy clean unperturbed images, and (ii) be robust large adversarial perturbations. The present state-of-thh ofetthe-rrt method for adversarial defence. adversarial training (Szegedy et al. 2013
Goodfellow et al. 2014; Tramer et al. 2018 Madry al.] 2017 Miyato et al., 2018). in which models are trained on perturbed images, offers robustness at the expense of test accuracy (Tsipras) al., 2018). Up until the recent method of Shafahi al. (2019b), multi -tep adversarial training had
taken many days to train large datasets (Kannan et al. 2018 Xie al. 2018).
Assessing the empirical effectiveness of an adversarial defence requires careful testing with multiple attacks (Goodfellow et al.) 2018). Furthermore, existing defences are vulnerable to new, stronger attacks: Carlini & Wagner (017a) and Athalye et al. (2018) advocate designing specialized attacks to circumvent prior defences, while Uesato al. (2018) warn against using weak attacks to evaluate robustness. This has led the community to develop theoretical tools to certify adversarial robustness. Several certification approaches have been proposed: through linear programming (Wong & Kolter 2018 Wong et al., (018)) or mixed-integer linear- preeraproggamming (Xiao et al.] (018); semi-definite relaxation (Raghunathan
al. 2018b a); randomized smoothing (Li al.. 2018 Cohen al.
2019); or estimates of the local Lipschitz constant (Hein & Andriushchenko [017]] Weng et al.
2018 Tsuzuku et al. 2018). The latter two approaches have scaled well ImageNet-Ik: Notably, randomized smoothing with ad adversarial training has shown great promise for rigorous certification against attacks in the l2 norm (Salman et al. 2019), although it is not yet clear how randomized
smoothing may be adapted to other norms.
In practice, certifiably robust networks often perform worse than adversarially trained models, which in general lack theoretical guarantees. In this article, we work towards bridging the gap between theoretically robust networks and empirically effective training methods. Our approach relies on minimizing a loss regularized against large input gradients
> 2
E ((((;; w) v) + (x.
V V. (( ())

---

Under review as a conference paper at ICLR 2020
where ||-||= is dual to the measuring adversarial attacks (for example the & norm for attacks measured in the loe norm). Heuristically, making loss gradients small should make gradient based attacks more challenging.
Drucker & LeCun (199I) implemented gradient regularization using 'double backpropagation
which has been shown to improve model generalization (Novak et al. (2018). It has been used to
improve the stability of GANs (Roth et al. 2017 Nagarajan & Kolter (017)) and to promote learning
robust features with contractive auto-encoders (Rifai al. (0011). While it has been proposed for
adversarial attacks robustness (Lyu et al. 2015 Ross & Doshi- Velez 2018 Roth et al. (2018] Ororbia
II et al. 2017; Heein & Andriushchenko/ 2017 Jakubovitz & Giryes 2018)] SSionn 2018:/Simon Gabriel al.
2018). experimental evidence has been mixed, in particular, input gradient regularization has so far not been competitive with multi-step adversarial training.
In particular, the heuristic that "gradients should be small" has motivated several other approaches to adversarial robustness which were later shown to suffer from problems of gradient obfuscation (Athalye al.) (0188). This is the phenomenon whereby gradients are in some sense hidden or mini- mized, so that gradient-basee attacks fail to produce adversarial examples. Due to this phenomenon, the community now greets adversarial robustness methods based on notions of small gradients with a healthy dose of skepticism. It is therefore necessary to demonstrate new methods do not suffer from gradient obfuscation (Carlimi et al., 2019), both empirically and theoretically: empirically, by attacking models with gradient-free methods or non-local attacks; and theoretically using tight certification techniques.
Our main contributions in this work are the following. First, we motivate using input gradient regularization of the loss by deriving new theoretical robustness bounds. These bounds use the gradient of the loss, and an estimate of the error of the linear approximation of the loss, called the modulus of continuity. These theoretical bounds show that small loss gradient and a small modulus of continuity are sufficient conditions for adversarial robustness. Second, we empirically show that input gradient regularization competitive with standard adversarial training (Madry al.) (017)) at fraction of the training time. We verify that training with gradient regularization does not lead to gradient obfuscation, by attacking our trained models with gradient-free methods, and through certification bounds. Finally, we scale input gradient regularization to ImageNet-Ik (Deng et al. 2009) by using finite differences to estimate the gradient regularization term, rather than double backpropagation (which does not scale). This allows to train adversarially robust networks on ImageNet- 1k with only 50% more training time than that of standard (non robust) training.
2
ADVERSARIAL ROBUSTNESS BOUNDS FROM THE LOSS
2.1 BACKGROUND
Much effort has been directed towards determining theoretical lower bounds on the minimum sized perturbation necessary to perturb image that it is misclassified by model. One promising approach, proposed by Hein & Andriushchenko (2017) and Weng et al. (0188) and which has scaled well to ImageNet- 1k, is to use the Lipschitz constant of the model. In this section, we build upon these ideas: we propose using the Lipsehitz constant of a suitable loss, designed to measure classification errors. Additionally, we propose tighter lower bound which uses the local gradient information of the loss, and an error bound on the first-order approximation of the loss.
Our notation is as follows. Write y [x; w) for model which takes input vectors to label probabilities, with parameters w. Let C(y1, be the loss and write ((x) == L(f(z, w), v), for the loss of model f.
Finding an adversarial perturbation is interpreted as a global minimization problem: find the closest image a clean image, in some specified norm, that is is also misclassified by the model
min ||m||| subject to f(x + v) misclassified
(I)
However, (I) is difficult and costly mon-smooth, non-convex optimization problem. Instead, Goodfellow et al. (2014) proposed solving a surrogate problem: find a perturbation of a clean image ve that maximizes the loss, subject the condition that the perturbation be inside norm-ball
2

---

Under review as a conference paper at ICLR 2020
of radius 8 around the clean image. The surrogate problem is written
{o
fo
if ||l|| S 6 otherwise
max ((x v) c(v); where c(v)
The hard constraint c(v) forces perturbations to be inside the norm-ball centred at the clean image x. Ideally, solutions of this surrogate problem 2 will closely align with solutions of the original more difficult global minimization problem. However, the hard constraint in (2) forces particular scale: it may miss attacks which would succeed with only slightly bigger norm. Additionally, the maximization problem (2 does not force misclassification, it only asks that the loss be increased.
The advantage of (2) is that it may be solved with gradient-based methods: present best-practice is to use variants of projected gradient descent (PGD), such as the iterative fast-signed gradient method (Kurakin et al. 2016; Madry et al., 2017) when attacks are measured in the &s norm. However, gradient-based methods are not al always effective: on non-smooth networks, such as those built of ReLU activation functions, a small gradient does not guarantee that the loss remains small locally. That is, ReLU networks may increase rapidly with a very small perturbation, even when local gradients are small. This deficiency was identified in (Papernot et al. 2016) and expanded on by Athaayy et al. (2018). In this scenario. PGD methods will fail to locate these worst-case perturbations, and give a false impression of robustness. Carlini & Wagner (2017b) avoid this scenario by incorporating decision boundary information into the loss; others solve (I) directly (Brendel et al.] 2018 Chen & Jordan 2019 Finlay et al. 2019) using gradient-fre methods or non-local attacks.
2.2 DERIVATION OF THEORETICAL LOWER BOUNDS
This leads us to consider the following compromise between and (2) Consider the following modification the Carlini & Wagner (2017b) loss {(x) = maXi fi(r) - fe(z), where e is the index of the correct label, and [ix)) is the model output for the i-th label. This loss has the appealing property the sign the loss determines if the classification is correct. Adversarial attacks are found by minimizing
min
|||||| subject to f(x + +v) t
The constant to determines when classification is incorrect; for the modified Carlini-Wagner Wagner loss, to 0. Problem (3) is closer to the true problem ( and will always find adversarial image. We use 3) to derive theoretical lower bounds on the minimum size perturbation necessary to misclassify an image. Suppose the loss is L-Lipschitz with respect to model input. Then we have the estimate
f(z + v) < ((x) L|li||
(4)
Now suppose is adversarial, with minimum ad adversarial loss E(x + = lo. Then rearranging we obtain the lower bound |||||| 2 1/ (fg -(x)))
Unfortunately, the Lipschitz constant is a global quantity, and ignores local gradient information; see for example Huster al. (2018). Thus this bound can be quite poor, even when networks have small Lipschitz constant. A tighter theoretical bound may instead be achieved by using (i) the local gradient, and (ii) bound on the error of a first order approximation of the loss. At image x and perturbations up to size e, the error of a first order approximation of the loss is bounded above by the modulus of continuity
(e))=
sup 2, |ells
f(x + v) [f(x)) (v, v((x))
Intuitively, this measures the maximum amount the gradient can change over perturbations of size e. Note that the quantity is defined for any norm. Using this quantity, for perturbations of size have the estimate
f(x + v) s ((x) + (v. v((2)) + weE) {(x)) + [[ +w
(6)
As before, assume v is adversarial, with minimum adversarial loss f(r + v) = lo- Rearranging (). we obtain the lower bound |||||| |[([))] (lo {(x) =())).
3

---

Under review as a conference paper at ICLR 2020
Remark 2.1. If the model and the loss twice continuously differentiable, the modulus of continuity can be derived from the maximum curvature. However w()) is defined even when the model and the loss are not twice differentiable.
We have proved the following.
Proposition 2.2. Suppose the loss ((x) is Lipschitz continuous with respect to model input x, with Lipschitz constant L. Let to be such that if ((x) < to, the model is always correct. Then lower bound on the minimum magnitude of perturbation V necessary to adversarially perturb an image a is
max ((2),,3
||
(L-bound)
L
Suppose in addition the modulus of continuity w(e) is defined as in S). Then the minimum adversarial distance is bounded below by E provided the following inequality holds
6o {(x) w(E) |[
ZE
(w-bound)
motivates the need for input gradient regularization. The Lipschitz constant L is the maximum gradient norm of the loss over all imputs. Therefore ([-bound) says that a regularization term encouraging small gradients (and so reducing I.) should increase the minimum adversarial distance. This aligns with (Hein & Andriushchenko (017)) who proposed the cross-Lipschitz regularizer, penalizing networks with large Jacobians in order to shrink the Lipschitz constant of the network.
Proposition 2.2
However, this is not enough: the gap la (xx) must be large well. Together (Z-bound) explains one form of "gaadient masking' (Papernot et al. 2017). Shrinking the magnitude of gradients while also closing the gap lu - {(z) effectively does nothing to improve adversarial robustness. For example, in one form defense distillation, the magnitude
the model Jacobian is reduced by increasing the temperature of the final softmax layer of the network. However, this has the detrimental side-effect
of sending the model output to (v.... ... ). where N is the number of classes, which effectively shrinks the loss gap to zero. An alternative form sends the temperature of the final softmax layer to zero, which pushes the model output to one-hot vector, however this sends the Lipschitz constant to 00. In either form, the lower bound provided by Proposition 2.2 approaches zero
Moreover, (ou-dound states that even supposing the loss's 's gradient is small, and the gap to - {(x) large, there may still be adversarially vulnerable images nearby due to errors in the first order approximation. Taken together, Proposition] 2.2 provides three sufficient conditions for training robust networks: (i) the loss gap fn {(x) should be large; (ii) the gradients of the loss should be small; and (iii) the error of the first-order approximation, measured by w(). should also be small. The first point will be satisfied by default when the loss is minimized. The second point will be satisfied by training with a loss regularized to penalize large input gradients. In Section we provide experimental evidence that the third point is satisfied with input gradient regularization implemented using finite differences.
These robustness bounds are most similar in spirit to Weng et al. (018)), who derive bounds using an estimate of the local Lipschitz constant of the model by sampling the model gradient norm locally. Indeed, in practice, I. and w(E) can only be estimated and are not exactly computable on all but the simplest networks. We estimate L and w(s) using the Extreme Value Theory approach outlined by Weng et al. (018). To estimate L, we sample the maximum gradient norm over batches from the data distribution, and estimate an upper bound by fitting generalized extreme value (GEV) distribution to these samples through maximum likelihood estimation (MLE). Similarly, we estimate (E)) by sampling the maximum of (5) over batches of the data distribution and || S E, and then fitting GEV using MIEE The values of L. and w()) used in our bounds are estimated from the fitted GEV distribution with threshold p = 0.001. Because these quantities are estimates, in practice the thcoretical bounds of Proposition] can only be implemented approximately, providing a heuristic measure of adversarial robustness. Notably however, these bounds are valid in any norm with corresponding dual. Furthemore, upon estimating L. and w(E), emphasize that these quantities only require one gradient evaluation and one model evaluation to provide an estimate on the robustness of image.
To date, the theoretical bounds in the Euclidean norm that show the greatest promise have arguably been those obtained through randomized smoothing (Li et al. 2018 Cohen et al.) 2019 Salman
4

---

Under review as a conference paper at ICLR 2020
et al. 2019). In randomized smoothing, models trained with inputs perturbed through Gaussian noise; robust prediction is achieved by averaging model predictions over inputs again perturbed with Gaussian noise, which requires many tens of thousands of model evaluations. It is well known that training with Gaussian noise is equivalent to squared lz norm gradient regularization (Bishop) 1995). Therefore expect that models trained with squared norm gradient regularization, should have similar adversarial robustness as those trained through randomized smoothing.
SQUARED NORM GRADIENT REGULARIZATION
Proposition 2.2 provides strong motivation for input gradient regularization method for promoting adversarial robustness. However, it does not tell us what form the gradient regularization term should take. In this section, we show how norm squared gradient regularization arises from quadratic cost.
In adversarial training, solutions of (2) are used to generate images on which the network is trained. In effect, adversarial training seeks a solution of the minimax problem
min E mmxx + e; w) -c()
u amp where TP is the distribution of images. This is robust optimization problem (Wald 1945 Rousseeuw & Leroy 1987). The cost function c(v) penalizes perturbed images from being too far from the original. When the cost function is the hard constraint from (2. perturbations must be inside a norm ball of radius 8. This leads to adversarial training with PGD (Kurakin et al., 2016 Madry et al. 2017). However this forces particular scale: it is possible that no images are adversarial within radius 6, but that there are adversarial images with only a slightly larger distance. Instead of using hard constraint, we can relax the cost function to be the quadratic cost c(v) 23 ||ve|/2. The quadratic cost allows attacks to be of any size, but penalizes larger attacks more than smaller attacks. With quadratic cost, there is less of a danger that local attack will be overlooked.
Solving 77 directly is expensive: on ImageNet-Ik, lk, both Kannan et al. (2018) and Xie et al. (2018) required large-scale distributed training with many dozens or hundreds of GPUs, and over a week of training time. Recent research suggests this hurdle may be surmounted (Shafahi et al. 2019b Zhang
(et al. 2019). However we take the view that 7 may be bounded above, and solved approximately When the loss is smooth, the modulus of continuity is w(s) =*C,, where C is the maximum positive Hessian norm over all inputs. By using the bound (6) with the quadratic loss c(v) = 28 |e: the 5 optimal value of max,, f(x + u) <(v) is 21- V ((x) provided 8 < C. This gives the following proposition.
Proposition 3.1. Suppose both the model and the loss are twice continuously differentiable. Suppose attacks are measured with quadratic cost 1 ||u||/ Then the optimal value of is bounded above by
28
min E x>P
((x;w) + 2 {x))
where X- I-8C-
That is, we may bound the solution of the adversarial training problem 7 by solving the gradient regularization problem (8), when the cost function quadratic. not necessary to know 8 or compute C; they are absorbed into X.
Input gradient regularization using the squared lg norm was proposed for adversarial robustness in Ross & Boshi-Velez (2018). It was expanded by Roth et al. (2018) to use a Mahalanobis norm with the correlation matrix of adversarial attacks.
When c(v) is the hard constraint forcing attacks inside the d norm ball and C' is small, supposing the
curvature term is negligible, we can estimate the maximum in (7) by {(x) + } v E(x)Il using the dual norm for the gradient. This is norm gradient regularization (not squared), and has been used for adversarial robustness on both CIFAR-10 "Simon-Gabriel et al. 2018), and MNIST (Seck et al. 2019).
3.1
FINITE DIFFERENCE IMPLEMENTATION
Norm squared input gradient regularization has long been used as a regularizer in neural networks:
Drucker & LeCun (191] first showed its effectiveness for generalization. Drucker & LeCun
5

---

Under review as a conference paper at ICLR 2020
Percent Misclassified
100
Percent Misclassified
100
75
75
undefended
=
50
50
A=1
7-step
25
25
0-
10
10 10-1
100
10
4
10 10 10
1/2 distance
t. distance
(a) l norm adversarial attacks
(b) Eo norm adversarial attacks
Figure 1: Adversarial attacks on the CIFAR-10 dataset, on networks built with standard RcLUs. Regularized networks attacked in & are trained with squared l2 norm gradient regularization: networks attacked in fx are trained with squared & norm regularization.
implemented gradient regularization with 'double backpropagation compute the derivatives of the penalty term with respect to the model parameters which is needed to update the parameters during training. Double backpropagation involves two passes automatic differentiation: one pass to compute the gradient of the loss with respect to the inputs x, and another pass the output of the first to compute the gradient of the penalty term with respect to model parameters w. In neural networks, double backpropagation is the standard technique for computing the parameter gradient of a regularized loss. However, it is not currently scaleable to large neural networks. Instead we approximate the gradient regularization term with finite differences.
Proposition 3.2 (Finite difference approximation of squared l2 gradient norm). Let d be the normal- ized input gradient direction: = V ((x)/1l Va E(z)]l2 when the gradient is nonzero, and set otherwise. Let h be the finite difference step size. Then, the squared Eg gradient norm is approximated by
[(x [ hd) {(x)
V 1(x) ~
The vector is normalized to ensure the accuracy the finite difference approximation: as can be seen by (6. the error of the finite difference is bounded by wh)/h. Without normalizing the gradient, this term can be large, which is undesirable in general. However, the finite difference error term does play role in promoting adversarial robustness, Proposition 2.2 shows it is necessary for w(E) to be small. By using finite differences, minimizing 8 also implicitly minimizes (h), due to the error term. Therefore the finite difference approximation provide indirect means for controlling the modulus of continuity w(s) necessary for adversarial robustness, through the finite difference error. This would otherwise be unavailable with an exact method computing the gradient norm.
The finite differences approximation ( allows the computation of the gradient of the regularizer (with respect model parameters w) be done with only two regular passes of backpropagation. rather than with double backpropagation. On the first, the imput gradient direction is calculated. The second computes the gradient with respect to model parameters by performing backpropagation on the night-handssie of 19. Double backpropagation is avoided by detaching d from the computational graph after the first pass. In practice, for large networks, we have found that the finite difference approximation of the regularization term is considerably more efficient than using double backprop- agation. On networks sized for MNIST, we have observed roughly a 10% speed improvement by using finite differences; CIFAR-10 10 networks finite differences roughly 50% faster than double backpropagation.
Our proposed training algorithm, with squared Euclidean input gradient regularization, is presented in Algorithm [ of Appendix A Other gradient penalty terms can be approximated as well. For example, when defending against attacks measured in the Lxo norm, the squared e, norm penalty can approximated by setting instead d SVgn(Vz (())) / when the gradient is nonzero.

---

Under review as a conference paper at ICLR 2020
Table 1: Adversarial robustness statistics, measured in the &oo norm. Topl error is reported on CIFAR-10; Top5 emror on ImageNet- Ik. On CIFAR-10 gradient regularization is reported using either double backpropagation (DBP) or finite differences (FD).
% clean eiTor
w-bound % error
empirical % error at
training time (hours)
2
265
256
206
CIFAR-10
Undefended 6 7-step AT, 5 2/255
4.36
100
100
70.82
98.94
2.06
6.17
16.02
100
16,80
53.90
10.82
Lx 7-step AT, e 8/255
16.33
26.59
58.32
22.86
46.02
12.10
Lx 'For Free' AT. m
6.82
20.08
22.61
63.95
1.91
6x 'For Free' AT, m =
19.94
38.52
100
34.09
61.48
2.04
squared & norm (DBP), = 0.1
6,50
25.69
100
41.63
97.31
9.85
squared & norm (DBP), À
12.62
33.05
99.43
33.95
80.37
9.89
squared & norm (FD), = 0.1
6.45
19,40
100
24.92
70.41
5,22
squared & norm (FD). 1. 0.1
9.02
20.66
55,30
18.47
58.69
5.15
ImageNet-1k
Undefended
6.94
100
100
90.21
98.94
20.30
squared norm, 0.1
7.66
74.7
100
70.56
97.53
32.60
squared norm,
10.26
63.7
100
52.79
95.93
33.87
4 EXPERIMENTAL RESULTS
In this section provide empirical evidence that input gradient regularization is effective tool for promoting adversarial robustness. We attack networks with gradient-free attacks, and demonstrate gradient regularization does lead gradient obfuscation. Moreover our results show that gradient regularization is comparable to adversarial training, both experimentally and through rigorous certification.
We train networks on the CIFAR- 10 dataset (Krizhessky & Hinton 2009). and ImageNet- 1k (Deng al., 2009). On the CIFAR dataset we use the ResNeXt-33 (2x32) architecture; on ImageNet-Iee Ik we use a ResNet-50 (He et al. 2016). The CIFAR networks were trained with standard data augmentation and learning rate schedules on a single GeForce GTI 1080 Ti. On ImageNet- Ik, we modified the training code Shaw et a submission to the DAWNBench competition (Coleman et al. 2018) and train with four GPUs Training code and trained model weights will be made available.
We train an undefended network baseline to compare various types of regularization. On CIFAR- 10, networks are trained with squared &2 and squared & gradient norm regularization. The former is appropriate for defending against attacks measured in lz; the latter for attacks measured in lx We set the regularization strength to be either À = 0.1 or I; and set finite difference discretization 0.01 unless otherwise specified. (For study of the effect of finite difference size h, refer to Table 4) We compare each network with the most common form of adversarial training, where models are trained using the hyperparameters in Madry et al. (2017) (7-steps of FGSM, projected onto an loo ball of either radius 755 or 255 On CIFAR-10 we also compare against the 'For Free' implementation of Shatahi al. (2019a), using the same hyperparameters. On ImageNet-Il we only train adversarially robust models with squared b regularization.
On each dataset, we attack 1000 randomly selected images. We perturb each image with attacks in both the Euclidean and L norms, with a suite of current statc-of-the- attacks: the Carlini-Wagner Wagner attack (Carlini& & Wagner 2017b); the Boundary attack (Brendel et al.) 2018) (a gradient- free attack); the LogBarrier attack (Finlay et al., 2019) (a non-local attack); and PGD (Madry et al.) (017) (in both the Eoo norm or the & norm). The former three attacks have all been shown effective at evading gradient masking and gradient obfuscation. PGD excels at finding images close to the original when gradients are not close to zero. We report our results using the best adversarial distance on a per image basis, in each norm. On the majority of test images, PGD finds the closest adversarial image. However on a significant portion of inputs, the closest adversarial image is found by one of the three other attacks.
7

---

Under review as a conference paper at ICLR 2020
Table Adversarial robustness statistics, measured in lg. Topl error is reported CIFAR-I0; Top5 error on ImageNet- 1k. On CIFAR-10 gradient regularization is reported using either both double backpropagation (DBP) or finite differences (FD).
% clean eтror
mean adversarial distance
training time (hours)
L-bound
w- bound
empirical
CIFAR-10
Undefended
4.36
0.006
0
2.06
Ls 7-step AT, & = 2/255
6.17
0.05
0.36
0.66
10.82
Ls 7-step AT, 8/255
16.33
0.18
0.55
0.74
12.10
Lx 'For Free' AT, 2, À 0.1
6.82
0.06
0.35
0.53
1.91
( 'For Free' AT,
19.94
0.03
0.29
0.49
2.04
squared la norm (DBP), À
5.91
0.006
0
9.60
squared & norm (DBP), À
4,35
0.01
0
0.15
9.74
squared &2 norm (FD), À = 0.1
8.03
0.14
0.34
0.63
5.18
squared & norm (FD), À = ImageNet-1k
20.31
0.30
0.50
0.81
5,08
Undefended
6,94
3. 3.63e-2
0.5
0.55
20.30
squared & norm, À 0.1
7.66
0.13
0.96
1.14
32.60
squared & norm, À
10.26
0.26
1.23
1.75
33.87
Adversarial robustness results for networks attacked in the to norm presented in Table These results are for networks built of standard ReLUs. Table and Figure @ a clear trade-off between accuracy and adversarial robustness, as the strength of the regularization is increased. On CIFAR-10, the undefended network achieves test error of 4.36%, but is not robust to attacks even foc distance 255 However with strong regularization parameter (A 1), test error increases 9.02% on clean images. At both adversarial distance e 255 and 255 adversarial training achieves slightly better robustness (by about 5% percent) than best models, comes over double the training time.
In Table also report heuristic measures of robustness using (w-bound). We skip reporting bounds using (o-ndd,)), for this bound returned vacuous results (an upper bound of 100% test error at all distances). On some test images the heuristic bound fails; this is due to the inherent difficulty of estimating w(E) accurately.
In Table 2 report results models trained for attacks in the lg norm. On CIFAR-10, the most robust model trained with regularization strength À = 1, and outperforms even the adversarially trained model. On ImageNet-lk, the best protection against adversarial attacks. Due to the long training time, we were not able train ImageNet-IK with standard multi-step adversarial training. In Table 2 we also report our heuristic measure of bounds on the minimum distance required to adversarially perturb, using the Carlini-Wagner loss Further w-bound results on CIFAR 10 are presented in Table Both (L-bound) and (w-buund) provide reasonable estimates of the minimum adversarial distance, though (w-uound) closer empirical results. These heuristic measures of robustness are further corroborated by Table of Appendix B which shows that both adversarial training and gradient regularization significantly decrease model gradients and the modulus continuity wE). also use randomized smoothing, tight and exact bound robustness, to verify our heuristic measures, see Table
the same pattern: the model trained with X 1 offers
CONCLUSION
We have provided motivation for training adversarially robust networks through input gradient regularization, by bounding the minimum adversarial distance with gradient statistics of the loss. We have shown that gradient regularization is scaleable to ImageNet-Ik, and provides adversarial robustness competitive with adversarial training, both theoretically and empirically. Moreover we have demonstrated that gradient regularization does not lead gradient obfuscation. Networks
This loss he modified for Top-5 mis-classification as well.
8

---

Under review as a conference paper at ICLR 2020
may be trained using gradient regularization in fractions of the training time of standard adversarial training.
We gave thcoretical per-image bounds the minimum adversarial distance, for non-smooth models, using the Lipschitz constant of the loss, and its modulus of continuity, which measures the error of a first order approximation of the loss. These bounds were empirically compared against otate-of-the-art attacks and other theoretical bounds. Although in practice these bounds give heuristic measures of robustness, they are valid in any norm, and provide per-image robustness measures at the cost of only one model evaluation and one gradient evaluation.
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Jennifer Dy and Andreas Krause (cds.). Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 274-283, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. URL http /ppposeeiinggaampsppas///////aaa http://mmapppp////////aaaaaaaala
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Sindić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning test time. In Hendrik Blockeel. Kristian Kersting, Siegfried Nijssen, and Filip Zelezný (eds.), Machine Learning and Knowledge Discovery in Databases, pp- 387-402 Berlin, Heidelberg, 2013. Springer Berlin Heidelberg. ISBN
Christopher M. Bishop. Training with noise equivalent to Tikhonov regularization. Neural Compu- tation, 7(1):088 108-110 1995. doi: 10.1162/ncco. 2..1162/hcoo.995.7.... 1995.7.1.108. URL https: http:::://doloor/1///0 10. neco 1 1995777 7.000
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018. URL
IGGWZ OGNCZ
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, A/Sec@ CCS 2017, Dallas, TX, USA, November 3, 2017, pp. 3-14, 2017a. doi:
URL
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness ncural networks. 2017 IEEE Symposium on Security and Privacy, SP 2017, San Joxe, CA. USA, May 22-26, 2017, pp. 39-57, 2017b. URL
htttos://dci.or
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian J. Good- fellow, Aleksander Madry, and Alexcy Kurakin. On evaluating adversarial robustness. CoRR, abs/ 0hss/190 1902.06705, 2019. URL/http: http:::://xxvvov// www.aaawwwwwoooooaaaaao..oooeeao
abs abs 19020060020
Jianbo Chen and Michael I. Jordan. Boundary a Query-efficient decision-based adversarial attack. CORR, ahs/1904.02144 2019. URL http:tppp//arrioorrg////////bb.p
Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robusteess via randomized smoothing. In Proceedings of the 36th International Conference Machine Learming, JCML 2019, 9-15 June 2019, Long Beach, California, USA. pp. 1310-1320, 2019. BBLL:ttee/pppccceei.......esses
cohen 9c.html
Cody Coleman, Daniel Kang, Deepak Narayanan, Luigi Nardi, Tian Zhao, Jian Zhang, Peter Bailis, Kunle Olukotun, Christopher Ré, and Matei Zaharia. Analysis of dawnbench, a time-to-accuracy machine learning performance benchmark. CoRR, 0tts/180.00144 2018 URL/EEEpp///axxxxxor///a//////
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Computer Society Conference Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pp. 248 255, 2009, doi: 109//VRR 22009 22009, 520694. URL http:://e...o11g11111011100//0.0
Harris Drucker and Yann LeCun. Double backpropagation increasing generalization performance. In IJCNN-91- Seattle International Joint Conference on Neural Networks, volume 2, pp. 145-150. IEFE, 1991.

---

Under review as a conference paper at ICLR 2020
Chris Finlay, Aram-Alexandre Pooladian, and Adam M. Oberman. The LogBarrier adversarial attack: making effecttive use of decision boundary information. CoRR, abs/1903. 2019. URL htl.http//axxxv.rrgg
abs/L903.1033
Ian Goodfellow, Patrick McDaniel, and Nicolas Paperoo. Making machine learning robust against adver- sarial inputs. Communications of the ACM, 61(7)566-6, June 2018. URL. httt:://dl.acm.orr
c1 tat wow.ooocmddooid2223333 9.314999
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and hamesssing adversarial examples. CoRR, abx/1412.6572, 2014. UtL.Ettt://azzxi.rrrg 14112///62 141226572
Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Computer Vision ECCV 2016 14th European Conference, Amstendam, The Netherlands, October 11-14, 2016, Proceedings, Part IV. pp. 630-645, 2016. URL hattp:://doi.rrg/D.... wtt.ooogg/...007//118.8. 319 319- 46493-0 38
Matthias Hein and Maksym Andriushchenko. Formal Guarantees on the Robustness of Classifier against Adver- sarial Manipulation. Advances in Neural Information Prucessing Systems 30: Annual Conference Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 2263 2273, 2017. URL UtL/tttpe//pppprrsaas.../op/pppp http:/cccpppppe/811--------
Saaprrraeaaaaaotoaaoaaaaaaaaaaee -uuaranteuuauaaeeeeeeeeee on- the robustness
01
Todd Huster, Cho- Yu Jason Chiang, and Ritu Chadha. Limitations of the lipschitz constant as defense against adversarial examples. In ECML PKDD 2018 Workshops Nemesis 2018, UrbReas 2018, SoGood 2018, /WAISe 2018, and Green Data Mining 2018, Dublin, Ireland, September 10-14, 2018, Proceedings, pp. 16-29, 2018. doi: 10. 100000978---0000 1-000-33--00--3333-2, / 13453-2 2
Daniel Jakubovitz and Raja Giryes. Improving DNN robustness to adversarial attacks using jacobian regu- larization. In Computer Vision ECCV 2018 15th European Conference, Munich, Germany, September 8-14. 2018, Proceedings, Part XII, pp. 541, 2018. doi: 10.1007/978--300000122888 030-01250-0----0
URL
/ttt::///dii.oog10000000000
32
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. CoRR, abs/1803.06373, 2018.
RRL/ittp://axxxvvcrrs abs//////aaaa 0aaib///88330033 1803.06373
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. CoRR, abs/1607.02533, 2016. /ttL:ttp:://axxwwcc/////////1/1/
Bai Li. Changyou Chen, Wenlin Wang. and Lawrence Carin. Second-order advrrarari attack and certifiable robustness. CoRR, abs/1809.03113, 2018. URL htl/:tttp///arrrsioo//////////// 0000////0
Chunchuan Lyu, Kaizhu Huang, and Hai-Ning Liang. A unified gradient regularization family for adversarial examples. In 2015 IEEE International Conference on Data Mining, ICDM 2015, Arlantic Ciry, NJ. USA, November 14-17, 2015, pp. 301-309, 2015. doi: D0.1009/CDM..215..00 URL https::/doi.ors L.0.109/ICDM.20135
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep leamingg models resistant to adversarial attacks. Co abs/1706.00683,, 2017. URI, http:://axxvv.rr// aass 33os/1706.6093]
Takeru Miyato, Shin-ichi Maeda, Shin Ishii, and Masanori Koyama. Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 2018.
Vaishnavh Nagarajan and J. Zico Kolter. Gradient descent GAN optimization is locally stable. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 5591 5600, 2017. URL BtLLLttppp/ppppprr.......o paver meper/7142-graad ent descent gan wwt.miptmmm............oo.....m. zat 1 1es.alooo-----all----------
Roman Novak. Yasaman Bahri, Daniel A. Abolatia, Jeffrey Pennington, and Jascha
Sensitivity and generalization in neural networks: an empirical study. In 6th International Conference on Learning Representations, ICLR 2018, Vancoveer BC, Canada, April 30 May 2018, Conference Track Proceedings, 2018. ht.tttss::/ htt.httts///ooeeeeiieeeeee /ttp://////pppeeeeeeeeeee///////
10

---

Under review as a conference paper at ICLR 2020
Alexander G. Ororbia Daniel Kifer, and C. Lee Giles. Unifying adversarial training algorithms with data gradient regularization. Neural Computation, 29(4):67-8877 2017, doi: 10. 1,.116//EEOO URL
Nicolas Paperot,, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami.
The limitations of deep learning in adversarial settings. In IEEE European Symposium on Security and Privacy, EuroS&P 2016, Saarbrüccen,, Germany, March 21-24, 2016, pp. 372-387, 2016. URL URLhttps
//dpi.crg/10.ni Btt.00///.roPP.P.1.... /
Nicolas Papernot, Patrick D. McDaniel, Ian J. Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, AsiaCCS 2017, Abu Dhabi, United Arab Emirates, April 2-6, http://.....01111113100022200.00 00444455555297333330
2017. pp. 519,,1191500 2017. URL
Richard Potember. Perspectives research in artificial intelligence and artificial general intelligenne relevant DoD, Technical report, The MITRE Corporation McLean United States, 2017. URL https://fas.org/
http/agnncy//c//// a aadd////oodddddd
Aditi Raghunathan, Jacob Steinhardt, and Perey Liang. Certified defenses against adversarial examples. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings, 201 8a. URL httppppss//oopppt.....eeeeee/eee
id=
Bys Syy44cb-pp Rb
Aditi Raghunathan, Jacoh Steinhardt, and Perey S. Liang. Semidefinite relaxations for certifying rohustness to ad- versarial examples. In Advances in Neural Information Processing Systems 31: Annual Conference Neural Information Processing Syxtems 2018, NewrlPS 2018, 3-8 December 2018, Montréal, Canada.. pp. 10900-
10910, 2018b. http:pppp///pprrrapppppppppppppp Stt.eraa......appp..p.p..aaaaaaa htt.://errr222-seeeeeeeeeiieeeee For www.cettiiyyii to www.eeeeeaaaaaaaaaaaaa..ee.e.. examples
Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedingy of the 28th International Conference on Machine Learning, ICML, 2011, I, Bellevue, Washington, USA, June 28- July 2, 2011, pp. 833-840, 2011. URL
https: /tt::///cmmmmomm wct.ccc//c0000. /ttp:///pppppppSSS1111 cmlpaper.ppf
Andrew Slavin Ross and Finale Doshi Velez. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances Arrificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp. Ocs index.pnp AAAI AAAIIE paper/ /verrvvieeee /iew///wieee/// 17337
2018. URL https:/ /tw.//awi.orrg////////////// UtLatpps///w..aa..rr)
Kevin Roth, Aurélien Lucchi, Sebastian Nowozin, and Thomas Hofmann. Stabilizing training of generative adversarial networks through regularization. In Advances in Newral Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 2015-2025, 2017. URL http///ccce//ppppppppppp..c/////
stabilizi -et regulouggggrruurrraarrra
Kevin Roth, Aurélien Lucchi, Sebastian Nowozin, and Thomas Hofmann. Adversarially robust training through structured gradient regularization. CoRR, a0s./805,,,7.6, 2018. URL http::pp//aaxiiiorrg...
abs 0bs/1805.08736
Peter J Rousseeuw and Annick Leroy. Robust regression and outlier detection, volume 1. Wiley Online Library, 1987.
Hadi Salman, Greg Yang, Jerry Li, Pengehuan Zhang, Huan Zhang, Ilya P. Razenshteyn, and Séhastien Buheck. Provably robust deep learning via adversarially trained smoothed classifiers. CoRR, has//196.04584, 2019. URL
abs 1906.04584
Ismaila Seck, Gaëlle Loosli, and Stephane Canu. Ll-norm double backpropagation adversarial defense. arXiv preprint 0rXiv:1903.0177 2019.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. Adversarial Training for Free! April 2019a. URL /tttps://arrxiv.rr.
abs 0tte/1904.22844
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John P. Dickerson, Christoph Studer. Larry S. Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! CoRR, Abs/1994.1228 12843, 2019h. URI.
1284
11

---

Under review as a conference paper at ICLR 2020
Andrew Shaw, Yaroslav Bulatov, and Jeremy Howard. ImageNetin 18 minutes. wtt.attta::///ggtbbbbommm
diux-dev/ 1atev/imaaeeee.00
Carl-Johann Simon-Gabrie Yann Ollivier, Bernhard Schölkopf. Léon Bottou, and David Lopez-Paz. Adversarial vulnerability of neural networks increases with input dimension. CoRR, abs/1802.01421, 2018. URL
http:://arxlv.org/abss/ (b00g//0aaaa33312222
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, J. Goodfellow, and Rob Fergus. Intriguing properties neural networks. CoRR, abs/1312.6199, 2013. URL http://axxxvv.rr////
0.5 00o//312.6199
Florian Tramèr, Alexcy Kurakin, Nicolas Papernot, lan Goodfellow, Dan Bonch, and Patrick McDaniel, Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Representations, 2018.
wttp:ummmiirkzzzz R2
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom. Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. CoRR, abs/1 1805 12152, 2018. URI http:://axxiv.rrg/eeee 1tt.b/////.111111
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. mipsnnitz-margin training: Scalable cer- tification of perturbation invariance for deep neural networks. In Advances in Neural In- formation Processing Systems 31: Annual Conference on Neural Information Processing Sys- tems 2018, NeurlPS 2018, December 2018, Montréal, Camada., pp. 6542-6551, 2018. http://///.aaatoeaooeeeeeeaaaaan
URL. seriiccatica-----eettaaae itw.:uurraaaiioonvavvaeeeeeeee iewttiiiiiiiiiiii
sca lable
for deep dew-deee-eeuaaa..e -ew.-----e-----eeewto..
Jonathan Uesato, Brendan O'Donoguue, Pushmeet Kohli, and Aäron van den Oord. Adversarial risk and the dangers of evaluating against weak attacks. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmüssson, Stockholm, Sweden, July 10-15, 2018, pp. 5032-5041, 2018. URL
http://ppceeuuuuaaaapepse/////os ha.htm
Abraham Wald. Statistical decision functions which minimize the maximum risk. Annals of Mathematics, pp. 265-2800 1945.
Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsich, and Luca Daniel. Evaluating robustness neural networks: An extreme value theory approach. 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC. Canada, April 30 May 3, 2018, Conference Truck Proceedings, 2018.
www.:/ewwwwee//ounn/eeoo Formm? -aiiddd---HB--.A
Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer ad- versarial polytope. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018. Stockhollsmmässan Stockholm, Sweden, July 10-15, 2018, pp. 5283 5292, 2018. URL http
//proseed ngs.m
Erie Wong, Frank R. Schmidt, Jan Hendrik Metzen, and Zico Kolter. Scaling provable adversarial defenses. In Advances Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, 3-8 December 2018, Montréal, Canada., pp. 8410 8419, 2018. URL
rohustness verification via inducing relu stability. CoRR, -ta. -lf-aaseeeeeeeeeee-e---eee
Kai Xiao, Vincent Tjeng, Nur Muhammad Shafiullah, and Aleksander Madry. Training for faster adversarial 2018. URU.http://arxivorrg/
Bbs/1809.0300g
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L. Yuille, and Kaiming He. Feature denoising for improv- 11.../11111311
ing adversarial robustness. CoRR, abs/1812.03411, 2018. htt.:httt///aii..cc.o
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once:
Accelerating adversarial training via maximal principle. CoRR, abs/ 10077777 2019. URL http:/7
arxiv.org/abs 11tp//9000007777
12

---

Under review as a conference paper at ICLR 2020
A TRAINING ALGORITHM
Algorithm Training with squared f--norm input gradient regularization, using finite differences
1: Input: Initial model parameters wo
Hyperparameters: Regularization strength X; batch size m; finite difference discretization h
2: while WN not converged do
3: sample minibatch of data {<x(9) Sil..... from empirical distribution
4: for 0 to do
5:
(6) V.l(x(1)
(8); wi)
if (0) 40
6: d( T
for &--norm normalized signed gradient
otherwise
7: detach d()) from computational graph
8: 2(6) (6) hd(
9:
end for
10:
((w)
1 m Di=1 ((x(1) (1(0;;
R(w) J - No (1((0,,(()))) - e(x01)) w)))
11:
12:
W+++1 f wi Te V. (C(w:) + >R((w))
13:
end while
B ADDITIONAL EMPIRICAL RESULTS
Percent Misclassified
100
Percent Misclassified
100
75
75
undefended squared squared norm,
50
0
10 1
10
2
10 -1
1 distance
f. distance
(a) &2- norm adversarial attacks
(b) foc norm adversarial attacks
Figure 2: Adversarial attacks on ImageNet-I 1K with the ResNet-50 architecture. Top5 error reported.
13

---

Under review as a conference paper at ICLR 2020
Table 3: Regularity statistics on selected models, measured in the & norm. Statistics computed using modified loss m fa(z) fc(x).
V {(x)
((0.5)
w(10)
mean
max
CIFAR-10
Undefended foo 7-step AT, 2/255
3.05
122.34
1.80
3.13
0.63
11.96
0.08
0.32
foo 7-step AT, E = 8/255 foo 'For Free', m
0.40
2.52
0.02
0,06
0.75
10.82
0.13
0.35
foc 'For Free', m = 8
0.93
16.08
0.11
0.31
squared l2 norm (DBP),
3.38
107.13
1.60
1.40
squared E2 norm (DBP), À
2.75
56.09
1.31
2.30
squared le norm (FD), À 0.1
0,58
4.43
0.08
0,50
squared f2 norm (FD), X =
0.35
1,33
0.02
0.05
ImageNet-1k
Undefended
1.12
17.51
1.7e-2
6.7e-2
squared l2 norm, À 0.1 squared norm, À =
0.46
4.85
5.5e-3 3
2.2e 2
0.27
2.12
3.4e-3
2.1e-2
Table 4: Study of empirical robustness in the fxx norm function of finite difference size h. All models here trained with squared & norm gradient penalty, with regularizer strength 1.
% clean eиror
empirical % error at
2
255
Undefended
4.36
70.82
98.94
double backpropagation
12.62
33.95
80.37
h 1e-4
39.68
50.95
84.20
h 1e-3
25.84
38.81
78,65
h = le-2
14.47
26.54
69.91
h 0.1
9.02
18.47
58.69
h = 1
5.51
45.93
83.86
Table 5: Certified error (%) various & radii on CIFAR-10, using the randomized smoothing certification technique Cohen et al. (2019).
0.25
0.5
0.75
1.0
1.25
1.5
1.75
2.0
Undefended
90
92
96
99
100
100
100
100
squared & norm, À
47
63
80
90
90
90
90
90
Cohen et al. (2019)
40
57
68
77
83
86
88
90
Salman el al. (219))
26
43
52
62
67
71
75
81
Table Certified test error (%) at various la radii on CIFAR-I0, using (obunndd
0.25
0.5
0.75
1.0
1.25
1.5
Undefended Loo 7-step AT, 2/255
100
100
100
100
100
100
19
100
100
100
100
foc 7-step AT, 8/255
31
47
61
77
100
100
squared & norm (FD), À = 0.1
22
99
100
100
100
100
squared &g norm (FD), A=
33
50
65
80
98
100
14


Reviews:
[{"review_id": "ryeFpq8mYS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper provides new understandings on adversarial robustness from the perspective of input gradient regularization. Input gradient regularization hasn't been able to achieve comparable robustness to  adversarial training. Built upon existing works, this paper derives two minimum perturbation bounds (L-bound and w-bound) to explain this, as well as other defenses such as Lipschitz regularization and defensive distillation. Taking a step further, this paper proposes to use the finite difference to estimate the input gradients, which not only gives a nice property for reduced modulus of continuity (eg. the w-bound), but also makes the regulation scalable to large networks and datasets. I quite like the theoretical connections derived in this paper. Empirical evidences support their claims, and demonstrate indeed comparable robustness of input gradient regularization to adversarial training.\n\nThe empirical results can be strengthened by including the normal input gradient regularization baseline (using double backpropagation), at least on cifar-10. This is less likely to change the conclusions, but  would be interesting to see the comparisons.\n\nNote that, there are already new progresses in adversarial training:\n[1] Wang, Yisen, et al. \"On the Convergence and Robustness of Adversarial Training.\" ICML, 2019.\n[2] Zhang, Hongyang, et al. \"Theoretically principled trade-off between robustness and accuracy.\" ICML, 2019.\n[3] Carmon, Yair, et al. \"Unlabeled data improves adversarial robustness.\" NeurIPS, 2019.\n[4] Uesato, et al. \"Are Labels Required for Improving Adversarial Robustness?\" NeurIPS, 2019.\n\n==========\nAfter rebuttal:\nThanks for the new results. My rating remains the same.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "rJecA7_sKr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper revisits the method of gradient regularization, which regularizes the loss function by adding the norm of the input gradient, aiming to improve adversarial robustness. The standard gradient regularization is implemented with \"double backpropagation\", which could be time consuming for large networks. This paper proposes to replace the exact gradient by its discretization via finite difference, which is computationally more efficient comparing to \"double backpropagation\".  Experiments are conducted to show the effectiveness of the method in reducing training time. \n\nOverall, the paper is well written but the contribution is very limited and I find some of the experimental comparisons unfair.  Thus I do not support publication of the paper. Here are my detailed arguments.\n\n1) The gradient regularization is not novel\nThe main contribution of the paper is to use the finite difference in the gradient regularization in order to improve the training time.  The method of gradient regularization is not new, hence the contribution is only the computational effectiveness. This would be fine if one could improve the state-of-the-art method's training time by a lot, but according to the experiments, the performance of adversarial robustness is far from the adversarial training, for example in CIFAR10 epsilon=8/255, there is a 12% drop, which is a huge gap. \n\n2) Unfair comparison in the experiments\nIn table 2, the performance of robustness with respect to the l2 norm is presented. However, in the baseline method of adversarial training, the used attack is in l_{infty} norm. This is unfair since l2 norm and l_{infty} norm has very different characteristics. Moreover, the gradient regularization uses l2 norm instead of l_{1} as in table 1, which makes the comparison unfair. \n\nFurthermore, the finite difference improves the training time versus the standard gradient regularization, but it does not imply that the performance will be the same. In particular, it would be better to include the performance of standard gradient regularization in table 1 and 2 as well. \n\n3) Comment on the motivation/theory\nThe theory part is fairly straightforward and it clearly shows that the norm of gradient (w.r.t input) itself is not sufficient to guarantee robustness. As a evidence, even under standard training (for example on MNIST), the gradient norm could be very small, in order of 10^{-4} but still have adversarial example with very small perturbation.  Thus, what we also need is to control how fast this gradient changes (Lipschitz constant or w-bound). However, the gradient norm regularization does not take into account how gradient changes. It is claimed that the finite discretization implicitly reduces how the gradient changes, however, I am not convinced by the argument since as long as we take h to be small, it is still a very local measure. An interesting question would be how h affect the performance and it is not discussed in the paper.  ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "Bker_uuRYS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed new regularizer for training robust models that can defend against evasion attack/adversarial examples. It looks to me there are two major novelties here, the authors suggest that 1) use dual norm on the gradient/jacobian as regularizer is tighter/better than the same norm or simply l2 norm; 2) the gradient/jacobian can be estimated by finite difference when moving the weight with a small step towards the gradient direction. \n\nI review the paper with a standard for empirical paper. Please kindly clarify the theoretical contributions if the authors thought the theory is novel and important. \n\nI donot think the empirical results are ready for publication. The final objective, when plugin finite difference into the norm regularizer, looks like logits squeezing https://openreview.net/forum?id=BJlr0j0ctX, or logits pairing https://arxiv.org/abs/1803.06373. Both methods are a little bit controversial. \n\nThere are several issues in the experiments. Several important baselines are missing. The paper did not compare with any of the regularizer-based robust models. When considering efficiency and fast training, the authors also did not compare with recent fast method Shafahi et al. 2019 Free and Zhang et al. 2019 YOPO. \n\nWhen comparing with PGD adversarial training (Madry), in table 1, there is a more than 10% drop on robust accuracy for CIFAR-10 when \\epsilon=8.\n\nFor l2 norm attack, how to interpret table 2? Why not provide accuracy under norm constraint like table 1? \n\nSome uncommon settings are used in the experiments such as ResNeXt-34 and attacking randomly selected 1000 images. \n\nSome relatively minor issues, could the authors elaborate on why the optimal value of max_v l( x+v) - c(v) is the squared dual norm of \\grad l? \n\nTypo in title: scaleable -> scalable\n\n\n========= after rebuttal =============\nI thank the authors for detailed replies. I still cannot support paper because\n(1) the authors emphasize the theoretical contribution and claims the bound are tighter. However, they did not directly compare with any certified robust methods, or previous bounds to support the argument. \n(2) The empirical results look suboptimal. The authors did not convince me why they sampled 1000 images for test for a small CIFAR-10 dataset. The proposed method is 10% less robust comparing to Madry's in table 1.  ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: (1) the authors emphasize the theoretical contribution and claims the bound are tighter. However, they did not directly compare with any certified robust methods, or previous bounds to support the argument. 
HM, not sure, need to check this

(2) The empirical results look suboptimal. The authors did not convince me why they sampled 1000 images for test for a small CIFAR-10 dataset. The proposed method is 10% less robust comparing to Madry's in table 1.  
Seems ok, understand authors response

1) The theoretical analysis are not terribly new, which is just a straightforward application of first-order Taylor expansion. This idea could be traced back to the very first paper on adversarial examples FGSM (Goodfellow et al 2014).
True

2) The novelty of the paper is to replace exact gradient (w.r.t input) by their finite difference and use it as a regularization. However, there is a misalignment between the theory and the proposed algorithm. The theory only encourages input gradient regularization, regardless to how it is evaluated, and previous studies have shown that this is not a very effective way to improve robustness. According to the experiments, the main empirical improvement comes from the finite difference implementation but the benefit of finite difference is not justified/discussed by the theory. Therefore, the empirical improvement are not supported by the theory. Authors have briefly respond to this issue in the discussion but I believe a more rigorous analysis is needed.  
This seems okay based on author response

3) Moreover, the empirical performance does not achieve state-of-the-art result. Indeed, there is a non-negligible  gap (12%) between the obtained performance and some well-known baseline. Thus the empirical contribution is also limited. 
Yea, for some cases
Decision: Reject