Geometry-aware Instance-reweighted Adversarial Training
Authors: Jingfeng Zhang
Jianing Zhu
Gang Niu
Bo Han
Masashi Sugiyama
Mohan Kankanhalli
Venue: ICLR 2021 Oral (2021)
Keywords: Adversarial robustness

Abstract: In adversarial machine learning, there was a common belief that robustness and accuracy hurt each other. The belief was challenged by recent studies where we can maintain the robustness and improve the accuracy. However, the other direction, whether we can keep the accuracy and improve the robustness, is conceptually and practically more interesting, since robust accuracy should be lower than standard accuracy for any model. In this paper, we show this direction is also promising. Firstly, we find even over-parameterized deep networks may still have insufficient model capacity, because adversarial training has an overwhelming smoothing effect. Secondly, given limited model capacity, we argue adversarial data should have unequal importance: geometrically speaking, a natural data point closer to/farther from the class boundary is less/more robust, and the corresponding adversarial data point should be assigned with larger/smaller weight. Finally, to implement the idea, we propose geometry-aware instance-reweighted adversarial training, where the weights are based on how difficult it is to attack a natural data point. Experiments show that our proposal boosts the robustness of standard adversarial training; combining two directions, we improve both robustness and accuracy of standard adversarial training.

Published as a conference paper at ICLR 2021
GEOMETRY-AWARE INSTANCE-REWEIGHTDD ADVER- SARIAL TRAINING
Masashi Sugiyama Jianing Zhu
Jingfeng Zhang
Gang Niu'
Bo Han
Mohan Kankanhall
RIIENN Center for Advanced Intelligence Project, Tokyo, Japan
NNational University of Singapore, Singapore
'Hong Kong Baptist University, Hong Kong SAR, China
'The University of Tokyo, Tokyo, Japan
jingfeng. zhang@riken ip. csinzhu@come
likbu.edu... .kk
ninnggniurrieen ip, bhanml @ hibbu tikbu.edu...
mohanl comp. nus edu.sg
ABSTRACT
In adversarial machine learning, there was a common belief that robustness and accuracy hurt each other. The belief was challenged by recent studies where we can maintain the robustness and improve the accuracy. However, the other direc- tion, we can keep the accuracy and improve the robustness, conceptually and practically more interesting, since robust accuracy should be lower than standard accuracy for any model. In this paper, we show this direction is also promising. Firstly, we find even peee-paramtterized deep networks may still have insufficient model capacity, because adversarial training has an overwhelming smoothing ef- fect. Secondly, given limited model capacity, argue adversarial data should have unequal importance: geometrically speaking, a natural data point closer to/farther from the class boundary is less/more robust, and the corresponding ad- versarial data point should be assigned with larger/smaller weight. Finally, to implement the idea, propose geometry-aware ial training, where the weights are based on how difficult it is to allack natural data point. Experiments show that proposal boosts the robustness of standard ad versarial training; combining two directions, we improve both robustness and accuracy of standard adversarial training.
mnstance-reweiglie
adversar-
INTRODUCTION
Crafted adversarial data easily fool the standard-traie deep models by adding human- imperceptible noise to the natural data, which leads to the security issue applications such as medicine, finance, and autonomous driving (Szegedy al., 2014; Nguyen al., 2015). To mitigate this issue, many adversarial training methods employ the most adversarial data maximizing the loss for updating the current model such as standard adversarial training (AT) (Madry et al., 2018), TRADES (Zhang et al., 2019), robust self-training (RST) (Carmon et al., 2019), and MART (Wang al., 2020b). The adversarial training methods seek to train an adversarially robust deep model whose predictions are locally invariant to a small neighborhood of its inputs (Papernot et al., 2016). By leveraging adversarial data to smooth the small neighborhood, the adversarial training methods acquire adversarial robusiness against adversarial data but often lead to the undesirable degradation of standard accuracy on natural data (Madry et al., 2018: Zhang et al., 2019).
Thus, there have been debates whether there exists trade-off between robustness and accuracy. For example, some argued an inevitable trade-off:: Tsipras et al. (2019) showed fundamentally dif- ferent representations learned by standard-trained model and an adversarial-traines model; Zhang et al. (2019) and Wang et al. (2020a) proposed adversarial training methods that can trade off stan- dard accuracy for adversarial robustness. On the other hand, some argued that there is no such the trade-off: Raghunathan et al. (2020) showed infinite data could eliminate this trade-off; Yang et al. (2020) showed benchmark image datasets are class-separated.

---

Published as a conference paper at ICLR 2021
Large weight adversarial data
Small-weight
adversarial dita
data
+
Altickible data
Decision bonndar
Adversary direction
Figure 1: The illustration of GAIRAT. GAIRAT explicitly gives larger weights the losses of adversarial data (larger red), whose natural counterparts are closer to the decision boundary (lighter blue). GAIRAT explicitly gives smaller weights on the losses of adversarial data (smaller red), whose natural counterparts are farther away from the decision boundary (darker blue). The examples of two toy datasets and the CIFAR-10 dataset refer to Figure 3.
Recently, emerging adversarial training methods have empirically challenged this trade-off. For ex- ample, Zhang al. (2020b) proposed the friendly adversarial training method (FAT), employing friendly adversarial data minimizing the loss given that some grongly-predicted adversarial data have been found. Yang et al. (2020) introduced dropout (Srivastava et al., 2014) into existing AT, RST, and TRADES methods. Both methods can improve the accuracy while maintaining the ro- bustness. However, the other whethhrr we can improve the robustness while keeping the accuracy-remains unsolved and is more interesting.
In this paper, we show this direction is also achievable. Firstly, we show over-parameterized deep networks may still have insufficient model capacity, because adversarial training has an overwhelm- ing smoothing effect. Fitting adversarial data is demanding for a tremendous model capacity: It requires large number of trainable parameters error on the adversarial training data (see Figure 2). The over- wee-parameterized models that natu- ral data entirely in the standard training (Zhang al., 2017) are still far from enough for fitting adversarial data. Compared with standard training fitting the natural data points, adversarial train- ing smooths the neighborhoods of natural data, that adversarial data consume significantly more model capacity than natural data. Thus, adversarial training methods should carefully utilize the lim- ited model capacity to fit the neighborhoods of the important data that aid fine-tune the decision boundary. Therefore, it may be unwise to give equal weights to all adversarial data.
training epochs to reach near-zero
Secondly, data along with their adversarial variants are not equally important. Some data are geo- metrically far away from the class boundary. They are relatively guarded. Their adversarial variants are hard be misclassified. On the other hand, some data are close to the class boundary. They are relatively attackable. Their adversarial variants are easily misclassified (see Figure 3). As the adversarial training progresses, the adversarially robust model engenders an increasing number of guarded training data and a decreasing number of attackable training data. Given limited model capacity, treating all data equally may cause the vast number of adversarial variants of the guarded data to overwhelm the model, leading to the undesirable robust overfitting (Rice et al., 2020). Thus, it may be pessimistic to treat all data equally in adversarial training.
To ameliorate this pessimism, we propose heuristic method, i.e., geometry-aware instance- reweighted adversarial training (GAIRAT). As shown in Figure 1, GAIRAT treats data differently. Specifically, for updating the current model, GAIRAT gives larger/smaller weight to the loss of an adversarial variant of attackable/guaree decision boundary. An attackable/guarded data point has small/large geometric distance, i.e., its distance from the decision boundary. We approximate its geometric distance by the least number of iterations x that projected gradient descent method (Madry et al., 2018) requires generate a mis- classified adversarial variant (see the details in Section 3.3). GAIRAT explicitly assigns instance- dependent weight to the loss of its adversarial variant based on the least iteration number is
data point which is more/less important in fine-tuning the
Our contributions are as follows. (a) In adversarial training, we identify the pessimism in treating all data equally, which is due to the insufficient model capacity and the unequal nature of different data (in Section 3.1.. (b) We propose a new adversarial training method, i.e., GAIRAT (its learning objective in Section 3.2 and its realization in Section 3.3). GAIRAT is a general method: Besides standard AT (Madry et al., 2018), the existing adversarial training methods such FAT (Zhang et al., 2020b) and TRADES (Zhang et al., 2019) can be modified to GAIR-FAT and Appendices B.1 and B.2, respectively). (c) Empirically, our GAIRAT can relieve the issue of robust.
(in
2

---

Published as a conference paper at ICLR 2021
Training data (different networks)
Training data (different étrain)
100
100
Budiee-25 (Sataal
Booeeks (PGD-22)
1/755 (Naturs)
Rochet (Natonal
Rorker as (P9D
5/256 (Naiaaa))
(PCD-10)
Thining ortor (%)
80
"Iaainin ertor (36)
80
13/235 (Natual))
14/233 (PGD-50)
Bidiie-2 (Natona)
60
60
40-
20-
20-
%
20
40
60
80
100
0
20
40
60
80
100
Epochs
Epocha
Figure 2: We plot standard training error (Natural) and adversarial training error (PGD-10) over the training epochs of the standard AT on CIFAR-10 dataset. Left panel: AT on different sizes of network. The red line represents standard test accuracy by standard training (ST). Right panel: AT on ResNet-18 under different perturbation bounds étrain
overfitting (Rice et al., 2020), meanwhile leading the improved robustness with zero or little degra- dation of accuracy (in Section 4.1 and Appendix C.I). Besides, we use Wide ResNets (Zagoruyko & Komodakis, 2016) to corroborate the efficacy of our geometry-aware instance-reweighted methods: Our GAIRAT significantly boosts the robustness of standard AT; combined with FAT, our GAIR- FAT improves both the robustness and accuracy standard AT (in Section 4.2). Consequently, we conjecture no inevitable trade-off between robustness and accuracy.
ADVERSARIAL TRAINING
In this section, we review adversarial training methods (Madry et al., 2018; Zhang et al., 2020b).
2.1 LEARNING OBJECTIVE
Let (X' doo)) denote the input feature space X' with the infinity distance metric diat (2, x²) | a'lloo and B, [x] fa' X dint (I, x') S c) be the closed ball of radius c centered at x in X. Dataset S = {(T;, b,)) where x; E X and ys E ) = {0, 1,.1, C - 1}.
The objective function of standard adversarial training (AT) (Madry et al., 2018) is
min JoEF
(lo().).u))
(1)
where
T = arg max max (T)(
(2)
where 7 the most adversarial data within the e-ball centered at x. fo(.) X - R²O is a score
function, and the loss function f: RC x + R is a composition of a base loss lp A-11 x ) IR (e.g., the cross-entropy loss) and an inverse link function & : RCC C-1 (e.g., the soft-max activation), in which C-1 is the corresponding probability simplex in other words, {(f((),, y) lp (e (fe(-)) )), y). AT employs the most adversarial data generated according to Eq. (2) for updating the current model.
The objective function of friendly adversarial training (FAT) (Zhang et al., 2020b) is
f = arg min ((fo y.) s.t. {(fo(t) - minyey ((fo(() v) 2 p.
ScB_[n [m.]]
Note that the outer minimization remains the same as Eq. (1), and the operator arg max is replaced by arg min. p is a margin of loss values (i.e., the misclassification confidence). The constraint of Eq. (3) firstly ensures is misclassified, and secondly ensures for ă the wrong prediction is better than the desired prediction y/ by at least p in terms of the loss value. Among all such satisfying the constraint, Eq. (3) selects the minimizing f(fe (ž), y1 by violation of the value p. There constraints on 3; if 2; is correctly classified. FAT employs the friendly adversarial data generated according to Eq. (3) for updating the current model.
3

---

Published as a conference paper at ICLR 2021
Closs
More attackable data
More guarded data More goarded data
Norplanp
O Class B:
More attackable data
Anttotubbile
Class boundary
Toy example!
Toy example 2
Figure 3: More attackable data (lighter red and blue) are closer to the class boundary; more guarded data (darker red and blue) are farther away from the class boundary. Left panel: Two toy examples. Right panel: The model's output distribution of two randomly selected classes from the CIFAR- dataset. The degree of robustness (denoted by the color gradient) of data point is calculated based on the least number of iterations k that PGD needs to find its misclassified adversarial variant.
2.2 REALIZATIONS
AT and FAT's objective functions imply the optimization of adversarially robust networks, with one step generating adversarial data and one step minimizing loss on the generated adversarial data w.r.t the model parameters 0.
The projected gradient descent method (PGD) (Madry et al., 2018) is the most common approxima- tion method for searching adversarial data. Given starting point z()) E X and step size a > 0, PGD works follows:
(++1) 11 (x + ((o
until a certain stopping criterion is satisfied. is the loss function; =0 refers to natural data or natural data perturbed by small Gaussian uniformly random noise; is the corresponding label for natural data; (+) is adversarial data at step t; and H, [mo] (-) is the projection function that projects the adversarial data back into the e-ball centered (0) necessary.
There are different stopping criteria between AT and FAT. AT employs fixed number of iterations K, namely, the PGD-A K algorithm (Madry et al., 2018), which is commonly used in many adversarial training methods such as CAT (Cai et al., 2018), DAT (Wang et al., 2019), TRADES (Zhang et al., 2019), and MART (Wang et al., 2020b). On the other hand, FAT employs the misclassificcttonnnnn -awae criterion. For example, Zhang al. (2020b) proposed the early-stopped PGD- K-7 algorithm (r S K; K is the fixed and maximally allowed iteration number): Once the PGD-K finds the current model misclassifying the adversarial data, it stops the iterations immediately (r 0) slides few more steps (r > 0). This misclassification-awe aware criterion is used in the emerging adversarial training methods such MMA (Ding et al., 2020), FAT (Zhang et al., 2020b), ATES (Sitawarin et al., 2020), and Customized AT (Cheng et al., 2020).
AT can enhance the robustness against adversarial data but, unfortunately, degrades the standard accuracy on the natural data significantly (Madry et al., 2018). On the other hand, FAT has better standard accuracy with pero-zero or little degradation of robustness (Zhang et al., 2020b).
Nevertheless, both AT and FAT treat the generated adversarial data equally for updating the model parameters, which not necessary and sometimes even pessimistic. the next sections, we in- troduce our method GAIRAT, which is compatible with existing methods such as AT, FAT, and TRADES. Consequently, GAIRAT can significantly enhance robustness with little or even zero degradation of standard accuracy.
GEOMETRY-AWARE
-HANNCE-REWEIGHEED ADVERSARIAL TRAINING
In this section, we propose geometry- aware instance-reweighee adversarial training (GAIRAT) and its learning objective well its algorithmic realization.
4

---

Published as a conference paper at ICLR 2021
3.1 MOTIVATIONS OF GAIRAT
Model capacity is often insufficient in adversarial training. In the standard training, the over- parameterized networks, e.g., ResNet-18 and even larger ResNet-50, have more than enough model capacity, which can easily fit the natural training data entirely (Zhang et al., 2017). However, the left panel of Figure shows that the model capacity of those wer-paramettrized networks is not enough for fitting the adversarial data. Under the computational budget of 100 epochs, the networks hardly reach zero error on the adversarial training data. Besides, adversarial training error only decreases by small constant factor with the significant increase of the model's 's parameters. Even worse, slightly larger perturbation bound Étrain significantly uncovers this insufficiency of the model capacity (right panel); Adversarial training error significantly increases with slightly larger étrain- Surprisingly, the standard training error on natural data hardly reaches zero with Curain = 16/255.
Adversarial training methods employ the adversarial data to reduce the sensitivity of the model's output w.r.t. small changes the natural data (Papernot et al., 2016). During the training process, the fly and are adaptively changed based the current model
adversarial data are generated to smooth the natural data's local neighborhoods. The volume of this surrounding is exponentially (11 + Etraia l large w.r.t. the input dimension |2), even if étrain is small. Thus, this smooth- ness consumes significant model capacity. In adversarial training, should carefully leverage the limited model capacity by fitting the important data and by ignoring the unimportant data.
More attackable/guarded data are closer to/farther away from the class boundary. We can measure the importance of the data by their robustness against adversarial attacks. Figure 3 shows that the robustness (more attackable or more guarded) of the data is closely related to their geometric distance from the decision boundary. From the geometry perspective, more attackable data are closer to the class boundary whose adversarial variants are more important to fine-tune the decision boundary for enhancing robustness.
Appendix A contains experimental details of Figures 2 and 3 and more motivation figures.
3.2 LEARNING OBJECTIVE OF GAIRAT
Let (xr v) be the geometry-aware weight assignment function the loss of adversarial variant ä. The inner optimization for generating I still follows Eq. (2) or Eq. (3). The outer minimization is
a Σ wxx y ()(((f(()))))
min F&EF n
(5)
The constraint firstly ensures that y: = arg max; fo (I;) and secondly ensures that w(,,, y:) is a non-increasing function w.r.. the geometric distance, ie.. the distance from data æ; to the decision boundary, in which w(,,, y:) 2 and 1/2 E w(x;; y:)
There are constraints when y; + arg max; fo(x,) : for those significantly far away from the decision boundary, we may discard them (outliers); for those x close to the decision boundary, we may assign them large weights. In this paper, do not consider outliers, and therefore assign large weight to the losses of adversarial data, whose natural counterparts are misclassified. Figure provides an illustrative schematic of the learning objective of GAIRAT.
burn-in period may be introduced, i.e., during the initial period of the training epochs, w(r;, yi) regardless of the geometric distance of input (I:; Mr). because the geometric distance less infor- mative initially, when the classifier is not properly learned.
3.3 REALIZATION OF GAIRAT
The learning objective Eq. (5) implics the optimization of an adversarially robust network, with one step generating adversarial data and then reweighting loss them according the geometric distance of their natural counterparts, and one step minimizing the reweighted loss w.r.t. the model parameters 0.
We approximate the geometric distance of a data point (x, y) by the least iteration numbers k(x, y) that the PGD method needs to generate adversarial variant & to fool the current network, given the
5

---

Published as a conference paper at ICLR 2021
Algorithm 1 Geometry- projected gradient descent (GA-PGD)
Input: data E X, label E yr model f. loss function &, maximum PGD step K, perturbation bound 6, step size or
Output: adversarial data è and geometry value k(2, y)
< z; k(x,g) 8) f- 0 while K > 0 do
if arg max; [() = then
x(x,)) y) + k(x; v) +
end if
I 11 (itin((a(((()).))))) +3)
+ K -1
end while
Algorithm 2 Geometry-aware instance-dependentt adversarial training (GAIRAT)
Input: network So. training dataset 5 {(1). learning rate 7), number of epochs T,
batch size m, number of batches M Output: adversarially robust network for epoch = l...................... do
for mini-batch = L...., .M do Sample a mini- batch { y;),; from S for .... (in parallel) do
Obtain adversarial data ã; of x; and geometry value n(ri, yi) by Algorithm 1 Calculate w(rt,y)) W:) according geometry value k(r;;; yi) by Eq.
end for
0 - 0 Vo
Ei
w(m ( ((f((z))..)
end for end for
maximally allowed iteration number K and step size Thus, the geometric distance is approxi- mated by x (precisely by x x a). Thus, the value the weight function w should be non-increasing w.r.t. K. We name k(z, y) the geometry value of data (z. y).
How to calculate the optimal w still an open question; therefore, we heuristically design different non-increasing functions We give example here and discuss more examples in Appendix C.3 and Section 4.1.
w1, y) = (1+ tanh (A 5 x (1-22 x k(2, v)/K))
(6)
2
where R/K 10, K N+, and À R. If +oc, GAIRAT recovers the standard AT (Madry et al., 2018), assigning equal weights to the losses of adversarial data.
Algorithm is geometry- aware PGD method (GA-PGD). which returns both the most adversarial data and the geometry value of its natural counterpart. Algorithm is geometry-aware instance- dependent adversarial training (GAIRAT). GAIRAT leverages Algorithms 1 for obtaining the adver- sarial data and the geometry value. For each mini-batch, GAIRAT reweighs the loss of ad adversarial data (ū,, 3/3 according the geometry value of their natural counterparts (æt, y:), and then updates the model parameters by minimizing the sum of the reweighted loss.
GAIRAT is general method. Indeed, FAT (Zhang et al., 2020b) and TRADES (Zhang et al., 2019) (see Appendices B.1 and B.2, respectively).
can be modified to GAIR-FAT and
Comparisons with SVM. The abstract concept of GAIRAT has appeared previously. For exam- ple, in the support vector machine (SVM), support vectors near the decision boundary are partic- ularly useful in influencing the decision boundary (Hearst et al., 1998). For learning models, the magnitude of the loss function (e.g., the hinge loss and the logistic loss) can naturally capture differ- ent data's ' geometric distance from the decision boundary. For updating the model, the loss function treats data differently by incurring large losses important attackable (close to the decision bound-

---

Published as a conference paper at ICLR 2021
ary) or misclassified data and incurring from the decision boundary) data.
very small losses on unimportant guarded (far away
However, in adversarial training, it is critical to explicitly assign different weights top of losses on different adversarial data due to the blocking effect: The model trained on the adversarial data that maximize the loss learns prevent generating large-loss adversarial data. This blocking ef- fect makes the magnitude of the loss less capable of distinguishing important adversarial data from unimportant ones for updating the model parameters, compared with the role of loss on measur- ing the natural data's importance in standard training. Our GAIRAT breaks this blocking effect by explicitly extracting data's geometric information to distinguish the different importance.
Comparisons with AdaBoost and focal loss. The idea of instance-dependent weighting has been studied in the literature. Besides robust estimator (e.g.. M-cstimator (Boos & Stefanski, 2013)) for learning under outliers (e.g.. label-noised data), hard data mining is another branch where our GAIRAT belongs. Boosting algorithms such as AdaBoost (Freund & Schapire, 1997) select harder examples train subsequent classifiers. Focal loss (Lin et al., 2017) is specially designed loss func- tion for mining hard data and misclassified data. However, the previous hard data mining methods GAIRAT measures the
leverage the data's losses for measuring the hardness; by comparison, hardness by how difficulty the natural data are attacked (i.e., geometry value k). This new measure- ment K sheds new lights on measuring the data's 's hardness (Zhu et al., 2021).
Comparisons with related adversarial training methods. Some existing adversarial training methods also "treat adversarial data differently", but in different ways our GAIRAT. For example, CAT (Cai al., 2018), MMA (Ding et al., 2020), and DAT (Wang et al., 2019) methods gener- ate the differently adversarial data for updating model over the training process. CAT utilized the adversarial data with different PGD iterations K. DAT utilized the adversarial data with differ- ent convergence qualities. MMA leveraged adversarial data with instance-dependent perturbation bounds e. Different from those existing methods, our GAIRAT treat adversarial data differently by explicitly assigning different weights on their losses, which can break the blocking effect.
Note that the learning objective of MART (Wang al., 2020b) also explicitly assigns weights, not directly on the adversarial loss but KL divergence loss (see details in Section C.7). The KL divergence loss helps strengthen the smoothness within the norm ball of natural data, which is also used in VAT (Miyato et al., 2016) and TRADES (Zhang et al., 2019). Differently from MART, our GAIRAT explicitly assigns weights on the adversarial loss. Therefore, we can easily modify MART to GAIR-MART (see experimental comparisons in Section C.7). Besides, MART assigns weights based on the model's prediction confidence on the natural data; GAIRAT assigns weights based on how easy the natural data can be attacked (geometry value k).
Comparisons with the geometric studies of DNN. Researchers in adversarial robustness em- ployed the first-order second-order derivatives w.r.t. input data to explore the DNN's geometric properties (Fawzi et al., 2017; Kanbak et al., 2018; Fawzi et al., 2018; Qin et al., 2019; Moosavi- Dezfooli al., 2019). Instead, have complementary but different argument: Data points them- selves are geometrically different regardless of DNN. The geometry value & in adversarial training (AT) is an approximated measurement of data's geometric properties due to the AT's smoothing effect (Zhu et al., 2021).
4 EXPERIMENTS
In this section, we empirically justify the efficacy of GAIRAT. Section 4.1 shows that GAIRAT can relieve the undesirable robust overfitting (Rice et al., 2020) of the minimax-base adversarial training (Madry et al., 2018). Note that some concurrent studies (Chen et al., 202 la;b) provided various adversarial training strategies, which can also mitigate the issue of robust overfitting. In Section 4.2, we benchmark our GAIRAT and GAIR-FAT using Wide ResNets and compare them with AT and FAT.
In our experiments, we consider ||ā ||ā=z|] 2//2000 S with the same e in both training and evaluations. All images of CIFAR-10 (Krizhevsky, 2009) and SVHN (Netzer al., 201 1) normalized into [0. 1].
7

---

Published as a conference paper at ICLR 2021
Weight aesigrumett fouctions
Natural data
PGD320
AT (otp Thuriingeee
70
00.8
601
GARRA (43). Trining
GAIRAT (u2) TIncnigg
Holkast cerur (86)
70-
0.5
(w1) constunt of
Sbandard ermo
50-
AT (Jd); Teal snor
GAIRNT lutt): decreasing,
30
GAIRAT (u2)) Thd
50-
Weele
0.4-
GAIRAT (wS): piecewise
30-
GAIURT (4:30
Tht orne
40
0.2
20
10
0.0
0.8
1.0
20
30
80
100
100
K/K
Epochs
Egochss
Geometry values statistic
Loss flatnees w.r.t. PGD-2000 data
Loes flatness w.r.t.
PGDD22 test data
10
Gecueta value a
8
Loss flattoo [V,(|)
8.
AT (wi)
GAIRAT (w22)
300
35-
6
GAIRAT (w:3)
25-
200
4
meam (AT)
Loss
2
median (AT)
Epoclts 100
60
Epodls
Epocle
Figure 4: Comparisons of AT (w1, red lines) and GAIRAT (w2, blue lines and W/s yellow lines) using ResNet-18 on the CIFAR- 10 dataset. Upper-left panel shows different weight assignment functions w.r.t. the geometry value k. Bottom-left panel reports the training statistic of the standard AT and calculates the median (dark red circle) and mean (light red cross) of geometry values of all training data at each epoch. Upper-middle and upper-right panels report standard training/test errors and robust training/test errors, respectively. Bottom-middie and bottom-right panels report the loss flatness w.r.t. friendly adversarial test data and most adversarial test data, respectively.
4.1 GAIRAT RELIEVES ROBUST OVERFITTING
In Figure 4, we conduct the standard AT (all red lines) using ResNet-18 (He al., 2016) on CIFAR- 10 dataset. For generating the most adversarial data for updating the model, the perturbation bound c= 8/255: the PGD steps number K 10 with step size a = 2/255, which keeps the same as Rice et al. (2020). We train ResNet-18 using SGD with 0.9 momentum for 100 epochs with the initial learning rate of 0. divided by 10 at Epoch 30 and 60, respectively. At each training epoch, we collect the training statistics, i.c., the geometry value k(x, y) of cach training data, standard//obust training and test error, the flatness of loss w.r.t adversarial test data. The detailed descriptions of those statistics and the evaluations are in the Appendix C.I.
Bottom-left panel of Figure 4 shows geometry value x of training data of standard AT. Over the training progression, there is increasing number of guarded training data with sudden leap when the learning rate decays to 0.01 at Epoch 30. After Epoch 30, the model steadily engenders increasing number of guarded data whose adversarial variants are correctly classified. Learning from those correctly classified adversarial data (large portion) will reinforce the existing knowledge and spare little focus on wrongly predicted adversarial data (small portion). thus leading to the robust overfitting, The robust overfitting is manifested by red (dashed and solid) lines in upper-middle and upper-right and nattom-midlee and nottom-right panels.
To avoid the large portion of guarded data overwhelming the learning from the rare attackable data, our GAIRAT explicitly give small weights the losses of adversarial variants of the guarded data. Blue (w2) and yellow (wa) lines in upper-left panel give two types of weight assignment functions that assign instance-dependent weight on the loss based on the geometry value k. In GAIRAT, the model is foreed to give enough focus those rare attackable data.
In GAIRAT, the initial 30 epochs is burn-in period, and we introduce the instance-dependent weight assignment w from Epoch 31 onward (both blue and yellow lines in Figure 4). The rest of hyper- parameters keeps the same as AT (red lines). From the upper-right panel, GAIRAT (both yellow and blue lines) achieves smaller error adversarial test data and larger error training adversarial data, compared with standard AT (red lines). Therefore, our GAIRAT can relieve the issue of the robust overfitting.
8

---

Published as a conference paper at ICLR 2021
Besides, Appendix C contains more experiments such as different learning rate schedules, dif- ferent choices of weight assignment functions w, different lengths of bur--in period, a different dataset (SVHN) and different networks (Small CNN and VGG), which all justify the efficacy of our GAIRAT. Notably, in Appendix C.6, we show the effects of GAIR-FAT on improving FAT.
4.2
PERFORMANCE EVALCATION ON WIDE RESNETS
Table 1: Test accuracy of WRN-32-10 on CIFAR-10 dataset
Best checkpoint
Last checkpoint
Defense
Natural
DifT.
PGD-20
Diff.
PGD+
Diff.
Natural
Diff.
PGD-20
Diff.
PGD+
Diff.
AT FAT GAIRAT GAIR-FAT
86.92
0.24
51,96
0.21
51,28 + 0.23
86.62
0.22
46.73
0.08
46.08
0.07
89.16
0.15
+2.24
51.24
0.14
-0.72
46.14
0.19
-5.14
88.18
0.19
+1.50
46.79 0.34
+0.06
45.80
0.16
-0.28
85.75
0.23
-1.17
57.81
0.54
-5.5
55.61
1 0.61
+4.33
85,49
0.25
1.13
53.76
0.49
+7.03
50.32
0.48
+4.24
88.59
0.12
11:77
56.21
0.52
53.50 1 0.60
12.22
88.44
1182
50.64
0.56
13.91
47.51
0.51
1143
We employ the large-capacity network, i.e., Wide ResNet (Zagoruyko & Komodakis, 2016), on the CIFAR-10 dataset. In Table I, we compare the performance of the standard AT (Madry al., 2018), FAT (Zhang et al., 2020b), GAIRAT and GAIR-FAT. We use WRN-32-10 10 that keeps the same as Madry et al. (2018). We compare different methods on the best checkpoint model (suggested by Rice et al. (2020)) and the last checkpoint model (used by Madry et al. (2018)), respectively. Note that results Zhang et al. (2020h) only compare the last checkpoint between AT and FAT; instead, we also include the best checkpoint comparisons. We evaluate the robust models based on the three evaluation metrics, i.e., standard test accuracy on natural data (Natural), robust test accuracy on adversarial data generated by PGD-20 and PGD+. PGD+ is PGD with five random starts, and each start has 40 steps with step size 0.01, which keeps the same as Carmon et al. (2019) (PGD+ has 40 x 5 200 iterations for each test data). We run AT, FAT, GAIRAT, and GAIR-FAT five repeated trials with different random seeds. Table reports the medians and standard deviations of the results. Besides, we treat the results of AT as the baseline and report the difference (Diff.) of the test accuracies. The detailed training settings and evaluations are in Appendix C.8. Besides, we also compare TRADES and EAIR-TRADDE using WRN-34-10, which in the Appendix C.9.
Compared with standard AT, our GAIRAT significantly boosts adversarial robustness with little degradation of accuracy, which challenges the inherent trade-off. Besides, FAT also challenges the inherent trade-off instead by improving accuracy with little degradation of robustness. Combin- ing two directions, i.e., GAIR-FAT, we can improve both robustness and accuracy of standard AT. Therefore, Table 1 affirmatively confirms the efficacy of our geometry- instance-reweighted methods in significantly improving adversarial training.
5 CONCLUSION AND FUTURE WORK
This paper has proposed a novel adversarial training method, i.c., geometry-aware instance- reweighted adversarial training (GAIRAT). GAIRAT gives more (less) weights to loss of the ad- versarial data whose natural counterparts are closer to (farther away from) the decision boundary. Under the limited model capacity and the inherent inequality of the data, GAIRAT sheds new lights on improving the adversarial training.
GAIRAT training under the PGD attacks can defend PGD attacks very well, but indeed, it cannot perform equally well on all existing attacks (Chen et al., 2021a). From the philosophical perspective, cannot expect defenses under one specific attack can defend all existing attacks, which echoes the previous finding that "it is essential to include adversarial data produced by all known attacks, as the defensive training is non-adaptive (Papernot et al., 2016)." Incorporating all attacks in GAIRAT yet preserving the efficiency is an interesting future direction. Besides, it still an open question to design the optimal weight assignment function in Eq. or to design proper network structure suitable to adversarial training. Furthermore, there is still a large room to apply adversarial training techniques into other domains such as pre-training (Hendrycks et al., 2019; Chen et al., 2020; Jiang et al., 2020; Salman et al., 2020), noisy labels (Zhu et al., 2021) and so on.

---

Published as a conference paper at ICLR 2021
ACKNOWLEDGMENT
JZ, GN, and MS were supported by JST AIP Acceleration Research Grant Number JPMJCR20U3, Japan. MS was also supported by the Institute for Al and Beyond, UTokyo. JNZ and BH were supported by the HKBU CSD Departmental Incentive Scheme. BH was supported by the RGC Early Career Scheme No. 22200720 and NSFC Young Scientists Fund No. 62006202. MK was supported by the National Research Foundation, Singapore under its Strategic Capability Research Centres Funding Initiative.
REFERENCES
Mislav Balunovic and Martin Vechev. Adversarial training and provable defenses: Bridging the gap. In ICLR, 2020.
Denni D. Boos and I.. A. Stefanski. M-Estimation (Extimating Equations), pp. 297-337. Springer New York, New York, NY, 2013.
Qi-Zhi Cai, Chang Liu, and Dawn Song. Curriculum adversarial training. In IJCAI, 2018.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In Symposium Security and Privacy (SP), 2017.
Yair Carmon, Aditi Raghunathan. Ludwig Schmidt, Percy Liang, and John C. Duchi. Unlabeled data improves adversarial robustness. In NeurlPS, 2019.
Chen Chen, Jingfeng Zhang, Xilie Xu, Tianlei Hu, Gang Niu, Gang Chen, and Masashi Sugiyama. Guided interpolation for adversarial training. arXiv:2 102.0777, 2021a.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial
robustness: From self-supervised pre-training fine-tuning. In CVPR, 2020.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In ICLR, 2021b.
Minhao Cheng, Qi Lei, Pin-Yu Yu Chen, Inderjit Dhillon, and Cho-Jui Hsieh. Cat: Customized adver- sarial training for improved robustness. arXiv:2002.06789, 2020.
Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via random- ized smoothing. In ICML, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In ICML, 2020.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In ICLR, 2020.
Alhussein Fawzi, Seyed-Mohsen networks: A geometrical perspective. IEEE Signal Processing Magazine, 34(6):50-62, 2017.
and Pascal Frossard. The robustness of deep
Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfoolli Pascal Frossard, and Stefano Soatto. Empirical study of the topology and geometry of deep networks. In CVPR, 2018.
Yoav Freund and Robert E Schapire. A decisiin-theoreticc generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 15(1):119-11) 1997.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against noun-bounded adversarial examples. arXiv:2010.035533, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Marti A. Hearst. Susan Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. Support vector machines. IEEE Intelligent Systems and their applications, 13(4):18-25 1998.
10

---

Published as a conference paper at ICLR 2021
Dan Hendrycks. Kimin Lee, and Mantas Mazeika, Using pre- pre-training can improve model robustness and uncertainty. In ICML, 2019.
Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. In NeurlPS, 2020.
Can Kanbak, Seyed-Mohsen Moosavi- and Pascal Frossard. Geometric robustness of deep networks: analysis and improvement. In CVPR, 2018.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009,
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. ICCV, 2017.
Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Takeru Miyato, Shin-ichi Macda, Masanori Koyama, Ken Nakac, and Shin Ishii. Distributional smoothing by virtual adversarial examples. In ICLR, 2016.
Seyed-Mohsen Moosavi-Dezfooli Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robust- ness via eurvature regularization, and vice versa. In CVPR, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Read- ing digits in natural images with unsupervised feature learning. In NeurlPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confi- dence predictions for unrecognizable images. In CVPR, 2015.
Nicolas Papernot. Patrick McDaniel, Arunesh Sinha, and Michael Wellman. Towards the science of security and privacy machine learning. arXiv:1611.038/1 2016.
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. In NewrlPS, 2019.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. ICML, 2020.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, 2020.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adver- sarially robust imagenet models transfer better? NeurIPS, 2020.
Vikash Schwag, Shiqi Wang, Prateek Mittal, and Suman Jana. Hydra: Pruning adversarially robust neural networks. NewrIPS, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
Chawin Sitawarin, Supriyo Chakraborty, and David Wagner. Improving adversarial robustness through progressive hardening. arXiv:2003.09347, 2020.
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1): 1958, 2014.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, 2014.
Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE transactions on pattern analysis and machine intelligence, 30(1 1978- 2008
11

---

Published as a conference paper at ICLR 2021
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In ICLR, 2019.
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-Margin training: Scalable certifica- tion of perturbation invariance for deep neural networks. In NeurIPS, 2018
Haotao Wang, Tianlong Chen, Shupeng Gui, Ting-Kut Kuei Hu, Ji Liu, and Zhangyang Wang. Once- for-all adversarial training: In-situ tradeoff between robustness and accuracy for free. In NeurlPS 2020, 2020a.
Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, 2019.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020b.
Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In ICML, 2018.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener- alization. NewrIPS, 33, 2020.
Yao-Y Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R. Salakhutdinov, and Kamalika Chaud- huri. A closer look at accuracy vs. robustness. In NeurlPS, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv:1605.07144 2016.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. ICLR, 2017.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li. Duane Boning, and Cho-Jui Hsich. Towards stable and efficient training of verifiably robust neural networks. In ICLR, 2020a.
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020b.
Jaaning Zhu, Jingfeng Zhang, Bo Han, Tongliang Liu, Gang Niu, Hongxia Yang, Mohan Kankan- halli, and Masashi Sugiyama. Understanding the interaction of adversarial training with noisy labels. arXiv: arXiv:2102.03.4 20.22002.0330 03482, 2021.
12

---

Published as a conference paper at ICLR 2021
Natural training data
PGD 10 training data
100
100
Standard training erior (%)
ResNet-10
3e
Robust training ertor (96
ResNet-10
ResNet-31
80
RosNet-18
ResNec-12
80
ResNet-18
ResNet-12
ResNet-26
ResNec-00
ResNet-26
ResNet-50
60-
60
40-
40-
20-
60
80
100
20-
%
20
40
60
80
100
0
20
40
60
80
100
Epochs
Epochs
Natural training data
PGD-10 10 training data
100
100
Standard traiing error (96)
1/255
8/255
1/255
8/256
80
< 2/255
10/255
80
2/285
10/255
4/265
12/255
Robust training ernnr
4/955
12/255
6/255
16/255
60
6/266
16/255
40-
40
20
20
0
60
Epochs
Epochs
Figure 5: We plot standard training error (the left two panels) and adversarial training enror (the right two panels) over the training epochs of the standard AT on CIFAR- 10 dataset. Top two panels: standard AT on different sizes of network. Bottom two panels: standard AT ResNet-18 under different perturbation bound étrain-
A MOTIVATIONS OF GAIRAT
We show that model capacity is often insufficient in adversarial training, especially when étrain is large; therefore, the model capacity should be carefully preserved for fitting important data.
In this section, we give experimental details of Figure 2 and provide complementary experiments in Figures and 6. In the left panel Figure 2 and top two panels of Figure 5, use standard AT to train different sizes of network under the perturbation bound étrain 8/ 8/255 on CIFAR- 10 dataset. In the right panel of Figure 2 and two bottom panels Figure 5, we fix the size of network and use ResNet-18; we conduct standard AT under different values of perturbation bound étrain E [1 /55, 16/255] The solid lines show the standard training error on natural data and the dash lines show the robust training error on adversarial training data.
Training details We train all the different networks for 100 epochs using SGD with 0.9 momen- tum. The initial learning rate is 0.1, reduced 0.01, 0.001 at Epoch 30, and 60, respectively. The weight decay is 0.0005. For generating the most adversarial data for updating the model, we use the PGD-10 attack. The PGD steps number バ 10 and the step size = c/4. There is a random start, i.e., uniformly random perturbations ([-Etrain- +-train)) added to natural data before PGD per- turbations for generating PGD-10 training data. We report the standard training error on the natural training data and the robust training error the adversarial training data that are generated by the PGD-10 attack.
We also conduct the experiments the SVHN dataset in Figure 6. The training setting keeps the same as that of CIFAR-10 experiments except using 0.01 as the initial learning rate, reduced to 0.001, 0.0001 Epoch 30, and 60, respectively. We find standard AT always fails when the perturbation bound larger than =
16 16/25 for the SVHN dataset due to the severe cross-over mixture issue (Zhang et al., 2020b); therefore, do not report its results.
13

---

Published as a conference paper at ICLR 2021
Natural training data
PGD 10 training data
100
100
Standari training erior (%)
ResNet-10
3esNec:31
ResNet-10
ResNet-31
80
RosNet-18
RusNec-12
80
ResNet-18
ResNet-12
Robust training ertor
ResNet-26
ResNec-00
ResNet-26
ResNet-50
60-
3
60
2
40-
40
0-
20-
60
80
100
20-
%
0
20
40
60
80
100
20
40
60
80
100
Epochs
Epache
Natural training data
PGD-10 10 training data
100
100
Standard traiing error (96)
1/255
8/255
1/255
8/256
80
e 2/255
10/255
80
2/285
10
4/265
12/255
Robust training emnrr
4,955
12/255
6/255
60
9/255
40
40
20
0
0
60
60
Epochs
Epochs
Figure 6: We plot standard training error (the left two panels) and adversarial training error (the right two panels) over the training epochs of the standard AT on SVHN dataset. Top two panels: AT on different sizes of network. Bottom two panels: AT on ResNet- 18 under different perturbation bound étrain-
Next, we show that more attackable (more important) data are closer to the decision boundary; more guarded (less important) data are farther away from the decision boundary.
In Figures and 8, we plot 2-d visualizations of the output distributions of a robust ResNet- on CIFAR-10 10 dataset. We take the robust ResNet- 18 at the checkpoint of Epoch 30 (red line in Figure 9) as our base model here. For each class in the CIFAR- 10 dataset, we randomly sample 1000 training datapoints for visualization. For each data point, we compute its the least number of iterations k that PGD requires to find its misclassified adversarial variant. For PGD, we set the perturbation bound 0.031, the step size = 0.31/4, and the maximum PGD steps K 10. Then, each data point has its unique robustness attribution, i.e., value ε. We take those data as the input of the robust ResNet and output 10-dimensional logits, and then, we principal components analysis (PCA) to project 10-dimensional logits into 2-dimension for visualization. The color gradient denotes the degree of the robustness of each data point. The more attackable data have lighter colors (red or blue), and the more guarded data has darker colors (red or blue).
From Figures 7 and 8, we find that the attackable data in general are geometrically close to the decision boundary while the guarded data in general are geometrically far away from the decision boundary. is also very interesting observe that not all classes are well separated. For example, Cat-Dog is less separable than Cat-Ship in second row of Figure 8.
14

---

Published as a conference paper at ICLR 2021
2
2
Automolsile Airplane
Bird Airplame
Airplane Cat
Deer Airplane
Dog Airplane
Frog Rigg-Arplane
Airplane- Horse
Mhip-Airplane
Treck-Airlanee
Bird Automolile
Cat- Aot-mooooobi
Deer- Automobile
Dog- Automobile
Frog- Automobile
Horse-Antoomoole Automobile
Ship Automobile
Truck Automobile
Bird Cat
Bird Deer
Brid Dog
Figure 7: Part A 2-d visualizations of the model's output distribution of natural training data from two separated classes from CIFAR-10 10 dataset. The degree of the robustness (denoted by the color gradient) of datum is calculated based on the least number of iterations λ that PGD requires to find its misclassified adversarial variant. The light blue and light red points represent attackable data which are close the class boundary; the dark blue and dark red points represent the guarded data which are far away from the decision boundary. (Top colorbars corresponds to the value k)
15

---

Published as a conference paper at ICLR 2021
2
2
Frug Band
Horse Horse-Bird Bird
Bird-Sla Slip
Bird-Truck
Cat-Deer
Cat-Dog
Cat-Frog
Cat-Horse
Cat-Ship
Cat-Truck
Deer- Dog
Doer- Doen-Frng
Deer- Horse
Deer Ship
Deer: Traek
Frog Dog
Dog Home
Ship Dog
Dog Truck
Frog Hose
Frog-Ship
Frog- Truck
Ship-Horse
Horse-Truck
Ship- Truck
Figure 8: Part B 2-d visualizations of the model's output distribution on CIFAR-10 dataset.
16

---

Published as a conference paper at ICLR 2021
B ALGORITHMS
B.1 GEOMETRY-AWARE WSSTACE-REWEIGHHED FRIENDLY ADVERSARIAL TRAINING (GAIR-FAT)
Algorithm 3 Geometry-avee -wware -ware early stopped PGD-K-T
Input: data E X, label y y' model f, loss function E, maximum PGD step K, step T, perturbation bound e, step size a Output: friendly adversarial data 7 and geometry value <(2, v)
< 2; kxx,)) 3) 4-0
while K > 0 do
if arg max; (() # and T = then
break
else if arg max; f() # then
T T+
else
x(x, y) t x(x, v) +1
end if
11 B|a.c] (asign(Va (F + +))
K + K -
end while
GAIRAT is general method, and the friendly adversarial training (Zhang et al., 2020b) can be easily modified a geometry-aware instance-reweighteed version, i.c. GAIR-FAT.
GAIR-FAT utilizes Algorithm 3 to generate friendly adversarial data (ž. and the corresponding geometry value <(x, y). and then utilizes Algorithm 2 to update the model parameters.
17

---

Published as a conference paper at ICLR 2021
B.2 GEOMETRY-AWaRE AWARE INSTANCE- REWEIGHTED TRADES (GAIR-TRADES)
Algorithm -waae PGD for TRADES
Input: data x: E X, label y E y, model S, loss function ÊKZ. maximum PGD step K, perturbation
bound step size a
Output: adversarial data ā and geometry value <(z, y)
< + (O I); k(x, y) 0
while K > do
if arg max; [īī = then
s(r,y) +- <( v) +1 end if
σ [I sign (exigg((Vxxx(((((0ff)) + + K
+
end while
Algorithm Geometry-awarc instance-reweighted TRADES (GAIR-TRADES)
Input: network fe. training dataset 5 = (();,)))) learning rate 17, number of epochs T,
batch size m. number of batches M
Output: adversarially robust network fo for epoch ...... do
for mini-batch = 1...., .M do Sample mini-batch { y/) }m from
for 1, ..... m (in parallel) do
Obtain adversarial data di of x; and geometry value n( y:) by Algorithm 4 Calculate w(z;, y:) according geometry value <(x;; y4) by Eq. (6)
end for
Calculate the normalized
w(x.)) Eg ((ag:)5))
for each data
0 0 Vo Ei
wrlco((o((i)).)) + (
end for end for
We modify TRADES (Zhang et al., 2019) to GAIRAT version, i.e. GAIR-TRADES (Algorithms and 5). Different from GAIRAT and GAIR-FATI GAIR-TRADES employs Algorithm generate adversarial data (ī, y) and the corresponding geometry value k(æ, y). and then utilizes both natural data and their adversarial variants to update the model parameters (Algorithm 5). Note that TRADES utilizes virtual adversarial data (Miyato et al., 2016) for updating the current model. The gener- ated virtual adversarial data do not require any label information; therefore. their supervision signals heavily rely on their natural counterparts. Thus, in GAIR- HAIR-TRADES, the instance-reweighinng func- tion w applies the loss of their natural data.
In Algorithm 4, NN I) generates random unit vector. & is small constant. LxL is Kullback- Leibler loss. Algorithm 5, B > 0 is a regularization parameter for TRADES. fc is cross-entropy loss. Kr. is Kullback-Leibler loss, which keeps the same Zhang et al. (2019).
18

---

Published as a conference paper at ICLR 2021
v Decision boundary
Natural test data
Natural training data
Small gradient magnitude: mild robust overfitting issue
Large gradient magnitude: : severe robust overfitting issue
Figure 9: Illustration of the reasons for the issue of robust overfitting.
C EXTENSIVE EXPERIMENTS
C.I GAIRAT RELIEVES ROBUST OVERFITTING
In this section, we give the detailed descriptions of Figure 4 and provide more analysis and comple- mentary experiments using the SVHN dataset in Figure 10.
In Figure 4, red lines (solid and dashed lines) refer to standard adversarial training (AT) (Madry al., 2018). Blue and yellow lines (solid and dashed) refer to our geometry-awar instance- reweighee adversarial training (GAIRAT). Blue lines represent that GAIRAT utilizes the decreasing w for as- signing instance-dependent weights (corresponding to the blue line in the bottom-left panel); yel- low lines represent that GAIRAT utilizes non-increasing piece-wise w for assigning instance- dependent weights (corresponding to the yellow line in the bottom-left panel). In the upper-left panel Figure 4, we calculate the mean and median geometry values n(2z, y) of all 50K training data at each epoch. Geometry value k(x,, y) of data (a,, y) refers the least number of PGD steps that PGD methods need to generate misclassified adversarial variant. Note that when the natural data is misclassified without any adversarial perturbations, the geometry value <(a, y) 0. The bottom-left panel calculates the instance-dependent weight w for the loss of adversarial data based on the geometry value K.
In the upper-middle panel of Figure 4, the solid lines represent the standard training error on the natural training data; the dashed lines represent the standard test error on the natural test data.
In the upper-right panel of Figure 4, the solid lines represent the robust training error on the adver- sarial training data; the dashed lines represent the robust error the adversarial test data. The adversarial training/test data are generated by PGD-20 attack with random start. Random start refers the uniformly random perturbation of [-,, e] added the natural data before PGD perturbations. The step size 2/255, which is the same as Wang et al. (2019).
In the hottom-middie and bottom-right panels of Figure 4, we calculate the flatness of the adver- sarial loss (ā), R)) w.r.t the adversarial data f. In the bottom-middle panel, adversarial data refer to the friendly adversarial test data that are generated by early-stopped PGD-20-0 (Zhang et al., 2020b). The maximum PGD step number is 20; 7 O means the immediate stop once the wrongly predicted adversarial test data are found. We use friendly adversarial test data to approximate the points decision boundary of the robust model fe. The flatness of the decision boundary is ap- proximated by average of | V across all 10K adversarial test data. We give the flatness value at each training epoch (higher flatness value refers higher curved decision boundary, Figure 9).
19

---

Published as a conference paper at ICLR 2021
Wright aesignment fonctions
Natural data
PGD.20
1.0
XT ( Trining CEIE
GAIRAT (4)): Thinax srice
40
Wright vals
GAIAT WAll: Drtinn
0..5
AT (wi1) constant of
Standard ecvor
XT (w]): Tok caee
GAIRAT ( decressing w
GAIRAT (utt::
30-
GATRAT (23)) pronass
GARRAT (wal): Tot cae
Boolust
30
0.2
20-
0.0
10
0.0
0.2
0.4
0.6
0.8
1.0
20
40
100
20
40
60
100
K/K
Epordas
Epoche
Goumetry valurs statisttic
Loss flataess w.E.t. PGD-20-0 test data
Loss fistuees w.r.t. PGD-20 test data
AT (w])
wwm.m mm
300-
8-
Loes flature [[E,]]
40-
GAIRAT (42)
Loss flatuee [[,A]]
250
Geametry value
6-
800
GAIRAT (w3)
200-
150-
4-
20
100-
mean (AT)
2-
10-
median (AT)
20
40
(
100
200
80
80
100
20
40
80
100
Pgosha
Epochs
Epochs
Figure 10: Comparisons of AT (Wi, red lines) and GAIRAT (w2, blue lines and w3, yellow lines) using ResNet-18 18 on SVHN dataset. Upper-left panel shows different instance-dependent weight assignment functions w w.r.t. the geometry value k. Bottom-left panel reports the standard AT training statistic and calculates the median (dark red circle) and mean (light red cross) of geometry values of all training data at each epoch. Upper-middle and upper-right panels report natural training/test errors and robust training/test errors, respectively. Bottom-middle and bottom-right panels report the loss flatness w.r.t. friendly adversarial test data and most adversarial test data, respectively.
For completeness, the bottom-right panel uses the most adversarial test data that are generated by PGD-20 (Madry al., 2018).
The magnitude of the norm of gradients, i.e., [ is a reasonable metric for measuring the mag- nitude of curvatures of the decision boundary. Moosavi-Dezfone et al. (2019) show the magnitude of the norm of gradients upper bound the largest cigenvalues of the hessian matrix of loss w.r.t. input a, thus measuring the curvature of the decision boundary. Besides, Moosavi-Dezfooli et al. (2019) even show that the low curvatures can lead to the enhanced robustness, which echoes our results in Figure 4.
The flatness values (red lines) increases abruptly at smaller learning rates (0.01, 0.001) at Epoch 30 and Epoch 60. shows that when we begin use adversarial data to fine-tune the decision boundary of the robust model, the decision boundary becomes more tortuous around the adversarial data (see Figure 9). This leads to the severe overfitting issue.
Similar to Figure 4, we compare GAIRAT and AT using the SVHN dataset, which can be found Figure 10. Experiments on the SVHN dataset corroborate the reasons for issue of the robust overfitting and justify the efficacy of our GAIRAT. The training and evaluation settings keep the same as Figure 4 except the initial rate of 0.01 divided by 10 at Epoch 30 and 60 respectively.
C.2 DIFFERENT LEARNING RATE SCHEDULES
In Figure 11, we compare our GAIRAT and AT using different learning rate schedules. Under the different learning rate schedules, our GAIRAT can relieve the undesirable issue of the robust overfitting, thus enhancing the adversarial robustness. To make the fair comparisons with Rice et al. (2020), we use the pre-activation ResNet-18 18 (He et al., 2016). We conduct standard adversarial training (AT) using SGD with 0.9 momentum for 200 epochs on CIFAR- 10 dataset. The different learning rate schedules are in the top panel in Figure 11. The perturbation bound = 8/255, the
20

---

Published as a conference paper at ICLR 2021
I Learingg tate schodoles
0.200
Prronti doeey
daeap Cycle
0.125
Litest
4 0.100
0.075
0.050
0.025
0.000
25
50
75
100
125
150
175
200
Epoclas
PCDD-20 20
decayl
2GD-20 (laoe
PGD ost data (aneliipia des
PGD-30 data
enee (S)
Bobra lo cnee (S)
B
43-
33
30
10-
25
116
235
150
375
200
D00
125
120
170
20
20
125
100
125
200
110 125 110 Spride
175
Rporte
Natual jjisovie dicay)
Natural
Natuaal data lunik danse)
Naturá (cydie)
[
30-
30
10-
10
25
50
12
HG
235
Lio
375 200
20 D00 125 100
160
200
25
200
12 100 125 200
%
20
15 110 125
11o 175 200
Sparte
Spock
Rpodee
Figure 11: The training results of standard AT and GAIRAT using pre-activation ResNet-18 18 under different learning rate schedules CIFAR-10 dataset. The top panel reports the different learning rate schedules. The four middle panels report the robust test error on adversarial data generated by PGD-20. The four bottom panels report the standard test error on natural data. The red lines represent AT's results under different learning rate schedules. The brown, green, blue and orange lines represent GAIRAT's results of different learning rate schedules.
PGD steps number K = 10 and the step size a 2/255. The training setting keeps the same as Rice et al. (2020)'.
GAIRAT has the same training configurations (including all hyperparamter settings) including the 100 epochs burn-in period, after which, GAIRAT begins to introduce geometry-aware instance- reweighted loss. We use the weight assignment function from Eq. (6) with À = -1.
At each training epoch, we evaluate each checkpoint using CIFAR-10 test data. In the middle panels of Figure 11, we report robust test error on the adversarial test data. The adversarial test data are generated by PGD-20 attack with the perturbation bound e 8/ 8/255 and step size a = 2/255. The PGD attack has a random start, i.e, the uniformly random perturbations of [-e,e] e] are added to the natural data before PGD iterations, which keeps the same as Wang et al. (2019); Zhang et al. (2020b). Note that different from Rice et al. (2020) using PGD-10, the computational budget, PGD-20 is a more informative metric for the robustness evaluation. In the bottom panels of Figure 11, report the standard test error on the natural data.
use PGD-20 because under
Figure 11 shows that under different learning rate schedules, our GAIRAT can relieve the issue of robust overfitting, thus enhancing the adversarial robustness with little degradation of accuracy.
C.3 DIFFERENT WEIGHT ASSIGNMENT FUNCTIONS
The weight assignment functions w should be non-increasing w.r.t. the geometry value k. In Fig- ure 12, besides tanh-type Eq. (6) (blue line), we compare different types of weight assignment functions. The purple lines represent a linearly decreasing function, i.e..
w(x y) = <(x, v)
K +
Robust Overfitting's GitHub
21

---

Published as a conference paper at ICLR 2021
Wright aesignment fonctions
Natural daca
PGD-30 trst.
1.0
Suaddard test acturary (S6)
80
Robont tast accupacy
50-
Wiight vals
0.6
8
30
8
80
36
GAIRAT Incar
30
0.2
HARRAT signoid
75
0.0
GAIRAT
tacb
60
20
1000
0.0
0.2
0.4
0.6
0.8
1.0
20
40
60
100
20
40
60
100
K
Exprdass
Epoche
Figure 12: Comparisons of GAIRAT with different weight assignment functions on CIFAR-10 dataset. When GAIRAT takes constant w 1 over the training epochs, GAIRAT recovers the standard adversarial training (AT) (red lines).
The green lines represent sigmoid-type decreasing function, i.e. ((),)))K)),
w(2,y) y) o(A
(8)
where (xx) Ite-x
Figure 12 shows that compared with AT, GAIRAT with different weight assignment functions have similar degradation of standard test accuracy on natural data, but GAIRAT with the tanh-type de- creasing function (Eq. (6)) has the better robustness accuracy. Thus, we further explore the Eq. (6) with different À in Figure 13.
Wright assigmment functions
Nataral lest dasa
PGD-20 test data
1.0
Standard test menuary (95)
80
60- 55-
Wigghe valoo
0.8
AA
20-
Rollst accuracy
50-
0.5-
GAIRAT (2-
GAIRAZ (1-2)
15
0.4
GAIRAT (A
CAIRAT 13 03
8
35
40
GAIRAT 22-
15
30-
GAIRAZ
25
0.0
GAIRA
7A
-50
60
10C
0.2
0.4
20
40
100
20
10
100
K/K
Expadas
Epoche
Figure 13: Comparisons of GARAT using the tanh-type weight assignment function (Eq. (6)) with
different À on CIFAR-10 dataset.
In Figure 13, when A +00, GAIRAT recovers the standard AT, assigning equal weights to the losses of the adversarial data. Smaller À corresponds to the weight assignment function w, assign- ing relatively smaller weight to the loss of the adversarial data of the guarded data and assigning relatively larger weight the loss of the adversarial data of the attackable data, which enhance the robustness more. With the same logic, larger corresponds to the weight assignment function w, assigning relatively larger weight to the loss of the adversarial data of the guarded data and assigning relatively smaller weight to the loss of the adversarial data of the attackable data, which enhances the robustness less. The guarded data need more PGD steps x. to fool the current model; the attackable data need less PGD steps to fool the current model.
The results in Figure 13 justify the above logic. GAIRAT with smaller À (lighter blue lines) has better adversarial robustness with bigger degradation of standard test accuracy. On the other hand GAIRAT with larger À (darker blue lines) has relatively worse adversarial robustness with minor degradation of standard test accuracy. Nevertheless, our GAIRAT (light and dark lines) has better robustness than AT (red lines).
Training and evaluation details We training ResNet-18 using SGD with 0.9 momentum for 100 epochs. The initial learning rate is 0.1 divided by 10 at Epoch 30 and 60 respectively. The weight 0.007, and PGD step
decay=0.0005. The perturbation bound numbers K = 10. For evaluations, we obtain standard test accuracy for natural test data and robust test accuracy for adversarial test data. The adversarial test data are generated by PGD-20 attack with
0.031; the PGD step size
22

---

Published as a conference paper at ICLR 2021
the same perturbation bound 0.031 and the step size 0.031/4, which keeps the as Wang et al. (2019). All PGD generation have a random start, i., the uniformly random perturbation of [-c, c] added to the natural data before PGD iterations.
Note that the robustness reflected by PGD-20 test data is quite high. However, when we use other attacks such C&W attack (Carlini & Wagner, 2017) for evaluation, both blue and red lines will degrade the robustness to around 40%. We believe this degradation
due to the mismatch be- tween PGD-adversaria training and C&W attacks, which is the common deflect of the empirical defense (Tsuzuku et al., 2018; Wong & Kolter, 2018; Cohen et al., 2019; Balunovic & Vechev, 2020: Zhang et al., 2020a). We leave this for future work.
Weight aesignment functions
Natural test data
PGD-20 test data
1.0
92.5
Wegglt value
0.8
XI (A-m
87.5-
90.0
Rubast tesl mocuray
85.0
522
0.4
DAURAT (2
62.5
RAMMAT
80.0
0.2
RAIRAT
77.5
0.0
QAIPA7
75.0
60
30
100
0.2
0.4
0.6
20
40
80
100
30g
200
400
80
100
6/K
Fgaach
Epochs
Figure 14: Comparisons of GARAT using the tanh-type weight assignment function (Eq. (6)) with different À on SVHN dataset.
In Figure 14, we also conduct experiments of GAIRAT using Eq. (6) with different À and AT using ResNet-18 18 on SVHN dataset. The training and evaluation settings keep the same as Figure 13 except the initial rate of 0.01 divided by 10 Epoch 30 and 60 respectively.
Interestingly, AT (red lines) on SVHN dataset has not only the issue of robust overfitting, but also the issue of natural overfitting: The standard test accuracy has slight degradation over the training epochs. By contrast, our GAIRAT (blue lines) can relieve the undesirable robust overfitting, thus enhancing both robustness and accuracy.
C.4 DIFFERENT LENGTHS OF BURN-IN PERIOD
Natural test data
PGD-20 test data
Standard lest neeuracy (%)
55
80
70
Robust trst accurary
45
40
AT
50-
80
35
GAIRAT: epochs burn-in period
30
GAIRAT: 20 epoclt burtria period
40
75-
GAIRAT: 20 epochs burtria period
60
80
100
GAIR XT) 600 spochs bursin period
20
GAIRATI 80 spochs burrsta period
20
40
60
80
100
20
40
60
80
100
Epochs
Epochs
Figure 15: Comparisons of GAIRAT (blue lines) with different lengths of the burn-in period on CIFAR-10 dataset. The longer GAIRAT has the burn-in period, the more alike GAIRAT becomes standard AT (red lines). AT can be seen as GAIRAT with 100 epochs burn-in period. Darker blue lines represent shorter lengths of burn-in period; lighter blue lines represent longer lengths of burn-in period.
In Figure 15, conduct experiments of GAIRAT under different lengths of the burn-in period using ResNet- 18 on CIFAR- 10 dataset. The training and evaluations details are the same as Appendix C.3 except the different lengths of burn-in period in the training.
23

---

Published as a conference paper at ICLR 2021
Figure 15 shows that compared with AT (red lines), GAIRAT with shorter length of burn-in period (darker blue lines) can significantly enhance robustness but suffers a little degradation of accuracy. On the other hand, GAIRAT with longer length of burn-in period (lighter blue lines) slightly enhance robustness with zero degradation of accuracy.
C.5 DIFFERENT NETWORKS SMALL CNN AND VGG-15 13
Natural test data (ResNet-18)
PGD-20 test data (ResNet- 18)
Standard best sonazey (96)
60
80-
55
70-
Robast accurwee
-50-
45
60
50-
AT (A +:0
335
GAIRAT (A 3)
130-
40-
GAIRAT (A 0)
30
20
60
80
100
20
100
Epocks
Epechs
Natural test dltta (Small CNN)
PGD-20 test data (Small CNN)
Stundar test accaracy (%)
80
55
70
5050
Roland acourace
45
60-
40-
50-
35
40-
25
80
20
20
20
40
60
80
100
20
40
60
80
100
Epochs
Epecas
Natural test data (VGG-13)
PGD-20 test data (VGG-13)
60
Stundar bes ancurocy (96)
80
70-
50
Robusd acmumey
60-
50-
40-
AT (A +00
30-
30
GAIRAT (A 3.3)
20:
20
GAIRAT (A 3.0)
10
60
80
100
100
Epochs
Epechs
Figure 16: Comparisons of different networks (VGG-13, Small CNN and ResNet-18) which
GAIRAT and AT use on CIFAR-10 dataset.
In Figure 16, besides ResNet-18. 18, we apply our GAIRAT to Small CNN (6 convolutional layers and 2 fully-connected layers) on CIFAR-10 dataset. Training and evaluation settings keeps the same as the Appendix C.3; we use 30 epochs burn-in period and Eq. (6) as the weight assignment function.
Figure 16 shows that larger network ResNet-18 18 has better performance than Small CNN in terms of both robustness and accuracy. Interestingly, Small CNN has less severe issue of the robust overfit- ting. Nevertheless, our GAIRAT are still quite effective in relieving the robust overfitting and thus enhancing robustness in the smaller network.
In Figure 16, also compare GAIRAT with AT using VGG-13 (Simonyan & Zisserman, 2015) on CIFAR-10 10 dataset. Under the same training and evaluation settings as Small CNN, results of VGG-13 once again demonstrate the efficacy of our GAIRAT.
24

---

Published as a conference paper at ICLR 2021
C.6 GEOMETRY-AWAL INSTANCE DEPENDENT FAT (GAIR-FAT)
In this section, we show that GAIR-FAT can enhance friendly adversarial training (FAT). Our geometry-aware instahee-reweightedd method general method. Besides AT, we can easily mod- ify friendly adversarial training (FAT) (Zhang et al., 2020b) to GAIR-FAT (See Algorithm 3 in the Appendix B.1).
In Figure 17, we compare FAT and GAIR-FAT using ResNet-18 on CIFAR-10 dataset. The training and evaluation settings keeps the same as Appendix C.3 except that GAIR-FAT and FAT has an extra hyperparameter
In Figures 17, the begins from 0 and increases by 3 at Epoch 40 and 70 respectively. The burn-in period is epochs. Figure 17, we use Eq. (6) with different À as GAIR-FAT's weight assignment function.
Different from AT, FAT has slower progress in enhancing the adversarial robustness over the training epochs, FAT can naturally resist undesirable robust overfitting. However, once the robust test accuracy reaches plateau, FAT still suffers a slight robust overfitting issue (red line in the right panel). By contrast, when introduce our instance dependent loss from Epoch 70, GAIR-FAT (light and dark blue lines) can get further enhanced robustness with near-zero degradation of accuracy.
Weight assignment functions
Natural test dasa
PGD-20 test data
Shandard tes aremecy (95)
90
80
40-
Weight salie
H00
20-
87
30
0.4
GAIR LA
8.
85
GAIR-AT [A
8
20-
445
Bolnet
GAIR
GAIR-FAT (A
90
1000
10%
80
900
100
0.0
0.2
0.4
20
100
20
40
Eporiss
Epochs
Figure 17: We compare FAT and GAIR-FAT using ResNet-18 on CIFAR-10 dataset using tanh-type weight assignment function with different X.
Natural test data
PGD-20 test data
Standard test accuracy (%)
90
Robust: test arcuracy (9)
50
88
40
70
86
30-
60
70
80
90
100
50
50
FAT (y: 0-1-21
FAT (-2-4)
20
45
40
GAIR-FAT FAT (r: 0-1-2)
40
GAIR-FAT 0-2-4
GAIR-FAT (F) 0-345)
70
80
90
100
30
0
20
40
60
80
100
20
40
60
80
100
Epochs
Epochs
Figure 18: Comparisons of GAIR-FAT and FAT under different schedules of dynamical using ResNet-18 on CIFAR-10 dataset. (r: 0-1-2) refers to r starting from 0 and increasing by at Epoch 40 and 70, respectively. (r: 0-2-4) refers to starting from 0 and increasing by 2 at Epoch 40 and 70, respectively. т: 0-3-6) refers to T starting from 0 and increasing by at Epoch 40 and 70, respectively.
Note that different from the FAT used by Zhang et al. (2020b) increasing from 0 to 2 over the training epochs, we increase the r from to As shown in Figure 18, we find out FAT with smaller (e.g., 1-3) does not suffer the issue of the robust overfitting, since the FAT with smaller has the slower progress in increasing the robustness over the training epochs. This slow progress leads to the slow increase of the portion of guarded data, which is less likely to overwhelm the learning from the attackable data. Thus, our geometry-aware instance dependent loss applied on FAT with smaller does not offer extra benefits, and it does not have damage as well.
25

---

Published as a conference paper at ICLR 2021
C.7
CCOMETRY-AWARE INSTANCE DEPENDENT MART (GAIR-MART)
Weight sesignimntt functions
Natural test
PGD-20 test data
1.0
Sandard test accumncy (50
150
0.8
30
0.0
40
3
MAIT
50
84
30
0.4
GALLMAREE LA.
GARBAART (A
10
20
30
20
801
011
100
190
20-
11.11
0.6
60
100
120
20
00
100
120
K/W
Epocaas
Epocaas
Figure 19: We compare MMA, MART and GAIR-MART with different weight assginment func- tions using ResNet-18 on CIFAR-10.
In this section, we compare method with MMA (Ding et al., 2020) and MART (Wang et al., 2020b). To be specific, we easily modify MART to a GAIRAT version, i.e., GAIR-MART. The learning objective of MART is Eq. (9); the learning objective of our GAIR-MART is Eq. (10).
The learning objective of MART is
Emargin (p(5, v) + (,,,,,) - (0,(,,,)))
(9)
our learning objective of of GAIR-MART is
AGA.Rmmargi (p(#, 0), v) + BlKx (p( p(r, ((,00)) 0)) (1 (,( ()),,
(10)
where Emorgin - log (ā, 0)) - log(1 max pa (z, 0)) and pa (2x, 0) is probability (softmax key
on logits) of æ belonging class To be specific, the first term (o, 0)) is commonly used CE loss and the second term log(1 max pv (i², 0)) is margin term used to improve the
key
decision margin of the classifier. More detailed analysis about the learning objective can be found in (Wang et al., 2020b). In Eq. (9) and Eq. (10), æ is natural training data, T is adversarial train- ing data generated by CE loss, and / > Raalgaariin log(p,( (ā, 0)) w - -log(1 max Pa ( 0)) and w refers to our weight assignment kyy
regularization parameter for MART. In Eq. (10),
function.
For MMA and MART, the training settings keep the same as the and 3 For fair comparisons, GAIR-MART keeps the same training configurations as MART except that we use the weight as- signment function w (6) to introduce geometry-aware instance-reweighted loss from Epoch 75 onward. We train ResNet-18 on CIFAR-I0 dataset for 120 epochs. For MMA, the learning rate is 0.3 from Iteration 0 to 20000, 0.09 from Iteration 20000 to 30000, 0.03 from Iteration 30000 to 40000, and 0.009 after Iteration 40000, where the Iteration refers to training with one mini-batch of data; For MART and GAIR-MART the learning rate is 0.01 divided by 10 at Epoch 75, 90, and 100 respectively. For evaluations, we obtain standard test accuracy for natural test data and robust test accuracy for PGD-20 ad adversarial test data with the same settings Appendix C.3.
Figure 19 shows GAIR-MART performs better than MART and MMA. The results demonstrate the efficacy of our GAIRAT method improving robustness without the degradation of standard accuracy.
Reweighing KL loss The learning objective of MART explicitly assigns weights, not directly on the adversarial loss but KL divergence loss. We ask what if you replace their reweighting scheme (1 (2, 0)) with our w. The learning objective is
Emargin (p (2, 0), v) + (Fkz(p(ū, p(x, 0)) w.
(II)
Figure 20 reports the results: It does not have much effect on adding the geometry- instance- dependent weight to the regularization part, i.c., KL divergence loss
MMMA' GitHub
MART's GitHub
26

---

Published as a conference paper at ICLR 2021
Weipht ussipmment functions
Natural
PGD.20 datn
0.8
accemney
WART (1
45-
0.4-
GAIR MART (3
GATRMAR 12-00
Shandard toat
60
50-
86
40-
35
51
76
33
80
901
100
110
120
80
50
100
100
:20
0.0
26
0.0
0.6
20
40
60
100
120
20
60
100
120
K/K
Epoeks
Epockts
Figure 20: Comparison of MART and GAIR-MART training ResNet-18 with Eq. (11) on CIFAR-10 dataset.
C.8 PERFORMANCE EVALUATION ON WIDE RESNET (WRN-32-1) 32-
In Table 1, we compare our GAIRAT, GAIR-FAT with standard AT and FAT. CIFAR-10 dataset is normalized into [0.1]: Each pixel is scaled by 1/255. We perform the standard CIFAR-10 data augmentation: a random 4 pixel crop followed by a random horizontal flip. In AT, we train WRN- 32-10 for 120 epochs using SGD with 0.9 momentum. The initial learning rate is 0.1 reduced to 0.01, 0.001 and 0.0005 at epoch 60, 90 and 110. The weight decay is 0.0002. For generating the adversarial data for updating the model, the perturbation bound Ctrain 0.031, the PGD step is fixed to 10, and the step size is fixed 0.007. The training settings come from FAT's Github. In GAIRAT, we choose 60 epochs burn-in period and then use Eq. (6) with À 0 as the weight assignment function; the rest keeps the same as AT. The hyperparameter 7 of FAT and begins from and increases by 3 at Epoch 40 and 70 respectively; the rest keeps the same AT. In GAIR-FAN choose 60 epochs burn-in period and then use Eq. (6) with = as the weight assignment function; the rest keeps the same FAT.
As suggested by results of the experiments in Section 4.1, the robust test accuracy usually gets significantly boosted when the learning rate is firstly reduced to 0.01. Thus, save the model checkpoints at Epochs 59-100 for evaluations, among which, the best checkpoint is selected based on the PGD-20 attack since PGD+ is extremely computationally expensive. We also save the last checkpoint at Epoch 120 for evaluations. We run AT, FAT, GAIRAT and GAIR-FAT with 5 repeated times with different random seeds.
As for the evaluations, we test the checkpoint using three metrics: standard test accuracy on natural data (Natural), robust test accuracy on adversarial data generated by PGD-20 and PGD+. PGD- 20 follows the same setting of the PGD-20 used by Wang et al. (2019)². PGD+ is the same as PGours used by Carmon al. (2019)². The adversarial attacks have the same perturbation bound étest 0.031. For PGD-20, the step number is 20, and the step size = tt.st/.. There is random start, i.e., uniformly random perturbations ([-teest, +- added to natural data before PGD perturbations. For PGD+, the step number is 40, and the step size
0.01. There are random starts for each natural test data. Therefore, for each natural test data, we have 40 x = 200 PGD iterations for the robustness evaluation.
In Table 1, the best checkpoint is chosen among the model checkpoints at Epochs 59-100 (selected based on the robust accuracy on PGD-20 test data). In practice, can a hold-out validation set to determine the best checkpoint, since (Rice et al., 2020) found the validation curve over epochs matches the test curves over epochs. The last checkpoint is the model checkpoint at Epoch 120. Our experiments find that GAIRAT reaches the best robustness at Epoch 90 (three trails) and 92 (two trails), and AT reaches the best robustness Epoch 60 (five trails). FAT reaches the best robustness at Epoch 60 (four trails) and 61 (one trail). GAIR FAT reaches the best robustness at around Epoch 90 (five trails). We report the median test accuracy and its standard deviation over repeated trails.
PGD attacks with different iterations In Table 1, each defense method has five trails with five different random seeds; therefore, each defense method has ten models (five last checkpoints and five best checkpoints). In Figure 21, for each defense, we randomly choose one last-checkpoint and
*FAT's Gitllub
$ DAT's GitHub
RSTTs GitHub
27

---

Published as a conference paper at ICLR 2021
Best checkpoints
Last checkpoints
0.62
GAIRAT GAIREAT
0.62
GAIRAT
Robust accuracy (%)
0.60
EXT
Robust accuracy
0.60
FAT
GAIREAT
0.58
0.58
0.56
0.56
0.54
0.54
0.52
0.52
0.50
0.50
0.48
0.48
20
40
60
80
100
0
20
40
60
80
100
PGD iterations
PGD
iterations
Figure 21: Comparison of PGD attacks with different PGD iterations on CIFAR-10 dataset.
one best-checkpoint and evaluate them using PGD-10, PGD-20, PGD-40, PGD-60, PGD-80, and PGD-100. All the PGD attacks use the same Ctest = 0.031 and the step size a = (2.5 . (test) /100. /100. We ensure that can reach the boundary of the c-ball from any starting point within it and still allow for movement on the boundary, which is suggested by Madry et al. (2018). The results show the PGD attacks have converged with more iterations.
C.9 PERFORMANCE EVALUATION ON WIDE RESNET (GAIR-TRADES)
Table 2: Test accuracy of TRADES and GAIR-TRADES (WRN-34-10) on CIFAR-10 10 dataset
Best checkpoint
Last checkpoiit
Defense
Natural
PGD-20
PGD+
Natural
PGD-20
PGD+
TRADES 6)
84.88
0.35
56.43 0.24
54.33
85.66 = 0.33
53.31 0.25
50.11
GAIR-TRADES
86,99
0.31
63.32 0.50
56.77
0.87
86.86
0.26
60.65 + L.0
0.93
GAIR-TRADES with TRADES. CIFAR- 10 dataset normalization and augmentations keep the same as Appendix C.8. Instead, we use WRN-34-10 10, which keeps the same as Zhang et al. (2020b). We train WRN-34-10 for 100 epochs using SGD with 0.9 momentum. The initial learning rate is 0.1 reduced to 0.01 and 0.01 at epoch 75 and 90. The weight decay is 0.0002. For generating the adversarial data for updating the model, the perturbation bound Etrain 0.031, the PGD step is fixed to 10, and the step size is fixed to 0.007. Since TRADES has a trade-off parameter B, for fair comparison, our TRADES uses the same 8 6. In GAIR-TRADES, we choose 75 epochs burn-in period and then use Eq. (6) with À = -1 as the weight assignment function. We run TRADES and TAAR-TRADES five repeated trails with different random seeds.
In Table 2, compare
The evaluations are the same Appendix C.8 except the step size = 0.003 for PGD-20 attack, which keeps the same as Zhang et (2020b)7.
In Table 2, the best checkpoint is chosen among the model checkpoints at Epochs 75-100 (w.r.t the PGD-20 robustness). The last checkpoint evaluated based on the model checkpoint at Epoch 100. Our experiments find that GAIR- TRADES reaches the best robustness at Epoch 90 (three trails), 96 (one trail) and 98 (one trail), and TRADES reaches the best robustness at Epoch 76 (three trail). 77 (one trails) and 79 (one trail). We report the median test accuracy and its standard deviation over repeated trails.
Table 2 shows that our
can have both improved accuracy and robustness.
C.10 BENCHMARKING ROBUSTNESS WITH ADDITIONAL UNLABELED (U) DATA
In this section, verify the efficacy of GAIRAT method by utilizing additional 500K U data pre- precpeeceeeed by Carmon et al. (2019) for CIFAR-10 dataset.
Carmon et al. (2019) scratched additional U data from 80 Million Tiny Images (Torralba et al., 2008); then, they used standard training to obtain a classifier to give pseudo labels to those U data.
TRADESS's GitHub
28

---

Published as a conference paper at ICLR 2021
Among those U data, they selected 500K U data (with pseudo labels). Combining 50K labeled CIFAR-10's training data and pseudo-labeled 500K data, they propose a robust training method named RST which utilized the learning objective function of TRADES, i.e,
(c + (lxx(
(12)
where ä is generated by PGD-10 attack with CE loss.
Based the RST method, introduce instance-reweigghtng mechanism, i.e., our GAIR-RST. To be specific, we change the learning objective function to
(c y) + 3 (wlxx((o())) So(z)) + (1 (oo
(13)
where the &cw refers the adversarial data generated by C&W attack (Carlini & Wagner, 2017)
and w is the as Eq. (6).
Table 3: Evaluations using standard WRN-28-11 10
Method/Paper
Natural
AA
Gowal et al. (2020)
89.48
62.60
Wu et al. (2020)
88.25
60.04
GAIR-RST (Ours)
89.36
59.64
Carmon et al. (2019)
89.69
59.53
Sehwag et al. (2020)
88.98
57.14
Wang et al. (2020b)
87.50
56.29
Hendrycks et (2019)
87.11
54.92
The results of other methods reported AA's GitHub
In Table 3, we compare the performance of our GAIR-RST with other methods that use WRN-28- 10 under auto attacks (AA) (Croce & Hein, 2020). All the methods utilized the same set of U data which are from RST's GitHub and the results are reported on the leaderboard of AA's GitHub". Our GAIR-RST use the same training settings (e.g., learning rate schedule, Grrain = 0.031) as RST. The evaluations are on the full set the AA (Croce & Hein, 2020) with étest = 0.031, which keeps the same as training.
The results show our geometry-aware instance-reweightedd method can facilitate competitive model by utilizing additional U data.
*RST's GitHub AA's GitHub
29


Reviews:
[{"review_id": "eFk4zWtOKes", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper focuses on adversarial learning. It improves the robustness while keeping the accuracy. To achieve this point, the authors find that adversarial data should have unequal importance, which naturally brings geometry-award instance-reweighted adversarial training (GAIRAT).\n\nPros:\n1. The paper has strong novelty in philosophy level. The common belief is that robustness and accuracy hurt each other. However, this paper shows that the robustness can be improved while keeping accuracy. As far as I know, this point has never been explored before.\n\n2. The paper is well motivated and easy to follow. First, the authors use Figure 1 to illustrate the GAIRAT, which explicitly gives larger weights on the losses of adversarial data. The authors use two toy examples in Figure 3 to explain GAIRAT more. Second, the whole logic of this paper is easy to follow. For example, after explaining motivations of GAIRAT, we can clearly see the objective function of GAIRAT and its realization.\n\n3. The paper is sufficiently justified in experiments. For example, PGD-200 has been used to verify the robustness of GAIRAT. From my personal opinion, this result is quite strong. Moreover, the authors upgrade their method by incorporing FAT and verify the robustness of GAIR-FAT.\n\nCons:\n1.In the top right panel of Figure 10, the SVHN experiments have a period of increasing robustness training error for GAIRAT. Could you explain this? \n\n2.Although authors show that model capacity is not enough in adversarial training, how large the DNN should be enough? What do you think?", "extra_scores": {}, "extra_text": {}}, {"review_id": "d4JvKmvAuO6", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe paper focused on the sample importance in the adversarial training. The authors firstly revealed that over-parameterized deep models on natural data may have insufficient model capacity for adversarial data, because the training loss is hard to zero for adversarial training. Then, the authors argued that limited capacity should be used for these important samples, that is, we should not treat samples equally important. They used the distance to the decision boundary to distinguish important samples and proposed geometry-aware instance-reweighted adversarial training. Experiments show the superiority over baselines. \n\nPros:\n- The finding on insufficient model capacity is very interesting. The following motivation for GAIRAT is intuitive and well explained. \n- The authors proposed a realized measurement to compute the distance to the decision boundary. This is inspiring for a series of decision-based work. \n- The experiments demonstrate the effectiveness of the proposed method. \n\nCons:\n- Treating data differently has been investigated in related work like MART and MMA. The authors should discuss the difference from these methods. \n- The capacity analysis provides a very good perspective to analyze adversarial training, however, the explanations in Figure 2 are a little bit weak. \n- The weight function of Eq. (6) lack some intuitive explanations. Why such a formula? Why choose these constants?\n- PGD steps are also investigated in CAT and DAT papers. The authors should also discuss the difference to them. \n- The experiments should compare with some baselines considering the example difference, such as MART, MMA. \n- The evaluations should test some modern white-box attacks, like auto-attack, only PGD is not convincing. Besides, Black-box attacks should be tested for a complete evaluation and checking the obfuscated gradients. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Pji8Q4aybzs", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper challenges the common belief of the inherent tradeoff between robustness and accuracy.\nInstead of recent methods improving accuracy while maintaining robustness, this paper proposes a geometry aware instance reweighed adversarial training (GAIRAT) method to improve robustness while maintaining accuracy. \n\nPros:\n1 The direction---improving robustness while maintaining accuracy---is novel and interesting. \n\nSpecifically, several papers are challenging the inherent tradeoff, e.g., using more data [1], utilizing early stopped PGD [2], and incorporating dropout [3]. This paper still challenges the inherent tradeoff. \nHowever, different from [2,3] improving accuracy while maintaining robustness, this paper goes the other direction. \nTo my knowledge, this is the first paper to explore this direction. \n\n[1] Understanding and Mitigating the Tradeoff Between Robustness and Accuracy, ICML 2020\n[2] Attacks Which Do Not Kill Training Make Adversarial Learning Stronger, ICML 2020\n[3] A closer Look at Accuracy vs. Robustness, NeurIPS 2020\n\n2 This paper has made two conceptual improvements. a) This paper explicitly argues that the overparameterized networks that have enough model capacity in standard training suffer from the insufficiency in adversarial training (though many studies have already shown AT needs the large model). b) This paper argues that under limited model capacity, adversarial data should have unequal importance. Unequal data's treatment was explored in the traditional ML methods several years ago, but it is rare in deep learning at this moment. \n\n3 The proposed GAIRAT method is effective, indeed increasing robustness while retaining accuracy. The experiments are comprehensive over different network structures, datasets and attack methods. The experiments in the appendix provide much useful information. \n\n\nCons:\n1.The design of weight assignment function in Section 3.3 seems heuristic. Would you explain some principles on assigning instance dependent weights? \n\n2.In Figure 4, the GAIRAT method can relieve undesirable robust overfitting. Would you explain more about this? For example, why the robust overfitting exists in standard adversarial training? how/why your GAIRAT methods relieve it?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an insightful study on the robustness and accuracy of the model. It was hard to simultaneously keep the robustness and accuracy. A few works tried to improve accuracy while maintaining the robustness by investigating more data, early stopping or dropout. From a different perspective, this paper aims to improve robustness while maintaining accuracy. 

There are some interesting findings in this paper, which could deepen our understanding of adversarial training. For example, the authors conducted experiments with different sizes of the network in standard training and adversarial training. The capacity of an overparameterized network can be sufficient for standard training, but it may be far from enough to fit adversarial data, because of the smoothing effect. Hence given the limited model capacity, adversarial data all have unequal importance. Though this technique is simple and widely studied in traditional ML, it is an interesting attempt in adversarial ML and the authors provide extensive experimental results to justify its effectiveness. 

In the authors' responses, the concerns raised by the reviewers have been well addressed. The new version becomes more complete by including more results on different PGD steps and the insights on designing weight assignment function. Also, the authors gave an interesting discussion on enough model size for the adversarial training, though it is still kind of an open question. I would thus like to recommend the acceptance of this paper.  
Decision: Accept (Oral)