Vulnerability-Aware Instance Reweighting For Adversarial Training
Authors: Olukorede Fakorede
Ashutosh Kumar Nirala
Modeste Atsague
Jin Tian
Venue: Accepted by TMLR ()

Abstract: Adversarial Training (AT) has been found to substantially improve the robustness of deep learning classifiers against adversarial attacks. AT involves obtaining robustness by including adversarial examples in training a classifier. Most variants of AT algorithms treat every training example equally. However, recent works have shown that better performance is achievable by treating them unequally. In addition, it has been observed that AT exerts an uneven influence on different classes in a training set and unfairly hurts examples corresponding to classes that are inherently harder to classify. Consequently, various reweighting schemes have been proposed that assign unequal weights to robust losses of individual examples in a training set. In this work, we propose a novel instance-wise reweighting scheme. It considers the vulnerability of each natural example and the resulting information loss on its adversarial counterpart occasioned by adversarial attacks. Through extensive experiments, we show that our proposed method significantly improves over existing reweighting schemes, especially against strong white and black-box attacks.

Published in Transactions on Machine Learning Research (07/2023)
Vulnerabilit--Aware Instance Reweighting For Adversarial Training
Olukorede Fakorede*
fakorede @iastate.ed edu
Deportment of Computer Science lowa State University
Ashutosh Nirala
aknirula Giastate. edu
Deportment of Computer Science lowa State University
Modeste Atsague
modeste@iastate edu
Department of Computer Science Towa State University
Jin Tian
jtian Giastate edu
Department of Computer Science lowa State University
Reviewed on OpenReview: https: // openreview. net/ forum? id=kdPtdd
Abstract
Adversarial Training (AT) has been found to substantially improve the robustness of deep learning classifiers against adversarial attacks. AT involves obtaining robustness by includ- ing adversarial examples in training a classifier. Most variants of AT algorithms treat every training example equally. However, recent works have shown that better performance achievable by treating them unequally. In addition, it has been observed that AT exerts uneven influence different classes in training set and unfairly hurts examples corre- sponding to classes that are inherently harder to classify. Consequently, various reweighting schemes have been proposed that assign uncqual weights to robust losses of individual exam- ples in a training set. In this work, we propose a novel instance-wise reweighting scheme. It considers the vulnerability of each natural example and the resulting information loss on its adversarial counterpart occasioned by adversarial attacks. Through extensive experiments, show that our proposed method significantly improves over existing reweighting schemes, especially against strong white and black-box attacks.
1
Introduction
The practical application of deep learning in safety-critical domains has been thrown into doubt following the observed brittleness of deep learning to well-crafted adversarial perturbations (Szegedy et al. 2013). This chilling observation has led to array of methods aimed at making deep learning classifiers robust to these adversarial perturbations. Prominent among these proposed methods is adversarial training (Goodfellow eet al. 2015 Madry et al. 2018). Advensarial training (AT) is an effective method that typically involves the introduction of adversarial examples in training a deep learning classifier. Several AT variants have
been proposed yielding modest improvements (Wang et al.] 2019 Ding al.] 2019 Kannan al.] 2018 Zhang et al. 2019). More recently, methods have been proposed to boost the performance of the existing
AT variants further, including adversarial weight perturbation (Wu et al.
2020 utilizing hypersphere

---

Published in Transactions on Machine Learning Research (07/2023)
embedding (Pag et al., 2020 Fakorede et al. 2023 and augmenting dataset with unlabeled and/or extra labeled data (Carmon et al. 2019 Alayrac et al.. 2019 Zhai et al. 2019h.
Despite the generally impressive performance of AT against adversarial attacks, Xu et al. (2021) raised concerns about fairness in AT. It is observed that AT encourages significant disparity in natural and robust accuracy among different classes. Furthermore, AT disproportionately hurts the robust accuracy of input examples that are intrinsically harder to classify by a naturally trained classifier. For example, a naturally trained PreactResNet-18 on the CIFAR-10 dataset classifies "cat" and "ship" at approximately 89% and 96% accuracy, respectively, while the robust accuracy of "cat" and "ship" produced by an adversarially trained PreactResNet-- on PGD-attacked CIFAR-10 datasct are approximately 17% and 59% respectively (Xu) et al. 2021). In other words, the gap between the standard accuracy and robust accuracy for "cat" is 72%, whereas the difference between the standard and robust accuracy for "ship" is 37%. This disparity suggests that adversarial examples corresponding to certain classes may be treated differently by AT.
Adversarial examples are crafted from natural examples that exhibit varying degrees of intrinsic vulnerability measured by their closeness to the class decision boundaries. Intuitively, adversarial examples corresponding to intrinsically vulnerable examples are moved farther across the decision boundary into wrong classes which makes them casy to misclassify. (Zhang et al. (2020) observed that AT encourages learning adversarial variants of less vulnerable natural examples at. the expense of the intrinsically susceptible ones as the training proceeds. This may partly explain the phenomenon of robust overfitting (Chen et al. 2020 Zhang et al. 2020). Therefore, the performance of AT may be improved by assigning higher weights to the robust losses of adversarial variants of vulnerable natural examples.
The idea of reweighting robust losses of adversarial examples has recently been explored in the literature. GAIRAT (Zhang et al.] 2020) assigns smaller or larger weights to the robust losses of adversarial examples based on the geometric distance of their corresponding natural counterpart to the decision boundary. Specif- ically, GAIRAT measures the geometric distance using the least number of PGD steps needed to misclassify the natural example. As a result, the GAIRAT reweighting function can only take a few values (correspond- ing to discrete PGD steps) and is unstable because its value depends largely on the initial starting point of each natural example which may change depending on the attack path (Liu et al.) 2021). Liu et al. (2021) proposed rewcighting robust losses based on the margin between the estimated ground-truth probability of the ad adversarial example and the probability of the most confusing label. We contend that this method does not exploit information about the intrinsic vulnerability of a natural example. More importantly, we observe that these methods only yield competitive robustness on attacks like PGD and FGSM, but underperform against stronger white-box or black-box attacks CW, AA, or SPSA.
This paper proposes a novel instanco-wise weight assignment function for assigning importance to the robust losses of adversarial examples used for adversarial training. Our weight assignment function considers the intrinsic vulnerability of individual natural examples from which adversarial examples used during training are crafted. We capture the intrinsic vulnerability of each natural example using the likelihood of it being correctly classified, which we estimate using the model's confidence about the example belonging to its true class. In addition, we argue that adversarial attacks have a unique impact on each individual example. This contributes to the disparity in robustness accuracy exhibited by examples of different classes. Hence, we compute the discrepancies between a model's prediction on cach natural example and its corresponding adversarial example as another measure for the vulnerability of the example.
We summarize the contributions of this paper as follows:
1. We propose a novel Vulterebility-awaree Instance Reweighting ( (VIR) function for adversarial training, The proposed reweighting function takes consideration of the intrinsic vulnerability of individual examples used for adversarial training and the information loss occasioned by ad adversarial attacks on natural examples.
2. We experimentally demonstrate the effectiveness of the proposed reweighting strategy in improving adversarial training.
3. We show that existing reweighting methods GAIRAT (Zhang et al. 2020) and MAIL et al., 2027) only yield significant robustness against attacks FGSM and PGD at the expense of stronger
2

---

Published in Transactions on Machine Learning Research (07/2023)
attacks CW (Carlini & Wagner 2017 Autoattack((roce & Hein 2020b). and FMN Pintor et al,
2027p. Using various datasets and models, we show that the proposed VIR method consistently improves over the existing reweighting methods across various white-bos and black-box attacks.
2 RELATED WORK
Adversarial Attacks. Since it became known that deep ncural networks (DNN) are vulnerable to norm-
bounded adversarial attacks (Biggio et al. 2013 Szegedy et al.
2013), a number of sophisticated adversarial
attack algorithms have been proposed [Goodieloww et al. 2015
Madry et al. 2018 Carlini & Wagner 2017] 2016). Adversarial attacks can broadly be
Athalye et al. 2018 Dong et al. 2018 Moosavi-Dezfooli et al.
classified into white-box and black-box attacks. White-box attacks are crafted with the attacker having full
access to the model parameters. Prominent white-box allacks include Fast Gradient Sign Method (Good- fellow et al.) 2015). DoepFool (Moosavi-Dezfooli et al. 2016), C&W (Carlini & Wagner 2017), Projected Gradient Descent (PGD) (Madry et al. 2018) etc. On the other hand, in black-box settings, the attacker has no direct access to the model parameters, and black-box attacks usually rely on a substitute model
(Papernot et al. 2017), or gradient estimation of the target model (Tlyas al. 2018 Uesato et al. 2018
Adversarial Robustness. To mitigate the potential threat of adversarial attacks, exlensive research has been conducted, leading to various methods (Guo et al. 2018 Song et al. 2017 Papernot et al. 2016 Madry 2018 Zhang et al. 2019 Atsague et al.] (022)) However, some of the proposed methods were later
ct al.
found
to be ineffective against strong attacks (Athalye et al. 2018). Adversarial Training (AT) (Goodfellow 2015 Madry et al., 2018), which requires training a classifier with adversarial examples, has been
et al.
found to be effective to a degree in achieving robustness to adversarial examples. Formally, AT involves
crafting adversarial examples during training and solving a saddle point problem formulatod as:
min ((x)))D
max L(fo('), (x)
(1)
where y is the true label of input feature x, LO is the loss function, O are the model parameters, and (x) (x' e X |xx' - xIIp c) represents the Ip norm ball centered around x constrained by radius In Eq. ( the inner maximization tries to obtain worst-case adversarial version of the input x that increases the loss. The outer minimization then trics to find model paramcters that would minimize this worst-case adversarial loss. The relative success of AT (Madry et. al. 2018) has inspired various AT variants such as Zhang et al. 2019 Wang et al. 2019 Ding et al. 2019 Kannan et al. 2018), to cite a few.
Re- roighegh Recent works by Zhang ct al. (2020) and Liu et al.) (2021) have argued for assigning different weights to the losses corresponding different veresariaal examples in the training set. [Zhang et al. (2020) assigns weights to an example based on its distance to the decision boundary. Examples that are closer to the decision boundary are assigned larger weights as follows:
w(xi, vi) 1 + tanh(((A + 5 (1 2k(,,, 3))KK))
where k(x, y1) is the least number of PGD steps to cause misclassification of x;, K = 10 is the number of PGD stops used for generating the attack, and is set to -1. The assignment function in Eq. (2 is used to reweight the robust losses on each adversarial example in Eq. (1
Unlike (Zhang al.) 2020) which uses a re-weighting function that is discrete (i.e. the weighting function depends on k PGD iterations) and path-dependent, (Liu al. (221)) proposed a re-weighting scheme that is continuous and math-independent based on the probability margin between the estimated ground-truth probability of an adversarial example and the probability of the class closest to the ground-truth as follows:
(M y: 0) = fo('') max fo(x');
(3)
3.77
The weight assignment function is then defined as:
w(x;, v:) sigmoid((1(PM))) (3))
(4)
3

---

Published in Transactions on Machine Learning Research (07 (022))
where y and & are hyperparameters.
This work takes a different perspective on reweighting robust losses, We consider the inherent vulnerability of the natural examples used for crafting adversarial examples and the impact of adversarial attacks on each natural example.
3 PRELIMINARIES
We use bold letters to denote vectors. We denote D (x,, Vili=1 as a data set of input feature vectors X; E C Rd and labels y/4 E V, where X' and y represent a. feature space and label set, respectively.
Let Jo X - Rº denote deep neural network (DNN) classifier parameterized by 0 where C represents the number of output classes. For any X, let the class label predicted by fo be Fa(x) arg max; fo()) where (x)x) denotes the k-th component of fo(x). fo(x), is the likelihood of x belonging to class KL(PIIQ) is used to represent the Kullback-Leibler divergence of distributions P and Q.
We denote ll as the 1p- norm over R., that is, for vector neighborhood for x is defined (x) {x' E X ||x xlIp I e). An adversarial example corresponding to a natural input x is denoted as x'. We refer to a DNN trained only on natural examples as standard network, and the one trained using adversarial examples as robust network or adversarially trained network.
Rd |xx|| (E (x) An
Standard Training
Vanila-Ar
TRADES (LI,
4.01
MART
945
6
25
614
25
90
55
51
27
20
40
149
52
372
13
70
21
19
10
20
21
664
11
05
45
52
30
18
2B
I0L
34
RTO
2
1
6
26
21
705
10
22
10
13
42
137
150
15
128
16
905
16
12
11
122
11
28
10
202
120
367
74
165
105
16
4
39
12
393
144
412
104
204
90
19
14
13
52
29
0
203
378
68
236
47
407
48
62
15
43
126
34
571
354
35
5
16
29
7
99
326
33.5
83
31
17
326
153
530
560
22
50
07
321
92
40
624
15
27
27
161
38
18
27
11
15
16
55
134
20
16
17
16
40
661
52
133
23
59
18
31
28
41
66
679
LIGNES
Figure 1: Confusion matrix displaying standard accuracy of ResNet18 the CIFAR-10 dataset and its robust accuracy under the PGD-100 attack trained by AT variants vanilla-A7 (Madry et al. 2018), TRADES (Zhang et al. 2019 and MART (Wang et al.) 2019) respectively.
4 PROPOSED METHOD
The proposed approach motivated by two major factors: (1) the unfairness exhibited by adversarial training adversarial examples crafted from intrinsically vulnerable natural examples are underrepresented in the later stages of adversarial training (Zhang et al. 2020), (2) the discrepancy between individual natural examples and their corresponding attacked variants.
We propose an effective reweighting strategy for assigning importance to robust losses based on these two factors. We will make a case for how the information about the inherent vulnerability of natural examples and the effect of adversarial attacks on these examples may be captured and utilized as inputs to the proposed weight assignment function.
4.1 Vulnerability of Natural Examples
Based on the findings in (Xu et al. 2021 Ma et al. 2022), supported by the confusion matrices in Figure is clear that adversarial training hurts adversarial examples of certain classes more than others, especially adversarial examples crafted from natural examples which are harder classify under natural training.
These natural examples characterized by their closeness to the class decision boundaries (Xu et al.
4

---

Published in Transactions on Machine Learning Research (07/2023)
Zhang et al. 2020 Following these findings, we propose reweighting component to pay attention to adversarial examples whose natural counterparts are more difficuul to classify.
2021
We argue that a natural example being vulnerable is suggestive of the features of that example correlating to another (wrong) but semantically similar class. Hence, a vulnerable example may be classified with reasonably high confidence to the wrong class. Findings in (Wang, 2021 also indicate that classes that are semantically closer to each other have smaller distances to their decision boundaries; e.g., in the CIFAR-10 dataset, 'cat' is semantically closer to 'dog' than 'airplane,' and 'cat' and 'dog' have smaller distances to the decision boundaries than 'airplane. In Figure [ classes '3' ('cat") and '5' ('dog'), which are semantically similar, exhibit more vulnerability under natural training. Input examples corresponding to these classes contain certain similar feat We believe that this relative feature similarity in semantically closer classes makes models less confident in distinguishing their examples. Given this insight, we define the vulnerability of a natural example in terms of a model's is estimated class probability.
Given a natural input probability of x belonging to y is given as fo(x) Note that y is not necessarily the same arg maxx fo (x)): We formally define the notion of relative vulnerability of inputs below.
with its corresponding true label y; and a model fo(.), the model's estimated
Definition 4.1 (Relative vulnerability of examples.) Given two input-label pairs (0I,V1) and (22, we say the pair 81) is more vulnerable than (22, if fo(2))), < fz2)))
Based on Definition 4.1, model fa(.) is less confident in estimating the true class of more vulnerable examples. We provide more theoretical explanation to our notion of relative example vulncrability in Appendix A
Given that adversarial training involves training on adversarial examples, and it unfairly hurts adversarial examples crafted from vulnerable examples, it is intuitive that adversarial training sets up its decision bound- ary to favor adversarial examples of inouinerable classes, as observed in (Xu et al. 2021). This is indicated by the relatively low robust accuracy recorded on certain classes. Given this insight, we consider assign- ing unequal weights to adversarial examples based on the relative vulnerability of their original (natural) examples with respect to their respective true labels.
Our proposed re-weighting strategy for adversarial training considers the vulnerability of each natural exam- ple used to generate adversarial examples for training. This vulnerability for an input x with label y is given by fo(x)y. Adversarial examples corresponding to natural examples which are intrinsically vulnerable are assigned higher weights. We define score function denoted S, y) for adaptively assigning importance to input example x based on model's confidence that x belongs to the true label y. The score function is as follows:
Sx vk) = a - - 77folxi
(5)
where x; is a natural example, ys is the true label of xi, el()) represents an exponential function, and y >= 1.0 is a real-valued hyperparameter. a is introduced for numerical stability to ensure that S, values are not too small and are useful. The formulation in Eq. 5 ensurees that higher values are returned for more valnerable examples which have lower fx())), values. Higher y values will result in a larger disparity in weights between vulnerable examples and less vulnerable examples.
4.2 Disparity between Natural and Adversarial Examples
The likelihood estimate of correctly classifying a natural example provides information about its intrinsic vulnerability before an adversarial perturbation is applicd to it. However, does not supply information about the discrepancies in the features learned by the model on natural examples and their corresponding adversarial variants, which largely explain the large disparity between natural and robust errors.
Iyas et al. 2019) and Tsipras et al. (2018) characterized Iearned fcatures into robust and non-robust. Robust features refer to features that remain correlated to the true label under ad adversarial perturbation. In contrast, non-robust features are highly predictive, however, they are brittle and can be anti-correlated with the true label under adversarial perturbations. [Tspras et al. (2018) showed that a DNN classifier able to learn any useful features on natural examples, but learns only robust features on adversarial examples, assigning zero infinitesinal weight values to the predictive non-robust features.
5

---

Published in Transactions on Machine Learning Research (07/2023)
The depletion of features in the adversarial examples results in different information loss for different ex- amples. We note that there may be variation in non-robust features learned by DNN classifier in the different adversarial examples used in training. For instance, consider two input-label pairs (x1, (x1, (x2, and their corresponding perturbed variants X1, X2. The robust features in x may have a stronger correlation to Y1 than the robust, features in x2 to 1/2 In fact, intrinsically vulnerable examples which are relatively weakly correlated with their classes may be affected more by an adversary since their features are further depleted. We propose score the vulnerability of an example x as weight for by the discrepancy be- tween the model's output prediction on and that on its adversarial variant x'. For simplicity, we score this discrepancies using the KT--divergence as follows:
Sa(x,, x)) = (L(fo(xx)llfo(///
(6)
where Sy denotes the proposed discrepancy score function, and (x) and fa (x') denote the model's predic- tions on natural and the corresponding adversarial examples respectively.
We note that the KI-divergence between a model's predictions on natural examples and adversarial examples is characterized a boundary error in TRADES (Zhang et al.) 2019), and is utilized as regularization term for improving robustness (see Eq. 9b. In contrast, here, it is used as weighting (a value) and the
gradient of the KL-divergence in Eq. 6 is not used.
4.3 Weight Assignment Function
We propose weight assignment function for re-weighting robust losses based on the observations in Sec-
tions 4.1 and 4.2 The proposed Vulnerability-aann -wwarerre Instance. Reweighting (VTR) function for assigning importance to the example I; in a training set is as follows:
w(x;, X4, vc) = Sx(,, v.)
+
where 3 is a hyperparameter. Sx(x,)) y) ensures that emphasis is given to vulnerable examples which are dis- proportionately hurt by adversarial training. The proposed weight assignment function assigns the largest weights to robust losses corresponding to vulnerable examples having the most significant discrepancy be- tween the model's outputs on the natural and adversarial variants. Furthermore, the least weights are assigned to robust losses corresponding to invulnerable examples having the least discrepancy between the model's outputs on them and their corresponding adversarial variants. The hyperparameter | allows lower-bound the reweighting function and balance the relative strength of the lowest and highest weight values.
As comparison, GAIRAT (Zhang et al. 2019 assigns importance to robust losses based on the h PGD steps required to atlack an example x. The intuition is that vulnerable inputs take fewer steps to move across the decision boundary. However, as noted in (Liu et al. 2021), this approach is problematic because it is path-dependent, Le, two inputs with similar starting points may reach their end points using different paths, thus making it unreliable. In addition, the reweighting function constrained to accepting a few discrete values. In contrast, the proposed reweighting function in eqn 7 takes continuous values and is path-independent.
4.4 Applying the Weight Assignment Function
We apply the proposed weight assignment function to prominent adversarial training methods vanilla AT (Madry et al. 2018) and TRADES (Zhang et al. 2019 by assigning importance to robust losses computed by these adversarial training methods. Specifically, we re-write the training objective of the vanilla AT as:
Σ
u(x, vi) LCE(fo(x))
where LOB refers to the eross-entropy loss function. The TRADES robust loss, originally stated as:
Σ Lce(fo()),))+ {I(f((x))ll((())
(9)
6

---

Published in Transactions on Machine Learning Research (07 (07/202) 2022)
is re-written as:
Σ Le w(x,, x1) -I((f(xx))ll(((x)
(10)
where À is regularization hyper- parameter. We term the training objectives in Eq. and 10 VIR-AT and VIR-TRADES respectively.
Burn-in Period. During the training, the weight w(x,, x4, 3i) is set to in the initial epochs. The application of the proposed weight assignment function is delayed to later epochs. This is because at the initial training phase, the deep model has not sufficiently learned, and thus is less informative. Disregarding this fact may mislead the training process. Zhang et al. (220)) used a similar approach in implementing their re-weighting strategy.
Our proposed VIR-AT algorithm is summarized in the following:
Algorithm VIR-AT Algorithm.
Input: neural network model with the parameters 0, step sizes Ki and Ki, and a training dataset D of size
Output: a robust model with parameters 8*
for epoch = to num num_ _empchhh__ss_____ do for batch to num batchs do sample mini- batch {(T,, y. from D;
mini-batch of size M. x - 0.001 : (0 1), where (N00,) /) is the Gaussian distribution with zero mean and
4:
for i= to M do
5:
6:
identity variance. for
7:
do
x + [x,,(x.) (x; + K1 sign(Vx; ((fo(x))..))
D II is a projection operator.
end for
(x,, 9i) e (fo(xi))
10:
11:
Sa(x xi) - (L(fo(x:) (x(x)
12:
w/ (X,, X,, )1) + (x,, 9/ Sa(x x44) + 3;
b (x,,,,x( v1) + 1 if epoch S 76
13:
end for
14:
- K2 a E (x,,
x w:) L( (
15: end for
16:
end for
5 EXPERIMENTAL SECTION
In this section, we verify the effectiveness of the proposed re-weighting function through extensive experiments various datasets including CIFAR-10 (Krizhevsky al.
2009
CIFAR-100 (Krizhevsky et al. 2009)
SVHN Netzer et al.) 2011). and TinyImageNet (Deng et al.
2009
We employed ResNet-18 (RN-18) (He
eet al. 2016) and Wide Nest-34400 (WRN-34-10) (He et al.
2016) as the backbone models for exploring
the effectiveness of the proposed method on CIFAR-10, while evaluated on Net-
CIFAR-100, SVHN, and TinylmageNet are
5.1 Experimental Settings
The models are trained for 115 epochs, using mini-batch gradient descent with momentum 0.9, batch size 128, weight decay 3.5e-3 (RN-18) and 7e-4 (WRN-34-10). The learning rates are set to 0.01 and 0.1 for RN-18 and WRN-34-10 respectively. In both cases, the learning rates are decayed by a factor of 10 75th, and then at 90th epoch. In VIR-AT and VIR- TRADES, we introduced the proposed reweighting function the 76th epoch following (Zhang et al.
2020

---

Published in Transactions on Machine Learning Research (07/2023)
The adversarial examples used during training are obtained by perturbing each image using the Projected Gradient Descent (PGD) (Madry et al. 20188) with the following hyperparameters: loc norm 88/255, step-size K = 2/255, and K = 10 iterations.
5.2 Baselines
We compare the robustness obtained using VIR-AT and VIR-TRADES with prominent AT methods vanilla- AT (Madry et al., 2018), TRADES (Zhang et al. 2019). and MART (Wang et al. 2019). In addition, we
compare with the state-of- teaf-the-rtt art reweighting schemes GAIRAT (Zhang al. 2020 and MAIL (Liu et al.
2021). We conducted additional experiments on recent data augmentattion-aase defense (Wang et al. 2023 and the results are provided in Appendix B
5.3 Hyperparameters
Baseline Hyperparanntters. The trade-off hyperparameter 1 is set to 6.0 for training WRN-34-10 and 4.0 for RN-18 with TRADES. As recommended by the authors, we set the regularization hyperparameter B to 5.0 for training with MART.
VIR Hyperparameters. The values of constants a and 8 heuristically determined and set to 7.0 and 0.007 respectively in VIR-AT and 8.0 and 1.6 in VIR-TRADES Similarly, set the value of y to 10.0 and 3.0 in VIR-AT and VIR- TRR-TRADES respectively. We set the value of y to 3.0 for training TinylmageNet with VIR-AT. Also, the value of 1 is set to 5.0 for training VIR- VR--TRDDE..
x
5.4 Threat Models
The performance of the proposed reweighting function was evaluated using attacks under White-bor and Black-box settings and Aulo allack.
White-box attacks. These attacks have unfettered access to model parameters. To evaluate robustness CIFAR-10 using RN-18 and WRN34-10, we apply the Fast Gradient Sign Method (FGSM) (Goodlellow et al. 2015) with 8/255 PGD attack with 8/255, step size
1/255, K 100; CW (CW loss Carlini & Wagner 2017) optimized by PGD-20) attack with c = 8/255, step size 1/255. In addition, we evaluated the robustness of trained models against 100 iterations of lox version of Fast Minimum-norm (FMN) attacks (Pintor et al. 2021). On SVHN and CIFAR-100, we apply PGD attack with = 8/255, step size K = 1/255, K 100. We limited the white-box evaluation on TinylmageNet to PGD-20.
Black-box attacks. Under black-box settings, the adversary has no access the model parameters. We evaluated robust models trained CIFAR-10 against strong query-based black-box attacks Square (An- driushchenko et al. 2020) with 5,000 queries and SPSA (Uesato et al.) 2018) with 100 iterations, perturbation size 0.001 (gradient estimation). learning rate = 0.01, and 256 samples for each gradient estimation. All black-box evaluations are made on trained WRN-34-10.
Ensemble Attacks. Trained models are tested on powerful ensembles attacks such as Autoattack CCroe & Hein 2020b), which consisting of APGD-CE (Croce & Hein| 2020b), APGD-T (Croce & Hein 2020b), FAB-T Croce & Hein 2020a), and Square (a black-box attack) (Andriushchenko al. 20200) attacks. In addition, evaluated the trained models on the Margin Decomposition Ensemble (MDE) attack (Ma
et al.
2023
5.5 Performance Evaluation
We summarize our results on CIFAR-10 using RN-18 and WRN-34-10 in Tables and 2 respectively. Moreover, we report results on CIFAR-100, SVHN, and Tiny Imagenet using RN-18 in Tables 3 and 4 Finally, black-box evaluations are made on trained WRN-34-10, and the results are reported in Table 5 Experiments were repeated four times with different random seeds; the mean and standard deviation are subsequently calculated. Results are reported mean + sld.
8

---

Published in Transactions on Machine Learning Research (07 (07/2023)
Table 1: Comparing white-box attack robustness (accuracy %) for ResNet-18 on CIFAR-10. For all methods,
distance c = 0.031. We highlight the best-performing method under cach attack.
DEFENSE
NATURAL
FGSM
PGD-100
CW
FMN-100
AA
MDE
AT
84. 12+
57.88+
51.58
51.75+
48.92-oas
47.92+0.6
47.90
+0.27
TRADES
83.56+0.5
57.82+0.2
52.07=026
52.26=a07
49.74 40.74
48.32+00.
48.29
+-0.9
MART
80.32.10.8
58.01 19.19
5.10322008
19.29gam
19.82
47.61
47.55
2:012
GAIRAT
23.3
60.20 20.20
54.91 0.19
10.95gam
38.622000
32.89
32.70
20.18
MAIL-AT
84.32=.010
60.11 20.30
25.25
48.88aa0
46.83 2:0.7
44.22
44.14
20.31
VIR-TRADES
82.03
59.62
stm.e dtn.0s
54.86
53.11 to.17
51.95
51.03
50.89
+0.12
VIR-AT
84.59
61.35
56.42= 0.18
52.1 1g
50.56
48.21 2:000
48.04
20.00
Table 2: Comparing white-box attack robustness (accuracy %) for WideeeesNet-34-100 CIFAR-10. For all methods, distance e = 0.031. The best-performing methods under each attack are highlighted.
DEFENSE
NATURAL
FGSM
PGD-100
CW
FMN-100
AA
MDE
AT
86.17 20.20
61.682on
54.454aat
55.17 20:33
54.05
51.90.m.e
51.74
20.10
TRADES
85.20 +0.2
61.47 +0.ss
54.81 +o.st
56.02+029
53.95 +0.10
53.09
52.71
+0.12
MART
84.59
62. 40.1
56.45+a.1
54.52+ou
53. 53.174na12
51.21 +o.2n +0.21
50.92
+0.15
GAIRAT
85. #030
62,67 +0.30
+0.30
57.09= 0.27
44. 14.96+022
44.50 +
12,29#0..
41.92
+0.15
MAIL-AT
$4.83=0..0
64. 09 40.32
58.86
51.26
51 .1.644
#0.10
47.04
VR-TRADES
84. -0.22
63.07
+0.17
57. 57.56+a.2
56.92 +0.19
-5.72+
54.55+0.22
54.14
+0.0
VIR-AT
87.13+0.8.
64.71 00.29
59.82 0.229
56.11+ a.e
54. 54.14
51.94+on
51.83
+0.12
Table 3: Comparing white-box attack robustness (accuracy %) for RN-18 on SVHN and CIFAR-100. For all methods, distance 0.031.
SVHN
CIFAR-100
DEFENSE
NATURAL
PGD-100
AA
NATURAL
PGD-100
AA
AT
92.94 20.48
54.742m.
15.94gom
59. 59.60a eas
28.57 8.2
21.75 0:21
TRADES
92.11in.e
55.24
15.6420.
60.73 1033
29.83 0.25
24.83 0.29
MART
91.8440.6
55.54
43.39
54.19+ 0:26
29.94
25.30
GAIRAT
90.47 was
61.37 = 0.26
37.27 0.51
58.43 0.28
25.74 6.11
17.57
d: 0.53
MAIL-AT
91.54
62.164
41.18
60.7410.15
27.6210007
22.44
VIR-TRADES
89. 24 #0.25
58.63
40.82
50.06
59.20 #0.a5
31.69
0.24
26.25
#0.25
VIR-AT
91 41.65
61.52 #0.43
45.91 0.41
59.85 #0.11
32.06=0.a6
24.73
40.02
Table 4: Comparing white-box attack robustness (accuracy %) for ResNet-18 on TinyImageNet. Perturbation size 8/255 and step size x 1/255 are used for all methods.
DEFENSE
NATURAL
PGD-20
AA
AT
48.79 7920s
23.962a.b
18.06
TRADES
49.11
22.89
16.81 20.19
MART
45.91 2024
26.03 10.35
19.23 10.23
GAIRAT
46.09 +0.14
17.21+naa
12.92
+0.28
MT
49.72
24.32
10.3
17.61
VIR- VIR-TRADES
51.17 10.19
25.82
40.13
18.68
10:1
VIR-AT
49.09 20:55
26.65
18.42 20:
Comparison with prominent reweighting methods. The results in Tables 44 show that, general, the proposed VIR method is more effective than the two prominent reweighting methods MAIL-AT and
9

---

Published in Transactions on Machine Learning Research (07 (07/2023)
Table Comparing black-box attack robustness (accuracy %) for Wideressne-344-10 trained on CTFAR-10.
DEFENSE
SQUARE
SPSA
AT
60.12 12
61.05++0.6
TRADES
59. 18
61.15+0mm
MART
58. 72=01s
58. +0.13
GAIRAT
51.97=000
52.15 40.00
MAIL-AT AT
2013
49.244.0.2
VIR- TRATTRDESS
59. 03
61.45++0.5
VIR-AT
60.51 20.11
62.59_m..1
GAIRAT, especially under stronger attacks FMN, CW, and Autoattack. For example, VIR-AT' signifi- cantly outperforms MAIL-AT against FMN-100 (+2.5%), CW (+ 5.0%), and Autoattack (+ 5.0%) atlacks CIFAR-10 using WRN-34-10. The proposed VIR-AT method achieves large improvement margins over GAIRAT against FMN-100 (+ 9.6%), CW (+ 11.1%), and Autoattack (+ 9.6%) on CIFAR-10 using WRN- 34-10. VIR-AT also consistently performs better than MAIL-AT and GAIRAT on CIFAR-100 and Tiny- ImageNet against PGD and Autoattack attacks. On SVHN, MAIL-AT slightly outperforms our VIR-AT method against PGD-100, however, VIR-AI performs significantly better against Autoattack by over 4%.
Experimental results presented in Table 5 show that VIR-AT performs better than GAIRAT and MAIL- AT against strong query-based black-box attacks Square and SPSA. Furthermore, the poorer performance recorded by GAIRAT black-box attacks compared to the PGD-100 white-boz attack may potentially indicate that GAIRAT encourages gradient obfuscation according to the arguments made in (Athalye et al. 2018
Comparison with non-reweighted variants. We applied the proposed reweighting method to a promi- nent variant of vanilla AT, TRADES (Zhang et al. 2019). Experimental results in Tables 1 show that VIR-TRA consistently improves upon TRADES against. all attacks, especially stronger attacks CW, FMN, and Autoattack.
In addition, the confusion matrices displayed in Figure clearly shows the improvement in class wise accura- cies of data-sets corresponding to harder classes. We show in Figure 3 the class-weight distribution (the sum of all sample weights in cach class). From Figure 3 we observe that our proposed instance- wisc rewcighting assigns the largest weight to the most vulnerable class '3' and the least weight to the least vulnerable class '1' for both VIR-AT and VIR-TRA. TIRESS
VIR-AT
VIR-TRADES
599
17
53
26
13
10
34
630
12
73
37
31
16
15
24
97
29
27
756
15
29
6
12
10
41
105
23
762
10
19
8
5
9
4
53
117
56
8
372
90
53
45
16
46
357
125
63
32
10
166
Thue Labols
53
14
182
159
472
104
180
14
ThM Labels
25
172
107
371
71
142
99
7
19
6
56
122
44
458
36
15
10
10
10
67
205
35
439
34
58
13
15
20
12
109
123
14D
83
569
14
18
14
13
633
22
131
13
161
36
19
22
28
15
11
20
700
67
36
116
18
42
15
12
10
22
47
675
48
129
25
60
19
19
49
694
Predicted
Predicted Labels
Figure Confusion matrix displaying robust accuracies under PGD-100 attack using VIR-AT and VIR- TRADES for ResNet18 on CIFAR-10 dataset.
Finally, these experimental results show that the existing reweighting methods GAIRAT and MAIL improved upon the vanilla AT against FGSM and PGD attacks but performed much worse against stronger attacks CW, FMN, and Autoattack. In contrast, proposed VIR-AT performed significantly better than AT
10

---

Published in Transactions on Machine Learning Research (07 (07/202) 2023)
Class Weight Distribution for VIR-AT
Class Weight Distribution for VIR-TRADES
2500
dass-weights
class-weights
3000
2500
60000
2000
40000
20000
500
Figure 3: Clash- distribution (sum of all sample weights in each class) for VIR-AT and VIR-TRADES for ResNet18 during training on CIFAR-10 dataset.
PGD-100 and FGSM without sacrificing performance against CW, FMN, and Autoattack. A likely explanation for the relatively poor performance of existing reweighting functions stronger attacks such as Autoattack and CW is that they significantly diminish the influence of less vulnerable examples. While upweighting losses of vulnerable examples can be helpful, over-relying on the vulnerable samples may be detrimental as observed in (Dong et al. 2021). Our proposed VIR function achieved a good balance between weighting vulnerable and less vulnerable examples as shown the ablation studies presented the next. section.
5.6
Ablation Studies and Impact of Hyperparameters
5.6.1
Ablation Studies
We conduct ablation studies on the proposed weight assignment. function using ResNet-18 on CIFAR-10. The training settings are the same as those described in Section 5.1 We study the influence each reweighting component on the robustness against white-box and black-boo attacks. The results are presented Table
Table 6: Ablation studies on VIR-AT showing the impact of reweighting components proposed in Eq. 5 and on white-box and black-box attack robustness (accuracy %).
REWEIGHTING COMPONENT
NATURAL
PGD-100
CW
AA
SQUARE
SPSA
Lce(fo(z)),
84. 12.122
51.5826.7
51. 51.75amm
47.9249M
55.321mm
56.8510m
S. y1) LCE(fo(x(). v)
84.35+a.17
34.50+am
49.61 +0.10
45.48
54.68 +0.15
56.08+ear
Sa(r,, v (c
83. 33.90+a1s
51.17+6.1
51.90+n.ss
47.95+4.2
56.22 +0.19
56.25+0.1
w(r, (.) LoceSf(a
84.59+0.18
56.42+0.18
52.18-o.11 0.18
48.21 +0.0
56.89+0.10
57.35+0.11
Reweighting LCE(fo(2)), y:) with only Se(z;, vs) improves robustness against PGD-100, but yeilds reduced robustness against CW and Autoattack. When (Ce(fo(x)) is reweighted using Sa(#;, no significant improvement. in robustness was observed against PGD-100, however, stable performance over stronger attacks are observed. Reweighting LCe(fe(z)), 34) with the proposed reweighting function significantly improves robustness accuracy against both whito-bos box and black-box attacks.
The ablation studies VIR-TRADES CIFAR-10 using ResNet-18 presented in Table
11

---

Published in Transactions on Machine Learning Research (07 (023))
Table 7: Ablation studies on VIR-TRADES showing the impact of reweighting components proposed in Eq. and on white-box and black-box attack robustness (accuracy %).
REWEIGHTING COMPONENT
NATURAL
PGD-100
CW
AA
SQUARE
SPSA
Lcx(fo(x.),3)
(T.(f((z))]fooz.)
83.562033
52.07 20:35
52.261m.r
48.322a19
55.47
56.362om
77 20
54.21
51.7016.1
49.952a15
55.18
36.1110a
(fo))(( KTLfoo))))
36.29
19.71
49.65.6.1
46.772a13
54..7
55.85
Los (f)
02.03+
54.86=0.17
53.11=0.1r
51.03+0.10
56.58+0.19
57.80+0.00
The results in Table 7 show that reweighting TRADES with S. 31) yields better performance than the original TRADES against PGD-100 (+ 2.14 %) and Autoattack (+ 1.63 %). However, yields lower per- formance against CW (-0.56 %) and natural examples (-5.61%). Reweighting TRADES using Sa(r,, af) improves the natural accuracy by +3.03% but yields lower performance against attacks. The proposed reweighting function 2(,,,, consistently improves VIR-TRADES over TRADES against PGD-100 (+ 2.97%), CW (+ 0.85%), Autoattack (+ 2.71%), Square attack (+ 1.1%), and SPSA (+ 1.44%). The re- sults show neither performance when combined.
nor x() is sufficient for improving TRADES and they achieve the best
5.6.2 Impact of Hyperparameters
We providc a brief discussion of the impact of the hyperparameters a, and 3 used in the proposed reweigth- ing function. Hyperparameter appears in the power of the exponential function in Eq. 5 Setting y high results a large disparity in weights between vulnerable examples and less vulnerable examples. a helps with the numerical stability of Eq. 5 Introducing 3 into the reweighting function in Eq. 7 allows for a balance between relative weights of vulnerable examples and less vulnerable examples. Without adding 3, the reweighting function could potentially return very low values, which significantly diminishes the influence of less vulnerable training samples. On the other hand, if 8 is set too high, the desired reweighting effect is lost. The values of these hyperparameters are heuristically determined. Experimental studies on the impact of the hyperparameters on the performance are shown in Appendix C
6 Conclusion
In this paper, we propose a novel vulnerability-wwar instance-win reweighting strategy for adversarial training. The proposed reweighting strategy takes into consideration the intrinsic vulnerability of natural examples used for crafting adversarial examples during adversarial training. We show that existing reweight- ing methods fail to achieve significant robustness against stronger white-box and black-box attacks. Lastly, experimentally show that the proposed reweighting function effectively improves adversarial training without diminishing its performance against stronger attacks. In addition, the proposed method shows improvement on every dataset evaluated.
References
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Push- meet Kohli. Are labels required for improving adversarial robustness? Advances in Neural Information Processing Systems, 32, 2019.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision, pp. 484 501. Springer, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of sccurity: Circunventing defenses adversarial examples. In International conference on machine learning, pp. 274-283. PMLRR 2018.
12

---

Published in Transactions on Machine Learning Research (07/2023)
Modeste Atsague, Ohukorede Fakorede, and Jin Tian. A mutual information regularization for adversarial training. In Asian Conference on Machine Learning, pp. 188-203. PMI.R, 2021.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smniié, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 jeee symposium on security and privacy (sp), pp. 39-57. Icce, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. Advances in Neural Information ion Processing Systems, 32, 2019.
Tianlong Chen, Zhenyn Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. In International Conference on Learning Representations, 2020.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference Machine Learning, pp. 2196-2205. PMLR, 2020a.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206 2216. PMLR, 2020b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In International Conference on Learning Representations, 2019.
Chengyu Dong, Liyuan Liu, and Jingbo Shang. Data quality matters for adversarial training: An empirical study, arXiv preprint wrXiv:2122.07337 2021.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial atlacks with momentum. In Proceedings of the TEEE conference on computer vision and pattern recognition, pp. 9185 9193, 2018.
Olukorede Fakorede, Ashutosh Nirala, Modeste Atsague, and Jin Tian. Improving adversarial robustness with hypersphere embodding and angular-basod regularizations. ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5. IEEE, 2023.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.657 2015.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using imput transformations. In International Conference on Learning Representations, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Conference on Machine Learning, pp. 2137-2146. PMLR, 2018.
Andrew Tlyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems. 32, 2019.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. 0rXiv:1803.00331 2018.
Adversarial logit pairing.
arviv preprint
13

---

Published in Transactions on Machine Learning Research (07/2023)
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Feng Liu, Bo Han, Tongliang Liu, Chen Gong, Gang Niu, Mingyuan Zhou, Masashi Sugiyama, et al. Proba- bilistic margins for instance reweighting in adversarial training, Advances in Neural Information Processing Systoms, 34:23258 23269, 2021.
Xingiun Ma, Linxi Jiang, Hanxun Huang, Zejia Weng, James Bailey, and Yu-Gang Jiang. Imbalanced gradients: a subtle cause of overestimated adversarial robustness. Machine Learning, pp. 1-26, 2023.
Xinsong Ma, Zekai Wang, and Weiwei Liu. On the tradeoff between robustness and fairness. In Advances in Neural Information Processing Systems, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2574 2582, 2016.
Yurii Evgen' eeiih Nesterov. A method of solving convex programming problem with convergence rate o o'bigl(k b(grr)). In Doklady Akademii Nauk, volume 269, pp. 543-547. Russian Academy of Sciences, 1983.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. Advances in Neural Information Processing Systems, 33:7779-7792, 2020.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP), pp. 582 597. IEEE, 2016.
Nicolas Papernot, Patrick McDaniel, lan Goodfellow, Somesh Jha, Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506-519, 2017.
Maura Pintor, Fabio Roli, Wieland Brendel, and Battista Biggio. Fast minimum-norm adversarial atlacks through adaptive norm constraints. Advances in Neural Information Processing Systems, 21220002-20066 2021.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. Advances in neural information processing systems, 31, 2018.
Leslie N Smith and Nicholay Topin. Super-convergence:: Very fast training neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications, volume 11006, pp. 369 386. SPIE, 2019.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Lever- aging generative models to undersland and defend against adversarial examples. ar preprint arXiv:1710.10766, 2017.
Christian Szegedy, Wojeiech Zaremba, Tlya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellww, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint ar arXiv:1805.12155 2018.
14

---

Published in Transactions on Machine Learning Research (07/2023)
Jonathan Uesato, Brendan O'donoghue, Pushmeet Kohli, and Aaron Oord. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference Machine Learning, pp. 5025-5034. PMLR, 2018.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adver- sarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations, 2019.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusson models further improve adversarial training. arXiv preprint WrXiv:2302.04633 2023.
Zi Wang. Zero-shot knowledge distillation from decision- based black-box model. In International Conference on Machine Learning, pp. 10675-10683) 10685. PMLR, 2021.
Dongxian Wu, Shu- Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 33, 2020.
Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness adversarial training. In International Conference on Machine Learning, pp. 11492 11501. PMLR, 2021.
Runtian Zhai, Tianle Cai, He, Chen Dan, Kun He, John Hopcroft, and Liwei Wang. Adversarially robust generalization just requires more unlabeled data. arXiv preprint arXiv:1906.00555, 2019.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theo- retically principled trade-off between robustness and accuracy, In International Conference on Machine Learning, pp. 7472-7482. PMLR, 2019.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanballi. Geometry- aware instanco-reweighted adversarial training. In International Conference on Learning Reprosentations, 2020.
A Appendix: Theoretical Explanation of Input Vulnerability
In this section, we show that, under a special setting, the example vulnerability directly corresponds to the probability of predicting the true class We consider binary classification task of natural examples sampled from Gaussian mixture distribution, following the settings described in (Carmon et al. 2019:
Schmidt et al., 2018 Xu et al. 2021 Ma et al. 2022). We formally define the settings below.
Definition A.1 (Gaussian Mixture Distribution.) Let -u, be the mean parameters corresponding to data sampled from two classes y = +11 according to Gaussian distribution.
Let 0-,0+ represent the variances of the classes -1 and +1 Then the Gaussian mixture model is defined by the following distribution over (x.y) E Rd x {+1}:
d=dim
y y~ +-1,++11 / (T..... [N(,, ify= W((-H,0II), =
(11)
I is the d-dimension identity matriz.
To design classes with different intrinsic vulnerabilities, we ensure that there is a K-factor difference between the variances of the classes such that a_ 0 K and K > 1. The larger variance of the examples corresponding to class intuitively suggests higher vulnerability than an example corresponding to class
15

---

Published in Transactions on Machine Learning Research (07 2023)
The analysis is done using linear classifier as follows:
f(x) sign((w, x) + b)
(12)
where w and respectively represent the model weights and bias term, and sign(z) returns 1 if = 2 0, else 1. The natural risk is denoted
Ro = E(x.y)D (1(f(x) + y)
PP([nat(2 + y)
(13)
Pr(y =-1. Pr(Snat(x) +1 -1) Pr(y= =11 Pr(fnat(x) =-1l +11
Pr(y=-1) Rea + Pr(y= = +1) R
(nat( and Ryjat (fnat) represent the class-wise risk of misclassifying -1 and +1 respectively, fnat is a naturally trained classifier.
Theorem ((Xu et al., 2021)) Given a Gaussian distribution D' naturally trained classifier fnat which minimizes the expected natural risk: Snot (x) = arg miny (x (I( # y). It has the class- wise natural risk:
R Ria Pr(M A K v A (K) - - A where A = K2-1 Van and {(K 2logK which is a positive constant and depends only on K. Therefore, class +1 has larger risk: (nat()nat) R+ (fnat).
R (fnat) = Pr(N
v A2 + (K))
K2-1
Theorem shows that the vulnerable class +1 (with larger variance) is more difficult to classify than -1, because the an optimal frat has higher standard error for +1 than -1.
Corollary 1 (Vulnerability of a data sample.) The vulnerabilily of an ezample may be estimated using a classifier's 's estimated dass-probability
Proof 1 The class-wise risks corresponding to classes -1 and +1 can be respectively written as:
R (nat) = Pr(Jmae(x)) = - 11y = +1) = Pr((w, x) + +b>00 0) R (Frat) = Pr(fnae(x) = + +1/y = -1) = Pr (ww (w, + <0
Let Pmot (natt and Piteefno (Fnas) respectively denote the correct probability estimates of classes -1 and +1.
Then,
(nat) = Pr(fnac(x) = 1/y = -1)
p PitaeSnnot) = Pr(Snac(x = +11y = +1).
From Theorem 1, Ryje (Frat) < ( It follows that P (Snat) > P (Fnat):
B Additional Experiments on Data Augmentation Based Defense
We perform additional experiments on the data sugmentation-basse defense using extra datasets generated using the Elucidating Diffusion Model (EDM) following Wang et al. (2023).
We utilize Wideresnet-28-100 as the backbone model for the experiments. The model was trained using 20M EDM-generated data for 400 epochs with the training batch size of 512. Common augmentation described in (Wang et al.] 2023 is applied on the training samples. We employ the SGD optimizer with Nesterov momentum [Nsterovv 1983) where the momentum factor and weight decay were set to 0.9 and 5e respectively. We use the cyclic learning rate schedule with cosine annealing (Smith & Topin 2019). where the initial learning rate set to 0.2. For VIR-TRADES, we used the settings described in section 5.3, while for VIR-AT, we set y to 5.0, 3 to 0.2 and a to 7.0.
The obtained results are reported in Table The experimental results show improvement of VIR-TRADES over TRADES and VIR-AT over AT CIFAR-10
16

---

Published in Transactions on Machine Learning Research (07 (07/2023)
Table 8: Comparing robustness (accuracy %) for Wideresnet-28-10 on CIFAR-10 trained on 20M FDM-
gencrated data.
DEFENSE
NATURAL
PGD- 100
CW
AA
TRADES
90.33
65
64.01 20.1
63.03
AT
91.36+00m
65.62 65.62+am0
62.42=a.11
VIR-TRI HR-TRADES
67. 67.23
65.47 toos
64.3120.r
VIR-AT
92.69 20.25
67.98
66.951a11
62.95
C Ablation Studies on Hyperparameters
We show in the following experimental results varying hyperparameter a. values. The results are obtained from training ResNet-18 on CIFAR-10 dataset.
Table 9: Ablation studies on VIR-AT showing the impact of the 3 hyperparameter on the performance of the proposed reweighting function.
REWEIGHTING FUNCTION
NATURAL PGD-100
CW
AA
S. (a Vi) Sa( (3
84.481on
57.20
51 25
47.32gm1
S. (x,, v:) Sa(r,, r + (3 0.007)
84.59+c.s
56.42=0as
52.18+a.1
48.21+0.00
S. (. v.) Sa(r,, x.) (3 0.1)
84.26+
33.52=001
52.20=0.10
48.17#a18
Table 10: Ablation studies on VIR-TRADES showing the impact of the mance of the proposed reweighting function.
3 hyperparameter on the perfor-
REWEIGHTTING FUNCTION
NATURAL
PGD-100
CW
AA
(2,, v.) Sa( + (3 0.5)
83.322015
53. 10
51.601a.
49. 49.294.0
(. v1) Sa(x), + (8 =0=1 =1.6)
8215
53.11-o.x 54.86+
51.03+o. +o.16
(, v.) Sa(x;, (3 2.0)
80.86+a.0
54.58+0.s
52.62+a..
50.56+a1s
17

---

Published in Transactions on Machine Learning Research (07 (07/202) 2023)
Table 11: Ablation studies VIR-AT showing the impact of the and y hyperparameters on the perfor- mance of the proposed reweighting function.
Y
NATURAL
PGT>-100
CW
AA
10
0.007
83.7510.1
5.1.88200s
52.20zais
18.2516.s
10
0.007
83.80
55.282c0
52. 13
18.18
10
0.007
84.1640.0
55.85
52.05+a11
48.154m.1
10
0.007
84.31
55.922.10
51.802a0
47.85
10
0.007
84.59+n.a
56.42
52.20+a.1
48.17+0.15
10
0.007
84.19+mm
56.68+6.2
52.01 +aoD
48.02+0.15
2
0.007
83.961015
54.752215
51.901a0s
47.8910.0
7
4
0.007
84.52100
54.9212.0
52.01 2010
47.97
6
0.007
84.36
55.822em
51.76200s
47.7010.0
7
0.007 8
84.15
56.30
51.91+a1
47.87
Table 12: Ablation studies on VIR-TRADES showing the impact of the a and y hyperparameters on the performance of the proposed reweighting function.
7
8
NATURAL
PGD-100
CW
AA
3.0
1.6
82.10
54.07
52.29
50.08
3.0
1.6
82.13a1s
54.10
50. 13
3
3.0
1.6
82.12ao.
54.162010
52.51
50.41
3.0
1.6
82,06+0.1
54.31+oas
52.54+o1s
50.43 +..os
3.0
1.6
$2.05+a.1
54.52+o.0
52.59+om
50.52+a.n
8
3.0
1.6
82.03+o.1
54.86+ou7
51.03++.1e
1.0
1.6
81.05
53.962a15
01.85
49.89
2.0
1.6
81.09
54.262201
52.41
22.41200s
50.28
4.0
1.6
81.91
40.00
54.42
o2
50.3440.11
18


Reviews:
[{"review_id": "VdGch2Pxwe", "reviewer": "Reviewer_m7G5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strength\n\nThis paper is overall well written and well motivated. The paper provides clear discussion on why the reweighting is needed, how the proposed reweighting criteria is proposed, and how is it different from previous methods. The formulation of the porposed method makes sense, and emperical results are promixing on multiple models and datasets. \n\n## Weakness\n\nThe proposed method invloves multiple hyperparameters in the formulation of the reweighting criteria, yet their usage and imact is not well discussed. More analysis and ablation study should be provided on how these hyperparameters impact the performance, and how should they be chosen in practice.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates on a better instance reweighting method to help metigate the discrepancy of robust accuracy between easy and hard samples. Based on the observations and shortcomings of previous work, this work proposes a vulnerability measurement unitlizing true class probablity and disparity between natural and adversarial output. Experiments are provided to show the effectiveness of the proposed method on improving robust accuracy, and ablation study is conducted on the proposed vulnerablity metric.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please discuss more on the impact of hyperparameters (e.g. $\\alpha, \\gamma$ in Equ. (5) and $\\beta$ in Equ. (10)) on the performance of the proposed method", "broader_impact_concerns": "No concern on broader impact", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6KfKXBnFwR", "reviewer": "Reviewer_bYBD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strength\n\n1. Experiments are done extensively on different models and different datasets against various attacks. Also, the results support the improvement well, demonstrating the success of the authors reweighting method.\n    1. The proposed method shows good performance against stronger attacks, e.g., CW and AutoAttack, whereas the existing methods failed to maintain their performances.\n2. Ablation studies demonstrate the effect of each weighting factor well. The observations on each factor would be useful for other researchers who study the reweighting in AT.\n\n# Weaknesses\n1. I can observe that $S_v$ (weighting factor from the vulnerability) is bounded, but $S_d$ (weighting factor from the KL divergence) is not bounded. This sounds like that, for some adversarial examples, the weighting could be mainly dominated by $S_d$.\n    1. Can the authors provide the distribution of $S_v$ values and $S_d$ values during the training? Experimental evidence that $S_d$ does not explode dramatically would be enough to support the authors' strategy.\n    2. If it is not possible to provide such evidence, isn’t it better to bound $S_d$ somehow?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new strategy for weighting instances during adversarial training. In the traditional adversarial training method (without weighting), depending on the original class label, adversarial examples have different effects on the resulting robust accuracy. Some prior works tried to reduce this unfairness among adversarial examples by re-weighting the training loss and the authors propose another weighting strategy.\n\nThe authors’ weighting is based on the vulnerability of the natural sample $\\mathbf{x}$ and the discrepancy between the model predictions $f_\\theta(\\mathbf{x}_i)$ and $f_\\theta(\\mathbf{x}’_i)$. First, the vulnerability factor $S_v$ is measured by the model’s confidence $f_\\theta(\\mathbf{x})_y$ in the event that natural sample $\\mathbf{x}$ belongs to class $y$. Then, the discrepancy factor $S_d$ is measured by the KL divergence of the model prediction probabilities $f_\\theta(\\mathbf{x})$ on a natural example $\\mathbf{x}$ from the model prediction probabilities $f_\\theta(\\mathbf{x}’)$  on the adversarial example $\\mathbf{x}’$ generated from $\\mathbf{x}$. Then the weighting $w$ is determined as a linear function $w=\\alpha S_v \\cdot S_d + \\beta$ for predetermined parameter $\\alpha$ and $\\beta$. Finally, the authors propose the use of this weighting with some burn-in period so that the model can learn enough information about the data.\n\nAn ample amount of experiments are performed to present the improvement from the proposed weighting method. For most experiments, the adversarial training with the proposed method demonstrated descent robust accuracy compared to the existing methods. Especially, the proposed method successfully improved the robust accuracy against some attacks, e.g., CW and AutoAttack, whereas the existing method failed to maintain the robust accuracies from the adversarial training. The authors also present some ablation studies to show the effects of each weighting factor.\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I can see some writing issues, e.g., grammar, notations, etc. I suggest the authors read through the writing.\n    1. I can see many citations in parentheses in the middle of a sentence, and this sometimes makes the writing hard to read. Please distinguish the use of `\\citet{}` (citation as a part of a sentence) and `\\citep{}`(citation not a part of a sentence).\n    2. The notations are not used consistently. For example, the class label $y$ is sometimes written in boldface in cases where it is definitely not a vector. Even boldface letters appear in different typefaces. Please go through the doc and make the notations consistent.\n2. In my opinion, the authors should emphasize the failures of existing weighting strategies against CW and AutoAttack more.\n    1. In particular, ablation studies show that the introduction of $S_d$ was crucial for the robustness against those attacks. Can the authors relate this to the reason why the prior methods fail against the stronger attacks?\n3. While determining the hyperparameters heuristically makes sense to me, it would be better to have some experimental study on these hyperparameters.\n    1. For example, my understanding of $\\beta$ is that it lower bounds the weighting so that the training loss (in Eq. (11)) does not disappear for very low $S_v$ and $S_d$. Based on this understanding, setting $\\beta=0.007$ seems quite small to me and I want to see the effect of setting this parameter higher.\n4. One minor request: I don’t think that Section 4.2 has more valuable information than the ablation study in Appendix. (Of course, it is a meaningful theoretical insight, but this paper is not a theory-intensive work, anyway.) Consider moving Section 4.2 to Appendix and moving the ablation study in Appendix to the main body of the paper.\n", "broader_impact_concerns": "I don’t see a particular broader impact concern regarding this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8mrvc5nzTz", "reviewer": "Reviewer_YFuN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The study of a challenging problem in adversarial machine learning, i.e.,  how to train adversarially robust models.\n2. The proposed idea is simple enough to be applied to any existing AT method.\n3. The improvement to TRADES according to AutoAttack seems to be significant.\n\nWeaknesses:\n1. Not sure if instance reweighting is better than class reweighting or class-class reweighting. Certain classes are known to be easily attacked into other classes. How the proposed approach can address this type of imbalance?\n2. Should the instance reweighting consider class imbalance problem, as some classes are notably harder than others? How the class-wsie accumulative of the instance reweighting aligns with the class distribution?\n3. It is not clear if the proposed method is compatible with recent data augmentation methods, e.g., the one [1] on the robustbench leaderboard.\n4. Since the reweighting will skew the logits distribution, the author should also test against the MDE attack [2] to prove it does not cause imbalanced gradients.\n\n[1] Better Diffusion Models Further Improve Adversarial Training.\n[2] https://github.com/HanxunH/MDAttack", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the imbalance problem in adversarial training and proposes to reweight the importance of each training sample based on the prediction disparity between the nature and adversarial version of the model. The assumption is that a large disparity indicates more vulnerability, which is proved from the perspective of risk-to-class-probability with Gaussian distribution. The proposed reweighting scheme demonstrates improvement in standard adversarial training and TRADES. \n\nOverall, the key contribution is the use of prediction change to reweight a training sample during adversarial training.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Show the class weight distribution (sum over all sample weights in the class)  caused by the reweighting during training.\n2. Test against the MDE attack.\n3. Show its compatibility with recent SOTA defense, i.e., the data augmentation based defense.", "broader_impact_concerns": "No obvious concerns have been identified.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have successfully addressed some major comments raised by the Reviewers during phase-1 review.  As a result, all reviewers are satisfied with the revision and agree that the paper is interesting for audience and provide sufficient evidence to support their claims. 
Decision: Accept (poster)