I MPROVING ADVERSARIAL ROBUSTNESS BY PUTTING MORE REGULARIZATIONS ON LESS ROBUST SAMPLES
Anonymous authors
Paper under double-blind review
A BSTRACT
Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention because it is easy to generate human-imperceptible perturbations of data to deceive a given deep neural network.
In this paper, we propose a new adversarial training algorithm that is theoretically well motivated and empirically superior to other existing algorithms.
A novel feature of the proposed algorithm is to apply more regularization to data vulnerable to adversarial attacks than other existing regularization algorithms do.
Theoretically, we show that our algorithm can be understood as an algorithm of minimizing a newly derived upper bound of the robust risk.
Numerical experiments illustrate that our proposed algorithm improves the generalization (accuracy on examples) and robustness (accuracy on adversarial attacks) simultaneously to achieve the state-of-the-art performance
I NTRODUCTION
It is easy to generate human-imperceptible perturbations that put prediction of a deep neural network (DNN) out.
Such perturbed samples are called adversarial examples (Szegedy et al., 2014) and algorithms for generating adversarial examples are called adversarial attacks .
It is well known that adversarial attacks can greatly reduce the accuracy of DNNs, for example from about 96% accuracy on clean data to almost zero accuracy on adversarial examples (Madry et al., 2018).
This vulnerability of DNNs can cause serious security problems when DNNs are applied to security critical applications (Kurakin et al., 2017; Jiang et al., 2019) such as medicine (Ma et al., 2020;
Finlayson et al., 2019) and autonomous driving (Kurakin et al., 2017; Deng et al., 2020; Morgulis et al., 2019; Li et al., 2020).
Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention.
Various adversarial training algorithms can be categorized into two types.
The first one is to learn prediction models by minimizing the robust risk - the risk for adversarial examples.
PGD-AT (Madry et al., 2018) is the first of its kinds and various modifications including Zhang et al.
Ding et al.
; Zhang et al.
(2021) have been proposed since then.
The second type of adversarial training algorithms is to minimize the regularized risk which is the sum of the empirical risk for clean examples and a regularized term related to adversarial robustness.
TRADES (Zhang et al., 2019) decomposes the robust risk into the sum of the natural and boundary risks, where the first one is the risk for clean examples and the second one is the remaining part, and replaces them to their upper bounds to have the regularized risk.
HAT (Rade & Moosavi-Dezfolli, 2022) modifies the regularization term of TRADES by adding an additional regularization term based on helper samples.
The aim of this paper is to develop a new adversarial training algorithm for DNNs, which is theoretically well motivated and empirically superior to other existing competitors.
Our algorithm modifies the regularization term of TRADES (Zhang et al., 2019) to put more regularization on less robust samples.
This new regularization term is motivated by an upper bound of the boundary risk.
Our proposed regularized term is similar to that used in MART (Wang et al., 2020).
The two key differences are that (1) the objective function of MART consists of the sum of the robust risk and regularization term while ours consists of the sum of the natural risk and regularization term and (2) our algorithm regularizes less robust samples more but MART regularizes less accurate samples more.
Note that our algorithm is theoretically well motivated from an upper bound of the robust risk but no such theoretical explanation of MART is available.
In numerical studies, we demonstrate that our algorithm outperforms MART as well as TRADES with large margins.
1. O UR C ONTRIBUTIONS
We propose a new adversarial training algorithm.
Novel features of our algorithm compared to other existing adversarial training algorithms are that it is theoretically well motivated and empirically superior.
Our contributions can be summarized as follows:
• We derive an upper bound of the robust risk for multi-classification problems.
• As a surrogate version of this upper bound, we propose a new regularized risk.
• We develop an adversarial training algorithm that learns a robust prediction model by minimizing the proposed regularized risk.
• By analyzing benchmark data sets, we show that our proposed algorithm is superior to other competitors in view of the generalization (accuracy on clean examples) and robustness (accuracy on adversarial examples) simultaneously to achieve the state-of-the-art performance.
• We illustrate that our algorithm is helpful to improve the fairness of the prediction model in the sense that the error rates of each class become more similar compared to TRADES.
P RELIMINARIES 2. R OBUST P OPULATION R ISK
Let X ⊂ R d be the input space, Y = { 1 , · · · , C } be the set of output labels and f θ : X → R C be the score function parameterized by the neural network parameters θ (the vector of weights and biases) such that p θ ( ·| x ) = softmax( f θ ( x )) is the vector of the conditional class probabilities.
Let F θ ( x ) =
arg max c [ f θ ( x )] c , B p ( x , ε ) = { x ′ ∈ X : ∥ x − x ′ ∥
p ≤ ε } and 1 ( · ) be the indicator function.
Let capital letters X , Y denote random variables or vectors and small letters x , y denote their realizations.
The robust population risk used in the adversarial training is defined as
[Equation]
where X and Y are a random vector in X and a random variable in Y , respectively.
Most adversarial training algorithms learn θ by minimizing an empirical version of the above robust population risk.
In turn, most empirical versions of (1) require to generate an adversarial example which is a surrogate version of
[Equation]
Any method of generating an adversarial example is called an adversarial attack.
2. A LGORITHMS FOR G ENERATING A DVERSARIAL E XAMPLES
Existing adversarial attacks can be categorized into either the white-box attack (Goodfellow et al., 2015;
Madry et al., 2018; Carlini & Wagner, 2017; Croce & Hein, 2020a) or the black-box attack (Papernot et al., 2016; 2017; Chen et al., 2017; Ilyas et al., 2018; Papernot et al., 2018).
For the whitebox attack, the model structure and parameters are known to adversaries who use this information for generating adversarial examples, while outputs for given inputs are only available to adversaries for the black-box attack.
The most popular method for the white-box attack is PGD (Projected Gradient Descent) (Madry et al., 2018).
Let η ( x ′ | θ, x , y ) be a surrogate loss of 1 { F θ ( x ′ )
̸
= y } for given θ , x , y .
PGD finds the adversarial example by applying the gradient ascent algorithm to η to update x ′ η and projecting it to B p ( x , ε ) .
That is, the update rule of PGD is
[Equation]
where ν > 0 is the step size, Π B p ( x ,ε ) ( · ) is the projection operator to B p ( x , ε ) and x (0)
=
x .
We define x pgd as x pgd :=
lim t →∞ x ( t ) .
For the surrogate loss η , the cross entropy (Madry et al., 2018) or the KL divergence (Zhang et al., 2019) is used.
For the black-box attack, an adversary generates a dataset { x i , ˜ y
i } ni
=1 where ˜
y i is an output of a given input x i .
Then, the adversary trains a substitute prediction model based on this data set, and generates adversarial examples from the substitute prediction model by PGD (Papernot et al., 2017).
2. R EVIEW OF A DVERSARIAL T RAINING A LGORITHMS
We review some of the adversarial training algorithms which, we think, are related to our proposed algorithm.
Typically, adversarial training algorithms consist of the maximization and minimization steps.
In the maximization step, we generate adversarial examples for given θ , and in the minimization step, we fix the adversarial examples and update θ .
In the followings, we denote (cid:98) x
pgd i as the adversarial example corresponding to ( x i , y i ) generated by PGD.
2.3. A LGORITHMS MINIMIZING THE ROBUST RISK DIRECTLY
PGD-AT Madry et al.
(2018) proposes PGD-AT which updates θ by minimizing
[Equation]
where ℓ ce is the cross-entropy loss.
GAIR-AT Geometry Aware Instance Reweighted Adversarial Training (GAIR-AT)
(Zhang et al., 2021) is a modification of PGD-AT, where the weighted robust risk is minimized and more weights are given to samples closer to the decision boundary.
To be more specific, the weighted empirical risk of GAIR-AT is given as
[Equation]
where κ θ ( x i , y i ) =
min (cid:16) min( { t :
F θ ( x ( t ) i ) ̸ = y i } ) , T (cid:17) for a prespecified maximum iteration T and w θ ( x i , y i )
= (1 + tanh(5(1 − 2 κ θ ( x i , y i ) /T ))) / 2 .
There are other similar modifications of PGA-AT including Max-Margin Adversarial (MMA) Training (Ding et al., 2020) and Friendly Adversarial Training (FAT) (Zhang et al., 2020).
2.3. A LGORITHMS MINIMIZING A REGULARIZED EMPIRICAL RISK
Robust risk, natural risk and boundary risk are defined by
[Equation]
Zhang et al.
(2019) shows
R rob ( θ ) =
R nat (
[Equation]
+
[Equation]
bdy ( θ ) .
By treating R bdy ( θ ) as the regularization term, various regularized risks for adversarial training have been proposed.
TRADES Zhang et al.
(2019) proposes the following regularized empirical risk which is a surrogate version of the upper bound of the robust risk:
[Equation]
HAT Helper based training (Rade & Moosavi-Dezfolli, 2022) is a variation of TRADES where an additional regularization term based on helper examples is added to the regularized risk.
The role of helper examples is to restrain the decision boundary from having excessive margins.
HAT minimizes the following regularized empirical risk:
[Equation]
where θ pre is the parameter of a pre-trained model only with clean examples, x helper i
[Equation]
x i ) .
MART Misclassification Aware adveRsarial Training (MART) (Wang et al., 2020) minimizes
[Equation]
where ℓ margin ( f θ ( (cid:98) x pgd i ) , y i )
=
− log p θ ( y i |
[Equation]
x pgd
[Equation]
) − log(1 − max k ̸ = y i p θ ( k | (cid:98) x
pgd i )) .
This objective function can be regarded as the regularized robust risk and thus MART can be considered as a hybrid algorithm of PGD-AT and TRADES.
A NTI -R OBUST W EIGHTED R EGULARIZATION (AR O W)
In this section, we develop a new adversarial training algorithm called Anti-Robust Weighted
Regularization (ARoW), which is an algorithm minimizing a regularized risk.
We propose a new regularized term which applies more regularization to data vulnerable to adversarial attacks than other existing algorithms such as TRADES and HAT do.
Our new regularized term is motivated by the upper bound of the robust risk derived in the following section.
3. U PPER B OUND OF THE R OBUST R ISK
In this subsection, we provide an upper bound of the robust risk for multi-classification problem which is stated in the following theorem.
The proof is deferred to Appendix A.
Theorem 1.
For a given score function f θ , let z ( · ) be an any measurable mapping from X to X satisfying
[Equation]
for every x ∈ X .
Then, we have
[Equation]
The upper bound (4) consists of the two terms : the first term is the natural risk itself and the second term is an upper bound of the boundary risk.
This upper bound is motivated by the upper bound derived in TRADES (Zhang et al., 2019).
For binary classification problems, Zhang et al.
(2019) shows that
[Equation]
(5) where
[Equation]
and ϕ ( · ) is an upper bound of 1 ( · < 0) .
Our upper bound (4) is a modification of the upper bound (5) for multiclass problems where ϕ ( · ) and f θ in (5) are replaced by 1 ( · < 0) and F θ , respectively.
A key difference, however, between (4) and (5) is the term 1 { p θ ( Y | z
( X ))
< 1 / 2 } at the last part of (4) that is not in (5).
It is interesting to see that the upper bound in Theorem 1 becomes equal to the robust risk for binary classification problems.
That is, the upper bound (4) is an another formulation of the robust risk.
However, this rephrased formula of the robust risk is useful since it provides a new learning algorithm when the indicator functions are replaced by their surrogates as we do.
Input dataset pgd b k ← arg max x ′ ∈B p ( x t,b,k ,ε ) KL( p θ ( ·| x t,b,k ) ∥
p θ ( ·| x ′
; x t,b,k ∈ R d k = 1 , . . .
, K 4: θ ←
3. A LGORITHM
By replacing the indicator functions in Theorem 1 by their smooth proxies, we propose a new regularized risk and develop the corresponding adversarial learning algorithm called the Anti-Robust
Weighted Regularization (ARoW) algorithm.
The four indicator functions in (4) are replaced by
• the adversarial example z
( x ) is replaced by (cid:98) x pgd obtained by the PGD algorithm with the KL divergence; • the term 1 ( Y ̸
= F θ ( X )) is replaced by the label smooth cross-entropy (Müller et al., 2019)
ℓ LS ( f θ ( x ) , y )
=
− y LS α ⊤ log p θ ( ·| x ) for a given α > 0 , where y LS α
=
(1 − α )
u y + αC 1 C , u y ∈ R C is the one-hot vector whose the y -th entry is 1 and 1 C ∈ R C is the vector whose entries are all 1; • the term 1 ( F θ ( X ) ̸
= F θ ( z ( X ))) is replaced by λ · KL( p θ ( ·| X ) ||
p θ ( ·| (cid:98) X pgd )) for λ > 0; • the term 1 { p θ ( Y | z ( X )) <
1 / 2 } is replaced by its convex upper bound 2(1 − p
θ ( Y | (cid:98) X pgd ));
to have the following regularized risk for ARoW, which is a smooth surrogate of the upper bound (4),
[Equation]
Here, we introduce the regularization parameter λ > 0 to control the robustness of a trained prediction model to adversarial attacks.
That is, the regularized risk (6) can be considered as a smooth surrogate of the regularized robust risk of R nat ( θ ) +
λ R bdy ( θ ) .
We use the label smoothing cross-entropy as a surrogate for 1 ( Y ̸ = F θ ( X )) instead of the standard cross-entropy to estimate the conditional class probabilities p θ ( ·| x ) more accurately (Müller et al., 2019).
The accurate estimation of p θ ( ·| x ) is important since it is used in the regularization term of
ARoW.
It is well known that DNNs trained by minimizing the cross-entropy are poorly calibrated (Guo et al., 2017), and so we use the label smoothing cross-entropy technique.
We set α = 0 .
2 in our numerical studies for simplicity even if it can be tuned optimally.
The ARoW algorithm, which learns θ by minimizing R ARoW ( θ ; { ( x i , y i ) } ni =1 , λ ) , is summarized in
Algorithm 1.
Comparison to TRADES A key difference of the regularized risks of ARoW and TRADES is that
TRADES does not have the term (1 − p θ ( y i | (cid:98) x pgd i )) at the last part of (6).
That is, ARoW puts more regularization to samples which are vulnerable to adversarial attacks (i.e. p θ ( y i | (cid:98) x pgd i ) is small).
Note that this term is motivated by the tighter upper bound of the robust risk (4) and thus is expected to lead better results.
Numerical studies confirm that it really works.
Comparison to MART Although the objective function in MART (3) has no theoretical basis, it is similar with the objective function of ARoW.
But, there are two main differences.
First, the supervised loss term of ARoW is the label smoothing loss with clean examples, whereas MART uses the margin cross entropy loss with adversarial examples.
Second, regularization term in MART is proportional to (1 − p θ ( y | x )) while that in ARoW is proportional to (1 − p θ ( y | (cid:98) x pgd )) .
In the numerical studies, we observe that ARoW outperforms MART with large margins.
These would be partly because ARoW is theoretically well motivated.
E XPERIMENTS
In this section, we investigate the ARoW algorithm in view of robustness and generalization by analyzing the three benchmark data sets - CIFAR10 (Krizhevsky, 2009) , F-MINST (Xiao et al., 2017) and SVHN dataset (Netzer et al., 2011).
In particular, we show that ARoW is superior to existing algorithms including TRADES (Zhang et al., 2019), HAT (Rade & Moosavi-Dezfolli, 2022) and
MART (Wang et al., 2020) as well as PGD-AT (Madry et al., 2018) and GAIR-AT (Zhang et al., 2021) to achieve state-of-art performances.
WideResNet-34-10 (WRN-34-10) (Zagoruyko & Komodakis, 2016) and ResNet-18 (He et al., 2016) are used for CIFAR10 while ResNet-18 (He et al., 2016) is used for F-MNIST and SVHN.
Experimental details are presented in Appendix B.
4. C OMPARISON OF AR O W TO TRADES, HAT AND MART
We compare ARoW to the the regularization algorithms TRADES (Zhang et al., 2019), HAT (Rade & Moosavi-Dezfolli, 2022) and MART (Wang et al., 2020) explained in Section 2.3.2.
Table 1 shows that ARoW outperforms the other regularization algorithms for various data sets and architectures in terms of both the standard and robust accuracies.
The selected values of the hyper-parameters for the other algorithms are listed in Appendix B.2.
To investigate whether ARoW dominates its competitors uniformly with respect to the regularization parameter λ, we compare the trade-off between the standard and robust accuracies of ARoW and other regularization algorithms when λ varies.
Figure 1 draws the plots of the standard accuracies in the x -axis and the robust accuracies in the y -axis obtained by the corresponding algorithms with various values of λ.
For this experiment, we use CIFAR10 and WideResNet-34-10 (WRN-34-10) architecture.
The trade-off between the standard and robust accuracies is well observed (i.e. a larger regularization parameter λ yields lower standard accuracy but higher robust accuracy).
Moreover, we can clearly see that ARoW uniformly dominates TRADES and HAT (and MART)
regardless of the choice of the regularization parameter and the methods for adversarial attack.
Additional results for the trade-off are provided in Appendix F.1.
4. C OMPARISON OF AR O W TO PGD-AT AND GAIR-AT
We compare ARoW with PGD-AT (Madry et al., 2018) and GAIR-AT (Zhang et al., 2021) which are the algorithms minimizing the robust risk directly.
Table 2 shows that ARoW outperforms PGD-AT and GAIR-AT in terms of the standard accuracy and the robust accuracy against to AutoAttack (Croce & Hein, 2020b).
GAIR-AT is, however, better for PGD 20 attack than ARoW. This would be mainly
[Table]
because of the gradient masking (Papernot et al., 2018; 2017) - PGD does not find an adversarial example well.
See Appendix C for details about gradient masking
[Table]
4. A NALYSIS WITH EXTRA DATA
For improving performance on CIFAR10, Carmon et al.
(2019) and Rebuffi et al.
(2021) use extra unlabeled data sets with TRADES.
Carmon et al.
(2019) uses an additional subset of 500K extracted from 80 Million Tiny Images (80M-TI) and Rebuffi et al.
(2021) uses a data set of 1M synthetic samples generated by a denoising diffusion probabilistic model (DDPM)
(Ho et al., 2020) along with the SiLU activation function and Exponential Moving Average (EMA).
Further, Rade & MoosaviDezfolli (2022) shows that HAT achieves the SOTA performance for these extra data.
Table 3 compares ARoW with the exiting algorithms for extra data, which shows that ARoW achieves the state-of-the-art performance when extra data are available even though the margins compared to
HAT are not significant.
Note that ARoW has advantages other than the high robust accuracies.
For example, ARoW is easy to implement compared to HAT since HAT requires a pre-trained model and it needs additional memory.
Moreover, as we will see in Section 4.5, ARoW improves the fairness compared to TRADE while HAT does not.
4. A BLATION STUDIES
We study the following three issues - (i) the effect of label smoothing to ARoW, (ii) the role of the new regularization term in ARoW to improve robustness and (iii) modifications of ARoW by applying tools which improve existing adversarial training algorithms
. 4.4. E FFECT OF L ABEL S MOOTHING
Table 4 indicates that label smoothing is helpful not only for ARoW but also for TRADES.
This would be partly because the regularization terms in ARoW and TRADES depend on the conditional
[Table]
class probabilities and it is well known that label smoothing is helpful for the calibration of the conditional class probabilities (Pereyra et al., 2017)
[Table]
Moreover
[Table]
the results in Table 4 imply that label smoothing is not a main reason for ARoW to outperform TRADES.
Even without label smoothing, ARoW is still superior to TRADES (even with the label smoothing).
Appendix F.2 presents the results of an additional experiment to assess the effect of label smoothing to the performance.
[Table]
4.4. R OLE OF THE NEW REGULARIZATION TERM IN AR O W
The regularization term of ARoW puts more regularization to less robust samples, and thus we expect that ARoW improves the robustness of less robust samples much.
To confirm this conjecture, we do a small experiment.
First, we divide the test data into four groups - least robust, less robust, robust and highly robust according to the values of p θ PGD ( y i | (cid:98) x pgd i ) ( < 0 .
3 ∼ 0 .
5 ∼ 0 .
7 and > 0 . 7 ), where θ PGD is the parameter learned by PGD-AT (Madry et al., 2018) 1 .
Then
[Table]
for each group
[Table]
we check how many samples become robust for ARoW and TRADES, respectively, whose results are presented in Table 5.
Note that ARoW improves the robustness of least robust samples most compared with TRADES.
We believe that this improvement is due to the regularization term in ARoW that enforces more regularization on less robust samples
[Table] [Table]
-AT instead of a standard non-robust training algorithm since all samples become least robust
4.4. M ODIFICATIONS OF AR O W
There are many useful tools which improve existing adversarial training algorithms.
Examples are
Adversarial Weight Perturbation (AWP) (Wu et al., 2020) and Friendly Adversarial Training (FAT)
(Zhang et al., 2020).
AWP is a tool to find a flat minimum of the objective function and FAT uses early-stopped PGD when generating adversarial examples in the training phase.
Details about AWP and FAT are given in Appendix F.4.
We investigate how ARoW performs when it is modified by such a tool.
We consider the two modifications of ARoW - ARoW-AWP and ARoW-FAT, where ARoW-AWP searches a flat minimum of the ARoW objective function and ARoW-FAT uses early-stopped PGD in the training phase of
ARoW.
Table 6 compares ARoW-AWP and ARoW-FAT to TRDAES-AWP and TRADES-FAT.
Both of AWP and FAT are helpful for ARoW and TRADES but ARoW still outperforms TRADES with large margins even after modified by AWP or FAT.
[Table]
4. I MPROVED F AIRNESS
Xu et al.
(2021) reports that TRADES (Zhang et al., 2019) increases the variation of the per-class accuracies (accuracy in each class) which is not desirable in view of fairness.
In turn, Xu et al.
(2021) proposes the Fair-Robust-Learning (FRL) algorithm to alleviate this problem.
Even if fairness becomes improved, the standard and robust accuracies of FRL are worse than TRADES.
In contrast, Table 7 shows that ARoW improves the fairness as well as the standard and robust accuracies compared to TRADES.
This desirable property of ARoW can be partly understood as follows.
The main idea of ARoW is to impose more robust regularization to less robust samples.
In turn, samples in less accurate classes tend to be more vulnerable to adversarial attacks.
Thus, ARoW improves the robustness of samples in less accurate classes which results in improved robustness as well as improved generalization for such less accurate classes.
The class-wise accuracies are presented in Appendix G.
[Table]
C ONCLUSION AND F UTURE W ORKS
In this paper, we derived an upper bound of the robust risk and developed a new algorithm for adversarial training called ARoW which minimizes a surrogate version of the derived upper bound.
A novel feature of ARoW is to impose more regularization on less robust samples than TRADES.
The results of numerical experiments shows that ARoW improves the standard and robust accuracies simultaneously to achieve state-of-the-art performances.
In addition, ARoW enhances the fairness of the prediction model without hampering the accuracies.
When we developed a computable surrogate of the upper bound of the robust risk in Theorem 1, we replaced 1 ( F θ ( X )
̸
= F θ ( z ( X )))) by KL( p θ ( ·| X ) ||
p θ ( ·| (cid:98) X pgd )) .
The KL divergence, however, is not an upper bound of the 0-1 loss and thus our surrogate is not an upper bound of the robust risk.
We employed the KL divergence surrogate to make the objective function of ARoW be similar to that of TRADES.
It would be worth pursuing to devise an alternative surrogate for the 0-1 loss to reduce the gap between the theory and algorithm.
We have seen in Section 4.5 that ARoW improves fairness as well as accuracies.
The advantage of
ARoW in view of fairness is an unexpected by-product, and it would be interesting to develop a more principled way of enhancing the fairness further without hampering the accuracy.
Appendices A P ROOF OF T HEOREM
In this section, we prove Theorem 1.
The following lemma provides the key inequality for the proof.
Lemma 2.
For a given score function f θ , let z ( · ) be an any measurable mapping from X to X satisfying
[Equation]
for every x ∈ X .
Then, we have
[Equation]
Proof.
The inequality holds obviously if 1 { F θ ( x ) ̸ =
F θ ( z ( x )) , Y ̸
=
F θ ( z ( x )) }
= 1 .
Hence, it suffices to show that 1 {∃ x ′ ∈
B p ( x , ε ) :
F θ ( x ) ̸ = F θ ( x ′ ) , F θ ( x )
= Y } =
when either F θ ( x ) =
F θ ( z ( x )) or Y =
F θ ( z ( x )) .
Suppose F θ ( x ) =
F θ ( z ( x )) .
It trivially holds that 1 ( F θ ( x ) ̸
= F θ ( z ( x ))) ≤
( F θ ( x ) ̸ = F θ ( x ′ ))
for every x ′ ∈ X since
( F θ ( x ) ̸
= F θ ( z ( x )))
=
and the equality holds if and only if F θ ( z ( x ))
=
F θ ( x ′ ) .
By the definition of z ( x ) , the left side of (A.1) is
0 since{∃ x ′ ∈
B p ( x , ε ) :
F θ ( x ) ̸ = F θ ( x ′ ) } = 0 , and hence the inequality holds.
Suppose Y =
F θ ( z ( x )) .
If F θ ( x ) =
Y and there exists x ′ in B p ( x , ε ) such that F θ ( x ′ )
̸
= F θ ( x ) , then we have F θ ( x ′ )
̸
= Y =
F θ ( x ) =
F θ ( z ( x )) .
In turn, it implies 1 ( F θ ( x ) ̸
= F θ ( z ( x )))
<( F θ ( x ) ̸ = F θ ( x ′ )) , which is a contradiction to the definition of z ( x ) .
Hence, the left side of (A.1) should be 0 , and we complete the proof of the inequality.
Theorem 1.
For a given score function f θ , let z ( · ) be an any measurable mapping from X to X satisfying
[Equation]
for every x ∈ X .
Then, we have
[Equation]
Proof.
Note that R rob ( θ ) =
R nat ( θ ) + R bdy ( θ ) where R nat ( θ ) =
E ( X ,Y ) 1 { F θ ( X ) ̸ = Y } and
R bdy ( θ
[Equation]
=
[Equation]
X ,Y
[Equation]
[Equation] [Equation]
X
[Equation]
F θ ( X ′ ) , F θ ( X ) =
Y } .
Since
[Equation]
the inequality (4) holds.
B D ETAILED SETTINGS FOR THE EXPERIMENTS WITH BENCHMARK DATASETS B. E XPERIMENTAL S ETUP
For CIFAR10, SVHN and FMNIST datasets, input images are normalized into [0, 1].
Random crop and random horizontal flip with probability 0.5 are used for CIFAR10 while only random horizontal flip with probability 0.5 is applied for SVHN.
For FMNIST, augmentation is not used.
For generating adversarial examples in the training phase, PGD 10 with random initial, p = ∞ , ε = 8 / 255 and ν = 2 / 255 is used, where PGD T is the output of the PGD algorithm (2) with T iterations.
For training prediction models, the SGD with momentum 0 .
9 , weight decay 5 × 10 − 4 , the initial learning rate of 0.1 and batch size of 128 is used and the learning rate is reduced by a factor of 10 at 60 and 90 epochs.
Stochastic weighting average (SWA) (Izmailov et al., 2018) is employed after 50-epochs for preventing from robust overfitting (Rice et al., 2020) as Chen et al.
(2021) does.
For evaluating the robustness in the test phase, PGD 20 and AutoAttack are used for adversarial attacks, where AutoAttack consists of three white box attacks - APGD and APGD-DLR in Croce & Hein (2020b) and FAB in Croce & Hein (2020a) and one black box attack - Square Attack (Andriushchenko et al., 2020).
To the best of our knowledge, AutoAttack is the strongest attack.
The final model is set to be the best model against PGD 10 on the test data among those obtained until 120 epochs.
B. H YPERPARAMETER SETTING
[Table]
5 e −- o
MART2 e −- x
PGD-AT - - 5 e −- o
GAIR-AT - - 5 e −- o
ARoW5 e −0.
o
ResNetTRADES - 5 e −- o
HAT 0.
5 e −- o
MART5 e −- x
PGD-AT - - 5 e −- o
GAIR-AT - - 5 e −- o
ARoW5 e −0.
o
SVHN
ResNetTRADES - 5 e −- x
HAT 0.
5 e −- x
MART5 e −- x
PGD-AT - - 5 e −- x
GAIR-AT - - 5 e −- x
ARoW5 e −0.
x
FMNIST
ResNetTRADES - 5 e −- x
HAT 0.
5 e −- x
MART5 e −- x
PGD-AT - - 5 e −- x
GAIR-AT - - 5 e −- x
ARoW5 e −0.
x
Table 8 presents the hyperparameters used on our experiments.
Most of the hyperparameters are set to be the ones used in the previous studies.
The weight decay parameter is set to be 5 e − 4 in most experiments, which is the well-known optimal value.
Only for MART (Wang et al., 2020) with
WRN34-10, we use weight decay 2 e − 4 as Wang et al.
(2020) did since MART works poorly with used in the experiments of Wang et al.
and we confirm that SWA is not helpful for MART.
Effects of SWA for all methods are provided in Appendix F.3.
C C HECKING THE G RADIENT M ASKING
[Table]
Gradient masking (Papernot et al., 2018; 2017) is the case that the gradient of the loss for a given non-robust datum is almost zero (i.e. ∇ x ℓ ce ( f θ ( x ) , y )
≈ 0 ).
In this case, PGD cannot generate an adversarial example.
We can check the ocuurence of gradient masking when a prediction model is robust to the PGD attack but not robust to attacks such as FAB (Croce & Hein, 2020a), APGD-DLR (Croce & Hein, 2020b) and SQUARE (Andriushchenko et al., 2020).
In Table 9, the robustness of GAIR-AT becomes worse much for the three attacks in AutoAttack except APGD (Croce & Hein, 2020b) while the robustness of ARoW remains stable regardless of the adversarial attacks.
Since APGD uses the gradient of the loss, this observation implies that the gradient masking occurs in GAIR-AT while it does not in ARoW.
Better performance of GAIR-AT for PGD 20 attack in Table 2 is not because GAIR-AT is robust to adversarial attacks but because adversarial examples obtained by PGD are close to clean samples.
This claim is supported by the fact that GAIR-AT performs poorly for AutoAttack while it is still robust to other PGD-based adversarial attacks.
Moreover, gradient masking for GAIR-AT is already reported by Hitaj et al.
D D ETAILED SETTING FOR THE EXPERIMENTS WITH EXTRA DATA
[Table]
In Section 4.
[Table]
we presented the results of ARoW on CIFAR10 with extra unlabeled data used in
Carmon et al.
(2019) and Rebuffi et al.
In this section
[Table]
we provide experimental details.
Rebuffi et al.
(2021) use the SiLU activation function and exponential model averaging (EMA) based on TRADES.
For HAT (Rade & Moosavi-Dezfolli, 2022) and ARoW, we use the SiLU activation function and exponential model averaging (EMA) with weight decay factor 0.995 as is done in
Rebuffi et al.
The cosine annealing learning rate scheduler (Loshchilov & Hutter, 2017) is used with the batch size 512.
The final model is set to be the best model against PGD 10 on the test data among those obtained until 500 epochs.
E A DDITIONAL R ESULTS WITH E XTRA D ATA E. R ESULTS ON CIFAR
For additional experiments, we analyzed CIFAR-100 in WRN-34-10 and found that ARoW still outperforms the other competitors.
[Table]
E. A DDITIONAL R ESULTS WITH E XTRA D ATA
In the main manuscript, we use architecture of ResNet18, while Rade & Moosavi-Dezfolli (2022) use PreAct-ResNet18.
For better comparison, we conduct an additional experiment with extra data where the same architecture - PreaAct-ResNet18 is used.
In addition, we set batch size to 1024 which is used in Rade & Moosavi-Dezfolli (2022).
Table 12 shows that ARoW outperforms HAT both on standard accuracy( +0 . 29% ) and robust accuracy( +0 . 11% ) against
[Table]
F A BLATION STUDY F. T HE TRADE - OFF DUE TO THE CHOICE OF λ
Table 13 presents the trade-off between the generalization and robustness accuracies of ARoW on
CIFAR10 due to the choice of λ, where ResNet18 is used.
The trade-off is obviously observed.
and robust accuracies of ARoW on CIFAR10 for varying λ .
[Table]
F. T HE EFFECT OF LABEL SMOOTHING
Table 14 presents the standard and robust accuracies of ARoW on CIFAR10 for various values of the smoothing parameter α in the label smoothing where the regularization parameter λ is fixed at 3 and
ResNet18 is used.
[Table]
F. E FFECT OF S TOCHASTIC W EIGHT A VERAGING (SWA)
We compare the standard and robust accuracies of the adversarial training algorithms with and without
SWA whose results are summarized in Table 15.
SWA improves the accuracies for all the algorithms except MART.
Without SWA, ARoW is competitive to HAT, which is known to be the SOTA method.
However, ARoW dominates HAT when SWA is applied.
[Table]
F. AWP AND FAT F.4. A DVERSARIAL W EIGHT P ERTURBATION (AWP)
For a given objective function of the adversarial training, AWP (Wu et al., 2020) tries to find a flat minimum in the parameter space.
Wu et al.
(2020) proposes TRADES-AWP, which minimizes
[Equation]
where θ l is the weight vector of l -th layer and γ is the weight perturbation size.
Inspired by
TRADES-AWP, we propose ARoW-AWP which minimizes
[Equation]
In our experiment, we set γ to be 0.005 which is the value used in Wu et al.
and do not use
SWA as did in original paper.
F.4. F RIENDLY A DVERSARIAL T RAINING (FAT)
Zhang et al.
(2020) suggests early-stopped PGD which uses a data-adaptive iterations of PGD when an adversarial example is generated.
TRADES-FAT, which uses the early-stopped PGD in TRADES, minimizes
[Equation]
where t i = min (cid:110) min { t : F θ ( (cid:98) x ( t ) i ) ̸ = y i } + K, T (cid:111) .
Here, T is the maximum iterations of PGD.
We propose an adversarial training algorithm ARoW-FAT by combining ARoW and early-stopped
PGD.
ARoW-FAT minimizes the following regularized empirical risk:
[Equation]
In the experiments, we set K to be 2, which is the value used in Zhang et al.
G I MPROVED FAIRNESS
[Table]
In Table 16, we present the per-class robust and standard accuracies of the prediction models trained by TRADES and ARoW.
We can see that ARoW is highly effective for classes difficult to be classified such as Bird, Cat, Deer and Dog.
For such classes, ARoW improves much not only the standard accuracies but also the robust accuracies.
For example, in the class ‘Cat’, which is the most difficult class (the lowest standard accuarcy for TRADES and ARoW), the robustness and generalization are improved by 4.1 percentage point (26 . 1% → 30 . 2% ) and 9.2 percentage point (65 . 9% → 75 . 1%) by ARoW compared with TRADES, respectively.
This desirable results would be mainly due to the new regularization term in ARoW. Usually, difficult classes are less robust to adversarial attacks.
By putting more regularization on less robust classes, ARoW improves the accuracies of less robust classes more.