Phase Transition from Clean Training to Adversarial Training
Abstract
Adversarial training is one important algorithm to achieve robust machine learningmodels.
However, numerous empirical results show a great performance degradation from clean training to adversarial training (e.g., 90+% vs 67% testing accuracyon CIFAR-10 dataset), which does not match the theoretical guarantee deliveredby the existing studies.
Such a gap inspires us to explore the existence of phasetransition phenomenon with respect to the attack strength: adversarial training isas well behaved as clean training in the small-attack regime, but there is a sharptransition from clean training to adversarial training in the large-attack regime.
Wevalidate this conjecture in linear regression models, and conduct comprehensiveexperiments in deep neural networks.
Introduction
Among various algorithms towards adversarially robust models, adversarial training is a popular andsimple way to improve the adversarial robustness of the model.
Many studies improve adversarialtraining from either theoretical or empirical aspects.
To justify the theoretical properties, an abundant literature studies the performance of adversariallyrobust models under common statistical models, e.g., linear regression, Gaussian mixture model andetc.
in Javanmard et al.
; Mehrabi et al.
(2021); Javanmard and Soltanolkotabi (2020); Danet al.
Taheri et al.
Xing et al.
(2021b).
In addition to the simple models, some other studies work on deep learning models.
For example,
Gao et al.
; Zhang et al.
(2020b); Allen-Zhu and Li (2020) show the convergence of adversarialtraining in neural networks when the attack strength is sufﬁciently small.
Other literature, e.g., Zhanget al.
; Wang et al.
(2019b,a), propose improvements in the training loss of adversarial training.
However, there are some gaps between the promising theoretical results and the performance ofadversarial training in real practice.
First, the aforementioned theoretical works consider either simplemodels or sufﬁciently small attack strength and conduct a similar analysis for adversarial training asfor clean training, in the sense that adversarial training only introduces some additional diminishingerror.
Such settings are different from the setups used in real practice.
Second, some studiesalso reveal several potential concerns of adversarial training in deep learning, e.g., the smoothnessproblem Lee and Chandrakasan (2020);
Xie et al.
(2020), overﬁtting issue Rice et al.
Wu et al.(2020).
Based on RobustBench ( https://robustbench.github.io/ ), the best robust accuracyfor CIFAR-10 using state-of-the-art methods is 66.6% Rebufﬁ et al.
(2021), much worse than theclean accuracy of above 90% in clean training.
As our attempt to ﬁll these gaps, this work focuses on the role of adversarial attack strength.
Forinstance, we are wondering whether taking 8/255 L 1 attack is too strong for CIFAR-10 adversarialtraining using the current state-of-the-art neural networks architectures and optimizer conﬁgurations
Although clean training generalizes well in this setup and adversarial training is an adaptation ofclean training, they may act differently.
This paper investigates how adversarialtraining deviates from clean training as theattack strength grows.
An illustration ofthis can be found in Figure 1.
Under smallattack strength ✏ , the trajectory of adversarial training is close to that of clean trainingso that they converge to the same ﬂat minima.
As ✏ increases, the adversarial training process deviates from the clean trainingdramatically.
Our aim is to determine a critical attack level, denoted as ✏ ⇤ , such that (i)when ✏ ⌧ ✏ ⇤ , (S)GD trajectories of cleantraining and adversarial training are similarin some sense so that they can be handledsimilarly; but (ii) when ✏  ✏ ⇤ , the
ad-
versarial training will signiﬁcantly deviatefrom clean training.
Some other literature also supports our conjecture that adversarial training differs from clean trainingunder large attack strength.
Gong et al.
; Frederickson et al.
(2018) show that, as the attackstrength increases, the attacked samples are more detectable against clean samples.
Such a differencein the training samples leads to the discrepancy between clean and adversarial training.
In addition,the trained model from adversarial training is different from the clean model in terms of its Hessian
Yao et al.
Furthermore, the neural network capacity required by adversarial training is largerthan clean training Xie and Yuille (2019);
Bubeck and Sellke (2021).
Contributions The main contributions of this paper can be summarized as follows:
• Directly deﬁning the critical value ✏ ⇤ requires comparing the whole optimization trajectories
between clean training and adversarial training.
The whole path comparison can be rather difﬁcult.
Thus instead, from theoretical intuitions obtained in simple models, we identify the critical ✏ ⇤ via asurrogate measure that only relies on the training losses at trajectory destination.
In particular, wepropose to use clean testing loss in clean training (denote as R ⇤ (0) ) as a benchmark for adversarialtraining.
If the attack strength ✏ is small enough so that the (proper) adversarial training loss issmaller than R ⇤ (0) , then the adversarial training is similar to the clean training (and vice versa).• To compute ✏ ⇤ , a naive way is to repeatedly perform the adversarial training from scratch (i.e.,initializing from a random start) for each possible ✏ until the corresponding adversarial trainingloss hits R ⇤ (0) .
This could be time-consuming.
To speed up the computation, we also propose amethod based on adversarial ﬁne-tuning and extrapolation to approximate ✏ ⇤ .Due
to the page limit
we postpone many details of approximating ✏ ⇤ to Appendix A.Last but
not
least, we
emphasize
that ✏ ⇤ severs as a diagnosis of the adversarial training, but not a ruleof how to choose the attack level
That
is when ✏
⌧ ✏ ⇤
one
can safely assume adversarial training
works as well
as
clean training
when ✏  ✏ ⇤ , it indicates that the attack strength is too large for the
current data set using the current training conﬁguration
, and
certain remedy is
necessary such that
the
adversarial
training performs as good as its clean
counterpart
It is
beyond our scope to provide acomplete solution of how to adjust adversarial training, but some discussions are given in Section 7.
Other Related Works
Clean Training in Deep Learning Many existing literature studies deep neural networks in cleantraining.
For the training performance, studies such as Du et al.
; Du and Lee (2018) workon the optimization convergence of the empirical training loss.
In terms of the testing performance,there are many works showing the good performance of clean training in different aspects, e.g.,double-descent phenomenon and benign over-ﬁtting Belkin et al.
; Hastie et al.
; Ba et al.(2020); Bartlett et al
[Table]
Baratin et al.
(2020), generalization bounds using PAC-Bayes method or other methods Neyshaburet al.
(2017a); Kohler and Langer (2019);
Hu et al.
Taheri et al.
Adversarial Training In the literature, there are several strands of theoretical studies relatedto adversarial training, e.g., the statistical properties or generalization performance of the globaloptimizer of adversarial training loss without any consideration of solving the optimization problem
Javanmard et al.
Yin et al.
; Raghunathan et al.
; Min et al.
; Zhai et al.(2019);
Hendrycks et al.
; Chen et al.
(2020a)
; Xing et al.
(2021b);
Dan et al.
(2020), the (local)convergence of adversarial training algorithms under convexity assumptions Sinha et al.
Wang et al.
(2019a)
, the adversarial training loss in deep neural networks (Gao et al., 2019; Zhanget al., 2020b; Wu et al., 2020), and the generalization properties of deep neural networks, e.g. upperbound in Allen-Zhu and Li (2020) and lower bound in Bubeck and Sellke (2021).
Adversarial Training
To introduce adversarial training, let l denote the loss function and f ✓ ( x ) be the model with parameter✓ .
The (population) adversarial loss is deﬁned as R ( ✓ , ✏ ) :
=
E [ l ( f ✓ [ x + A ✏ ( f ✓ , x, y )] , y )] , where
A ✏ is an attack of strength ✏ > 0 and intends to deteriorate the loss in the following way
[Equation]
where B p ( x, r ) is a L p ball centering at x with radius r .
Given n i.i.d.
samples S = { ( x i , y i ) } ni =1 ,the adversarial training minimizes the sample version of R ( ✓ , ✏ ) w.r.t.
✓ :
[Equation]
n
[Equation] [Equation]
) , (2) and the estimator b ✓ ( ✏ ) aims to minimize b
R ( ✓ , ✏ ) , and the minimized adversarial training loss is b
R S ( ✏ ) .
We rewrite b
R ( ✓ , ✏ ) as b
R ( ✓ ) for simplicity when there is no confusion.
The minimization in (2) is often implemented through an iterative two-step (min-max) update.
In thet -th iteration, we calculate the adversarial sample e x ( t ) i
=
x i + A ✏ ( f ✓ ( t ) , x i , y i ) based on the current✓ ( t ) , and then update ✓ ( t +1) based on the gradient of the adversarial training loss while ﬁxing e x ( t ) i ’swith learning rate ⌘ t .
The algorithm runs for T iterations.
Note that for some loss function l or modelf ✓ (e.g. deep neural networks), there may not be an analytic form for A ✏ , thus numerical methods,e.g. FGM and PGD, are utilized to approximate A ✏ .
Denote ✓ ( t ) ( ✏ ) as the adversarially trained modelusing attack strength ✏ .
Intuitions from Simple Models
To obtain insights into the critical strength ✏ ⇤ , we consider the linear regression models.
Brieﬂyspeaking, we justify that ✏ ⇤ is in ⇥ ( p d/n ) under L 2 attack by comparing the training trajectoriesof clean and adversarial training, as well as establishing an explicit connection between the criticalbound and the generalization error of clean training.
Simple Linear Regression For simple linear regression problem, to measure the difference betweenthe trajectories of adversarial training and clean training, if ✏ ⌧ p d/n 1 , then asymptotically, there isno difference between adversarial training and clean training during the training process.
Throughoutthe training (with proper early stopping), we always have that the updating gradient of adversarialtraining is dominated by the one in clean training.
To be speciﬁc, we have the following result:
Theorem 1.
Assume Y = ✓ > 0 X
+ " where X 2 R d follows multivariate normal distribution withzero mean and covariance I d , and " is a Gaussian noise with constant variance.
The true coefﬁcient✓ 0 satisﬁes k ✓ 0 k = ⇥ (1) so that V ar ( Y ) = ⇥ (1) as well.
Consider L 2 -adversarial training.
When ✏ ⌧ p d/n , with zero initialization 2 , proper learning rate ⌘ and number of steps T , theoptimization converges to the global risk minimizer, i.e., ✓ ( T ) (0) !
b ✓ (0) and ✓ ( T ) ( ✏ ) !
b ✓ ( ✏ ) .
Besides, with probability tending to 1, for all t  T , the updates in clean and adversarial traininghave insigniﬁcant difference (small attack in Figure 1):
[Equation]
When ✏  p d/n , with probability tending to 1,
• If lim inf d/n > 0 3 , both k b ✓ ( ✏ ) k and k b ✓ ( ✏ ) k / k b ✓ (0) k converge to 0 (implying a training trajectory
wandering around 0, i.e., large attack in Figure 1).
That is, the empirical adversarial risk minimizer
is asymptotically zero, giving all training and testing adversarial predictions as zero.
• If lim d/n = 0 , k b ✓ ( ✏
[Equation]
[Equation]
✓
[Equation]
k
[Equation]
1 , i.e., the adversarial training trajectory will
be statistically different from the clean training process (medium attack in Figure 1).
• Such a difference between b ✓ ( ✏ ) and b ✓ (0) implies the difference in the training trajectories.
The proof (and details of ⌘ and T ) of Theorem 1 is postponed
to the
appendix.
Theorem 1 implies that ✏ ⇤ = ⇥
( p d/n ) , which gives an appropriate asymptotic order for the criticalstrength level under linear regression model.
However, reproducing such an analysis to ﬁnd ✏ ⇤ forcomplex statistical models is more mathematically involved or even intractable.
This motivates us toseek a simpler surrogate measure to identify ✏ ⇤ .
We ﬁnd that the difference between the clean training loss and clean testing loss can be a proper choiceof such a surrogate measure, as illustrated in Theorem 2 below.
Using the loss to do comparison helpsavoid directly comparing model parameters (i.e., ✓ ( t ) ( ✏ ) vs ✓ ( t ) (0) ), which eases the comparison forcomplex models such as neural networks in practice.
Denote  as the difference between the cleantraining loss and clean testing loss of clean trained model, i.e.,  = R ( b ✓ (0) , 0)  b
R ( b ✓ (0) , 0) .
Inpractice, one can use R ( ✓ ( T ) (0) , 0)
b
R ( ✓ ( T ) (0) , 0) as an estimate of  .
Theorem 2.
Under the model setup as in Theorem 1, regardless of the growth of dimension d and sample size n , when ✏ ⌧ p
[Equation]
/n , with high probability,
[Equation]
When ✏  p d/n , the above amount does not go to 0.
The proof of Theorem 2 is postponed to the appendix.
Both Theorems 1 and 2 describe a phase-transition under linear models with the same phase-transition boundary.
This equivalency justiﬁes our idea of deﬁning critical attack strength ✏ ⇤ implicitly via the empirical loss and generalization gap  .
Furthermore, since in real practice mostly we have
R ( b ✓
[Equation]
b ✓
[Equation]
[Equation]
+  , ✏ ⇤ can be deﬁned as
[Equation]
It is worth mentioning that it is not coincident that both the gap of training losses and the difference oftraining trajectories occur at the same point.
Under linear models, the loss function is always convexfor both clean and adversarial training; hence the similarity of gradient descent trajectories highlydepends on the similarity of the loss functions.
Remark 1.
Theorems 2 and 1 reveal a negative relationship between ✏ ⇤ and n , i.e., larger n leadsto a larger discrepancy between adversarial training and cleaning training.
However, it does notimply that a larger n hurts the adversarial training.
With larger n , the adversarial training may actdifferent to clean training, but by the Law of Large Numbers, a large n can force the adversarialtraining to converge to the correct place.
Remark 2.
The concept of ✏ ⇤ is for adversarial training, rather than commonly used data augmentation, e.g., via adding Gaussian noise (Reed and MarksII, 1999).
The noisy sample is randomlyallocated around the original sample.
As a result, it does not hurt the training too much whenconsidering the average effect of Gaussian noise, which is not the case for the adversarial attack.
[Equation]
strengths are and the
) are and zero
Details are in Table B.1 and Table B.1 in the appendix.
Simulation Evidences We use a simulation study to demonstrate the above observations in thetheorems.
In particular, we would like to validate numerically (1) the adversarial training parameteris similar to the one in the clean training when ✏ ⌧ ✏ ⇤ and vice versa, and (2) the adversarial trainingloss is similar to the clean training loss when ✏ ⌧ ✏ ⇤ and vice versa.
To verify (1), we calculate k ✓ ( t ) ( ✏ )  ✓ ( t ) (0) k / k ✓ ( t ) (0) k for each step t = 1 , ..., 100 , and repeatthis experiment 100 times to obtain the average.
Due to page limit, we postponed the detailedconﬁgurations to the appendix.
As shown in Figure 2, when the attack strength is large, there is agreat difference between the clean trained model parameters and adversarial trained model parameters.
Note that in left panel in Figure 2, we only compare the curves for the same ( d, n ) .
The comparisonamong different ( d, n ) is not meaningful because the attack strengths are different.
To verify (2), we repeat 100 times of clean and adversarial training to obtain the distributions of thelosses.
We standardize the adversarial training loss using the mean and variance of the clean-trainedclean loss.
If the adversarial loss distribution is close to clean-trained clean loss, it means that thethese two losses are similar.
As shown in the right panel of Figure 2, when the attack strength is large,there is a signiﬁcant difference between the distributions of the losses.
The two attack strengths aretaken so that 0 .
01 < ✏ ⇤ and 0 . 05 > ✏ ⇤ .
Two-Layer Neural Networks
While the linear regression problem enjoys the above properties, following Ba et al.
Xing et al.
(2021a), the two-layer neural network with vanishing initializationcan be proved to share similar properties as follows:
Proposition 1 (Two-layer neural networks) .
Consider a two-layer neural network with smoothactivation function
[Equation]
with h as the number of hidden nodes.
When h ! 1 , with vanishing initialization of ✓ j and a j , if weﬁx the second layer ( a j ) and only train the hidden layer ( ✓ j ), the network parameter satisﬁes similar(but not the same 4 ) property as Theorem 1, and the loss results are the same as Theorem 2.
The details of Proposition 1 and the main idea of the proof are postponed to the appendix.
Experiments in Deep Neural Networks
In this section, we study the performance of the proposed ✏ ⇤ in terms of (1) whether our choice of ✏ ⇤is reasonable in deep learning, and (2) what implications the ✏ ⇤ can bring towards commonly usedconﬁgurations in deep learning.
5.1 General Conﬁgurations
Here we describe some general setups in the implementation for both this section and the next section.
For datasets CIFAR-10, CIFAR-100, SVHN, we modiﬁed the code of Rice et al.
(2020) for ourimplementation.
We keep all the existing conﬁgurations (optimizer, neural network architecture,transformer) from the original code.
In particular, if there is no speciﬁcation, we use an SGD optimizerwith batch size 128 to train on the full training set for 200 epochs.
The learning rate is initialized as0.1 for CIFAR and 0.01 for SVHN, and is divided by 10 at the 100th and 150th epochs.
We consider
PreActResNet18 and WideResNet34 as those in Rice et al.
After training 200 epochs, weﬁnd the epoch with the smallest adversarial testing loss as the ﬁnal model for early stopping.
For
MNIST, we implement a CNN with two convolution layers and two fully connected layers.
Besides, we also use the loss TRADES and MART in Zhang et al.
(2019) and Wang et al. (2019b).
5.2 Verifying the effectiveness of ✏ ⇤
In this section, we present some metrics of adversarial training to argue that our choice of ✏ ⇤ isreasonable.
We use CIFAR-10 with PreActResNet18 and L 1 in this section.
To obtain ✏ ⇤ , we ﬁrst run a clean training to get the clean testing loss, then run adversarial trainingfor a wide range of ✏ ’s and perform a linear interpolation to obtain ✏ ⇤ .
For both clean training andadversarial training, we train from scratch and summarize the results in Table 1.
From Table 1, onecan see that the clean testing loss for clean training is 0.1837.
The threshold ✏ ⇤ is the attack strengthsuch that the adversarial training loss is 0 . 1837 .
From the table, when ✏ = 0 .
5 , the adversarial trainingloss is 0.1802, implying that ✏ ⇤ is approximately 0.5.
in PreActResNet18 using CIFAR-10.
[Table]
We conduct comparisons in some aspects, e.g., connectivity, overﬁtting, and FGM catastrophicoverﬁtting, to show that the adversarial training with ✏ ⌧ ✏ ⇤ is more similar to the clean trainingcompared to stronger attacks.
Connectivity
The connectivity of deep neural networks aims to answer whether there is a path inthe parameter space between two deep neural networks (of the same architecture), such that all theneural networks along this path have good prediction performance.
It is a useful tool to study the losslandscape of deep neural networks, e.g., Chao et al.
Good connectivity implies that the twoneural networks are in the same “basin” of the loss.
To numerically ﬁgure out a path, we use the method in Garipov et al.
In order to have agraphical presentation, we take num bend as 3, i.e., besides the start and the end neural networks,there is only one neural network to be trained in the path parameters.
Figure 3 shows the connectivitybetween the clean trained model and the robust model.
In each subﬁgure, the left point representsthe clean model at the 101st epoch, the right point is the adversarially trained model at the 101stepoch, and the upper point is the extra checkpoint to be trained.
We take the 101st epoch because theadversarial training generally achieves the best adversarial testing loss around the 101st epoch.
Thesethree points together determine the black curve in the ﬁgure, and they are in the same 2D plane.
Wecalculate the clean training loss for other points in this plane to obtain the contour.
Due to the pagelimit, we postpone the full-size ﬁgures to the appendix.
From Figure 3, one can observe that when ✏ = 0 and ✏ = 0 .
5 , the low-loss region is almost symmetricto the lower two points.
The middle part is also well connected, i.e., most of the connecting pathbelongs to the low-loss region (red area).
When ✏ > 0 .
5 , the low-loss region gradually shifts towardsthe clean model.
This indicates that the adversarial training gradually moves out of the “basin”.
generalization error
the ✏  1 , this ratio is small
Generalization and Overﬁtting In the ture, it is observed that in deep neural networkswith clean training, overﬁtting training data doesnot affect the generalization performance.
Somestudies, e.g., Belkin et al.
(2018, 2019a)
; Li et al.(2021); Chatterji et al.
(2021), provide theoretical justiﬁcations towards this phenomenon.
Our numerical results show no harmful overﬁtting in clean training, and the overﬁtting inadversarial training with ✏ ⇤ is also not severe.
As shown in Figure 4, when ✏ gets larger, theratio (generalization gap)/(generalization gap atthe 101st epoch) increases after the 100th epoch.
FGSM and Catastrophic Overﬁtting Basedon Andriushchenko and Flammarion (2020), FGSM (i.e., the FGM approximation of L 1 -PGD attack)leads to catastrophic overﬁtting when attack strength is large, yielding an almost-zero adversarialtesting accuracy using L 1 -PGD attack.
Since FGM is only an approximation of PGD, the FGMattack is not as strong as the PGD attack, and its adversarial testing performance under PGD attack isexpected to be worse.
However, for L 1 attack, the performance drop in testing accuracy is far moresevere, and Andriushchenko and Flammarion (2020) reveals some threats in FGSM which causethis phenomenon.
This catastrophic overﬁtting is a second type of overﬁtting different from the onediscussed in the previous paragraph.
Based on our intuition of ✏ ⇤ , the above concern does not hurt the training process when ✏ < ✏ ⇤ .
First,since ✏ is small, the difference between FGM and PGD is small.
Second, since adversarial training issimilar to clean training and clean training does not suffer from severe over-ﬁtting problems, thoseproblems can be avoided.
To verify this, we train CIFAR-10 using 10000/50000 samples with PGD/FGM adversarial trainingin different levels of L 1 attacks.
We take ✏ as 1/255, 2/255, 4/255, 8/255, 12/255, 16/255 to seewhether it obtains a large adversarial testing loss with an almost-zero adversarial testing accuracy.
The results are summarized in Table 2.
The column “Stable( ✏ ⇤ )” indicates whether FGSM adversarialtraining is stable or not when taking ✏
= ✏ ⇤ , and it is stable in this experiment.
In addition, the column“Unstable ✏ ” refers to the minimal ✏ under which catastrophic overﬁtting occurs.
[Table]
The previous section conducts numerical experiments to justify our choice of ✏ ⇤ .
In this section, weuse this method in various datasets and neural network architectures to provide more insights andstudy how ✏ ⇤ is affected by these factors.
[Table]
5.3.1 General Observations
An important observation in Table 3 is that the commonly used attack strengths are greater than ✏ ⇤ .
In the literature, we usually use L 1 attack with strength 8 / 255 for CIFAR-10/100 and SVHN
[Table]
and0.3 for MNIST.
From our experiments, we observe that using all the 50000 training samples, thecorresponding ✏ ⇤ for CIFAR-10/100, SVHN [Table] and MNIST are 0 .
5.3.2 More Detailed Observations
Wide neural networks Different neural network architectures lead to different ✏ ⇤ .
For MNIST,using a simple neural network of two convolution layers and two fully connected layers, when thereare 16 hidden nodes for the ﬁrst FC layer, ✏ ⇤ is 26.1/255.
When there are 128 hidden nodes, itbecomes 48/255.
This observation veriﬁes the argument in the existing literature that wider neuralnetwork structures are essential in adversarial training to enlarge the model capacity Xie and Yuille(2019); Author (2021).
For CIFAR-10, ✏ ⇤ changes little when expanding the width of the neuralnetwork from 1 to 10, implying that WideResNet34-10 may not be sufﬁcient for adversarial training.
Note that one need to be cautious when enlarging the network size, as wider neural networks tend tooverﬁt the data even in clean training.
Improved adversarial loss Some literature tries to improve the adversarial training process viaimproving the loss.
We would like to examine how ✏ ⇤ changes along these loss functions as well.
We run TRADES and MART in CIFAR-10 with PreActResNet18 for this experiment, and the ✏ ⇤ are0.7/255 and 0.8/255, respectively.
Although 0.7/255 and 0.8/255 are larger than 0.5/255 (i.e., ✏ ⇤ forvanilla adversarial training; See Table 3), they are still less than 8/255.
Training size From Table 3, for both CIFAR-10 and CIFAR-100, a larger training size indicates asmaller ✏ ⇤ .
There are two reasons for this phenomenon.
First, when enlarging the sample size, thegeneralization gap gets smaller; thus, ✏ ⇤ is smaller.
Second, as observed in Author (2021), whenincreasing the training size, the neural network has a larger norm, implying that a larger modelcapacity is needed to ﬁt the attacked samples.
As a result, the neural network architecture cannothandle the adversarial training properly, so the corresponding ✏ ⇤ is smaller.
Faster Approximation of ✏ ⇤
To compute ✏ ⇤ , a naive way is to repeatedly perform the adversarial training for each possible ✏ untilit hits the threshold.
This could be very time-consuming when solving the adversarial training fromscratch (i.e., initializing from a near-zero random start).
For instance, it takes 20 minutes to train the
CIFAR-10 dataset for clean training but takes 10 hours to complete adversarial training for a given ✏ .
Hence we propose to speed up this process via a linear extrapolation approximation.
Due to the page limit, the intuition, potential difﬁculties, and ﬁnal algorithm for approximating ✏ ⇤are postponed to Section A in the appendix.
Brieﬂy speaking, due to good connectivity between theclean and adversarial training when ✏ ⌧ ✏ ⇤ as in Figure 3, we consider using adversarial ﬁne-tuning b ✏ (FGM, a ) represents that the adversarial ﬁne-tuning uses FGM attack with a runs of ﬁne tuning.
[Table]
(Chen et al., 2020b) to replace the whole adversarial training (from scratch) process in the gridsearch for ✏ ⇤ .
To overcome the potential over-ﬁtting and learning rate tuning problems, we conductadversarial ﬁne-tuning under similar ✏ ’s and use extrapolation to approximate ✏ ⇤ .
Since we are usingextrapolation rather than interpolation, we also provide theoretical support to justify the correctnessof the algorithm.
The numerical results are shown in Table 4.
One can see that the proposed algorithmgives a good estimate of ✏ ⇤ .
What Can We Do If ✏  ✏ ⇤ ?
Our paper focuses on how to determine ✏ ⇤ and the consequence of ✏  ✏ ⇤ , and it is beyond our scopeto study how to adjust adversarial training when ✏  ✏ ⇤ .
We provide some potential solutions.
In general, when ✏  ✏ ⇤ , one needs to overcome a series of potential problems in adversarial training.
In the literature, methods such as MART Wang et al.
(2019b), Dynamic Wang et al.
(2019a),
FAT
Zhang et al.
(2020a), HAT Rade and Moosavi-Dezfooli (2021), smoothing Xie et al.
Xinget al.
(2021a) can overcome some of the problems.
However, they may not be sufﬁcient to resolveevery problem caused by the fundamental gap between adversarial training and cleaning trainingrevealed by this work.
Alternatively, we suggest two other ways: (1) adjusting the neural networkarchitecture to enlarge ✏ ⇤ , and (2) utilizing more information/data to force adversarial training toconverge to the correct place.
For (1), in our numerical experiment, enlarging the size of the neural network can make ✏ ⇤ larger.
Ifcontrolling the over-ﬁtting issue properly, this can be a solution to improve adversarial robustness.
Similarly, empirical experiments in various literature (e.g., Xie and Yuille, 2019; Rice et al., 2020;
Gowal et al., 2021) show that wider neural networks lead to better performance.
Some other studies(e.g., Huang et al., 2021) also study how the neural network architecture affects robustness.
For (2), there are several ways to utilize more information from the data.
For example, Gowalet al.
(2021) trains a clean classiﬁer and an unlabeled data generator to generate extra syntheticdata, which facilitate the adversarial training.
This framework does not introduce any new datasource but improves adversarial robustness, implying that vanilla adversarial training overlooks someinformation from the data.
Although ✏ ⇤ may be smaller than the actual ✏ , using more information/datacan force the training to converge to the correct place.
Similar studies can be found in Carmon et al.(2019);
Xing et al.
(2021b).
Conclusion
Observing the great gap between empirical results in adversarially robust models and the theoretical studies in this area, we conjecture that adversarial training acts differently from our commonunderstanding of clean training in deep learning.
Through intuitions in simple statistical models, wedesign a metric of similarity to determine whether adversarial training is “similar” to clean trainingor not.
Our results reveal that the commonly used adversarial training setups in literature take a largeattack strength so that it is different from clean training given the current neural network architecturesand data.
We reveal some potential factors which affect ✏ ⇤ .
Besides, since adversarial trainingfrom scratch is time-consuming, we propose to use adversarial ﬁne-tuning and extrapolation to doapproximate ✏ ⇤ .
Such a method can reduce 80% running time compared to adversarial training fromscratch while leading to reasonable estimates.
A future direction of this work is to theoretically understand how the neural network architecture affects ✏ ⇤ , and based on which, to provide proper guidance on architecture selection that accommodatesstronger attacks (e.g., the commonly used 8/255 attack strength in practice).