Stability Analysis and Generalization Bounds of Adversarial Training
Anonymous Author(s)
AffiliationAddress
Abstract
In adversarial machine learning, deep neural networks can fit the adversarial examples on the training dataset but have poor generalization ability on the test set.
This phenomenon is called robust overfitting, and it can be observed when adversarially training neural nets on common datasets, including SVHN, CIFAR-10,
CIFAR-100, and ImageNet.
In this paper, we study the robust overfitting issue ofadversarial training by using tools from uniform stability.
One major challenge isthat the outer function (as a maximization of the inner function) is nonsmooth, sothe standard technique ( e.g., [19]) cannot be applied.
Our approach is to considerη -approximate
smoothness: we show that the outer function satisfies this modifiedsmoothness assumption with η being a constant related to the adversarial perturbation.
Based on this, we derive stability-based generalization bounds for stochasticgradient descent (SGD) on the general class of η -approximate smooth functions,which covers the adversarial loss.
Our results provide a different understanding ofrobust overfitting from the perspective of uniform stability.
Additionally, we showthat a few popular techniques for adversarial training ( e.g., early stopping, cycliclearning rate, and stochastic weight averaging) are stability-promoting in theory.
Introduction
[Figure] Deep neural networks (DNNs) [27, 21] have become successful inmany machine learning tasks and rarely suffered overfitting issues[51].
A neural network model can be trained to achieve zero training error and generalize well to the unseen data.
While in the setting of adversarial training, robust overfitting is a dominant issue[35].
Specifically, robust overfitting characterizes a training procedure shown in Fig.
After a particular epoch, the robust test accuracy (black line) starts to decrease, but the robust training accuracy(blue line) is still increasing.
This phenomenon can be observedin the experiments on common datasets, e.g., SVHN, CIFAR-10,
CIFAR-100, and ImageNet.
Recent works [18, 34] mitigated theoverfitting issue using regularization techniques such as stochasticweight averaging (SWA) and early stopping, but it still has a largegeneralization gap.
Therefore, it is essential to study this issue from a theoretical perspective.
Inthis paper, we study robust overfitting issue of adversarial training, or adversarial generalization, byusing tools from uniform stability.
Uniform stability analysis [7] in learning problems has been introduced to measure generalizationgap instead of uniform convergence analysis such as classical VC-dimension [42] and Rademachercomplexity [5].
Gneralization gap can be bounded in terms of uniform argument stability (UAS).
Formally, UAS is the gap between the output parameters θ of running an algorithm A on two datasets S and S ′ differ in at most one sample, denoted as δ ( S, S ′ )
= ∥ θ ( S ) − θ ( S ′ )
∥ .
In a standard trainingproblem with n training samples, assuming that the loss function is convex, L -Lipschitz and βgradient Lipschitz, running stochastic gradient descent (SGD) with step size α ≤ 1 /β for T steps,the UAS is bounded by O ( LαT/n ) [19].
Since the generalization gap is controlled by the numberof samples n , this bound (at least partially) explains the good generalization of a standard trainingproblem.
Beyond [19], [6] considered the case that the loss function is non-smooth.
Without theβ -gradient Lipschitz assumption, they showed that the UAS is bounded by O ( Lα √ T + LαT/n ) andprovided a lower bound to show that the additional term O ( Lα √ T ) is unavoidable.
In adversarialsettings, two works had discussed the stability of adversarial training to our knowledge.
Firstly, [14] considered the stability of minimax problems.
Their work includes a discussion on analgorithm called GDmax (gradient descent on the maximization of the inner problem), which canbe viewed as a general form of adversarial training, and provided the stability of GDmax when theinner problem is further assumed to be strongly concave.
Under the strongly concave assumption,the outer function is smooth.
Then, the generalization bound O ( LαT/n ) can be applied in this case.
However, the inner problem is not strongly concave in practice.
The bound O ( LαT/n ) does notmatch the poor generalization of adversarial training observed in practice.
Another stability analysis of adversarial training is the work of [46].
Before they propose their algorithm, they use the stability bound
O ( Lα √ T + LαT/n ) [6] to characterize the issue of adversarialgeneralization.
However, the bound is ϵ -independent. Considering two cases, ϵ → 0 and ϵ is large( e.g., 16/255). For the first case, adversarial training is very close to standard training and have goodgeneralization ability.
For the second case, adversarial generalization is very poor. The bound is thesame in these two different cases. It cannot provide an interpretation on adversarial generalization.
In summary, existing stability-based generalization bounds [19, 6] have limited interpretation onadversarial generalization.
In this paper, we provide a new stability analysis of adversarial trainingusing the notion of η -approximate smoothness.
We first show that, under the same assumptionsas [19, 46], even though the outer function (adversarial loss) is non-smooth, it is η -approximatelysmooth (see Definition 4.1), where η is a constant linearly depending on the gradient Lipschitzof the inner function and the perturbation intensity ϵ .
Then, we derive stability-based generalization bounds (Thm. 5.1 to 5.4) for stochastic gradient descent (SGD) on this general class ofη -approximate smooth functions, which covers the adversarial loss. Our main result can be summarized in the following equation. Running SGD on adversarial loss for T steps with step sizeα ≤ 1 /β , the excess risk, which is the sum of generalization error and optimization error, satisfies [Equation]
The excess risk of adversarial training have an additional term LηTα .
Our results suggest thatrobust test accuracy decreasesin ϵ when T is large, which is also observed in practice.
It providesan understanding of robust overfitting from the perspective of uniform stability.
Additionally, weshow that a few popular techniques for adversarial training ( e.g., early stopping, cyclic learning rate,and stochastic weight averaging) are stability-promoting in theory and also empirically improveadversarial training.
Experiments on SVHN, CIFAR-10, CIFAR-100, and ImageNet confirm ourresults.
Our contributions are listed as follows:
• Main results: we derive stability-based generalization bounds for adversarial training using the notion of η -approximate smoothness.
Based on this, we provide an analysis tounderstand robust overfitting.•
We provide the stability analysis of a few popular techniques for adversarial training andshow that they are indeed stability-promoting.•
We provide experiments on SVHN, CIFAR-10, CIFAR-100, and ImageNet.
The resultsverify the generalization bounds.•
Technical contribution: we develop a set of properties of η -approximately smooth function,which might be useful in other tasks.
The paper is organized as follows.
After discussing the related work in Sec. 2, the rest of the papercontains two parts.
The first part is the technical part to derive stability bounds.
The second part is toanalyze robust overfitting.
Specifically, in the first part, Sec. 3 introduces the preliminary knowledgeabout UAS.
Sec. 4 provides the Lemma and properties of approximately smooth functions and Sec. 5gives the stability bounds.
In the second part, Sec. 6 analyzes the robust overfitting in the theoreticalsettings and Sec. 7 presents the experiments.
Related Work
Adversarial Attacks and Defense.
Starting from the work of [39], it has been commonly realizedthat deep neural networks are highly susceptible to imperceptible corruptions to the input data [17,9, 8, 29].
A series of work aimed at training neural networks robust to such small perturbations[45, 18] and another line of work aimed at designing more powerful adversarial attack algorithms[2, 41, 9].
Adversarial Generalization.
A series of work tried to explain adversarial generalization in the
Uniform convergence framework, including VC-dimension [3, 30] and Rademacher complexity [25,49, 4].
Uniform algorithmic stability is another framework to study adversarial generalization [14,46].
The work of [37, 33, 50] have shown that in some scenarios achieving adversarial generalizationrequires more data.
The work of [47, 48, 24] studied the generalization in the setting of adversariallinear regression.
[38] studied the generalization of distributional robustness.
The work of [40, 24,11] analyzed adversarial generalization in Gaussian mixture models.
Uniform Stability.
Stability is a classical approach to proving generalization bounds.
It can betraced back to the work of [36].
After a few decades, it was well developed in analyzing the generalization bounds in statistical learning problems [7].
These bounds have been significantly improvedin a recent sequence of works [15, 16].
The work of [10] derived minimax lower bounds for excessrisk and discussed the optimal trade-off between stability and convergence.
Preliminaries of Stability
Consider the following setting of statistical learning.
There is an unknown distribution D overexamples from some space Z .
We receive a sample dataset S
= { z 1 , . . .
, z n } of n examples drawni.i.d.
from D .
The population risk and empirical risk are defined as:
R D ( θ ) def =
E z ∼D h ( θ, z ) and R S ( θ ) def = 1 n n (cid:88)
i =
h ( θ, z i ) , respectively, where h ( · , · ) is the loss function.
Risk Decomposition.
Let θ ∗ and ¯ θ be the optimal solution of R D ( θ ) and R S ( θ ) respectively.
Thenfor the algorithm output ˆ θ =
A ( S ) , the excess risk can be decomposed as
[Equation]
To control the excess risk, we need to control the generalization gap E gen and the optimization gap
E opt .
In the rest of the paper, we use E gen and E opt to denote the expectation of the generalization andoptimization gap.
To bound the generalization gap of a model ˆ θ =
A ( S ) trained by a randomizedalgorithm A , we employ the following notion of uniform stability .
Definition 3.1.
A randomized algorithm A is ε - uniformly stable if for all data sets S, S
′ ∈
Z n suchthat S and S ′ differ in at most one example, we have
[Equation]
Here, the expectation is taken only over the internal randomness of A .
We recall the importanttheorem that uniform stability implies generalization in expectation [19].
Theorem 3.1 (Generalization in expectation) .
Let A be ε -uniformly
stable.
Then, the expectedgeneralization gap satisfies
[Equation]
Therefore, we turn to the properties of iterative algorithms that control their uniform stability.
Stability of Adversarial Training
Adversarial Surrogate Loss.
In adversarial training, we consider the following surrogate loss
[Equation]
where g ( θ ; z ) is the loss function of the standard counterpart.
Usually, g can also be written in theform of ℓ ( f θ ( x ); y ) , where f θ is the neural network to be trained and ( x , y ) is the input-label pair.
We assume the loss function g satisfies the following smoothness assumption.
Assumption 4.1.
The function g satisfies the following Lipschitzian smoothness conditions:
[Equation]
where ∥ · ∥ is Euclidean norm. Assumption 4.1 assumes that the loss function is smooth, which are also used in the stability literature [14, 46], as well as the convergence analysis literature [43, 28].
While ReLU activation functionis non-smooth, recent works [1, 13] showed that the loss function of over-parameterized DNNs issemi-smooth.
It helps justify Assumption 4.1.
Under Assumption 4.1, the following Lemma [28]shows that the loss function of adversarial training is approximate smooth (defined later in Def. 4.1).
Lemma 4.1.
Let h be the adversarial loss defined in Eq.
(4.1) and g satisfies Assumption 4.1.∀ θ 1 , θ 2 and ∀ z ∈ Z , the following properties hold.
(Lipschitz function.)
∥
h ( θ 1 , z ) − h ( θ 2 , z ) ∥
≤ L ∥ θ 1 − θ 2 ∥ . For all subgradient d ( θ, z ) ∈ ∂ θ h ( θ, z ) , we have ∥ d ( θ 1 , z ) − d
θ 2 , z ) ∥
≤ L θ ∥ θ 1 − θ 2 ∥ + 2 L z ϵ .
If we further assume that g ( θ, z ) is µ -strongly concave in z , the adversarial surrogate loss h ( θ, z ) isalso smooth [38].
Therefore, the uniform stability of adversarial training follows [19, 14], see Appendix B.3.
However, g ( θ, z ) is non-strongly concave in practice.
The above results provide limitedexplanations of the poor generalization of adversarial training.
We need to discuss the generalizationproperties of adversarial training under Lemma 4.1.2.
4.1 Basic Properties of Approximate Smoothness
To simplify the notation, we use h ( θ ) as a shorthand notation of h ( θ, z ) .
To simplify the argument, we consider differentiable function h . The results can be extended to non-differentiable cases. Lemma 4.1 motivates us to analyze a function with the following modified smoothness assumption,which we call approximate smoothness assumption.
Definition 4.1.
Let β > 0 and η > 0 .
We say a differentiable function h ( θ ) is η -approximatelyβ -gradient Lipschitz, if ∀ θ 1 and θ 2 , we have
[Equation]
In definition 4.1, η controls the smoothness of the loss function h ( · ) .
When η = 0 , the function h isgradient Lipschitz.
When η → + ∞ , h is a general non-smooth function.
As our discussion before,the adversarial surrogate loss is 2 L z ϵ -approximately smooth.
As far as we know, this assumptionis rarely discussed in the optimization literature since it cannot improve the convergence rate froma general non-smooth assumption.
But it affects the uniform stability, as we will discuss later.
Weneed to develop the basic properties of approximate smoothness first.
Lemma 4.2.
Assuming that the function h is η -approximately β -gradient Lipschitz.
∀ θ 1 , θ 2 and∀ z ∈ Z , the following properties hold.
( η -approximate descent lemma
[Equation]
( η - approximately co- coercive
Assuming in addition that h ( θ, z ) is convex in θ for all z ∈ Z .
Let [ · ] + = max(0 , · ) .
We have
[Equation]
168 We defer the proof to Appendix A. Note that the loss function is L -Lipschitz for every example z ,we have E | h ( θ 1 , z ) − h ( θ 2 , z ) | ≤ L E ∥ θ 1 − θ 2 ∥ , for all z ∈ Z .
To obtain the stability generalizationbounds, we need to analysis the difference ∥ θ T 1 − θ T 2 ∥ , where θ T 1 and θ T 2 are the outputs of running
SGD on adversarial surrogate loss for T iterations on two datasets with only one different sample.
Next we provide the recursive bounds under the approximate smoothness assumption.
Algorithms.
We consider the stochastic gradient descent on the adversarial surrogate loss. i.e.,
[Equation]
where α t is the step size in iteration t , z i t is the sample chosen in iteration t .
We consider twopopular schemes for choosing the examples’ indices i t .
Sampling with replacement : One is topick i t ∼ Uniform { 1 , · · · , n } at each step.
Fixed permutation : The other is to choose a randompermutation over { 1 , · · · , n } and cycle through the examples repeatedly in the order determined bythe permutation.
Our results hold for both variants.
Properties of Update Rules.
We define G α,z ( θ ) =
θ − α ∇ h ( θ, z ) be the update rule of SGD.
The following lemma holds.
Lemma 4.3.
Assuming that the function h is η -approximately β -gradient Lipschitz.
∀ θ 1 , θ 2 and∀ z ∈ Z ,we have
( αη -approximately (1 + αβ ) -expansive.) ∥ G α,z ( θ 1 ) − G α,z ( θ 2 ) ∥ ≤ (1 + αβ ) ∥ θ 1 − θ 2 ∥ + αη.2.
( αη -approximately non-expansive.)
Assuming in addition that h ( θ, z ) is convex in θ for all z ∈ Z ,for α ≤ 1 /β , we have ∥ G α,z ( θ 1 ) − G α,z
( θ 2 ) ∥ ≤ ∥ θ 1 − θ 2 ∥ + αη.
( αη -approximately (1 − αγ ) -contractive.)
Assuming in addition that h ( θ, z ) is γ -strongly convexin θ for all z ∈ Z , for α ≤ 1 /β ,
we have ∥ G α,z ( θ 1 ) − G α,z ( θ 2 ) ∥ ≤ (1 − αγ ) ∥ θ 1 − θ 2 ∥ + αη.
The proof of Lemma 4.3 is based on Lemma 4.2 and is deferred to Appendix A. Lemma 4.3 providesthe recursive distance ∥ θ 1 − θ 2 ∥ from iteration t to t + 1 .
Based on this, we can recursively derivethe distance ∥ θ T 1 − θ T 2 ∥ .
Then, we can obtain the stability generalization bounds.
Stability Generalization Bounds
In the previous section, we have discussed the properties of approximate smoothness and developedtools we need to use.
In this section, we discuss the stability bounds.
5.1 Convex Optimization
We first consider the case that h ( θ, z ) is convex in θ for all z ∈ Z .
Theorem 5.1.
Assuming that h ( θ, z ) is convex, L -Lipschitz, and η -approximately β -gradient Lipschitz in θ for all given z ∈ Z .
Suppose that we run SGD with step sizes α t ≤ 1 /β for T steps.
Then,
[Equation]
The proof is mainly based on Lemma 4.3 that the update rule is approximately non-expansive inthis case.
We defer it to appendix A. We have shown that the adversarial surrogate loss is 2 L z ϵapproximately L θ -gradient Lipschitz.
Let η = 2 L z ϵ in Eq.
(5.1), we directly obtain the stabilitybounds for adversarial training.
In practice, the solution of the inner problem is sub-optimal.
Let ∆ ϵbe the maximum error between the optimal and sub-optimal attacks in each iteration.
We have thefollowing Corollary.
Corollary 5.1 (Uniform stability for sub-optimal attacks adversarial training.) .
Under Assumption4.1, assume in addition that g ( θ, z ) is convex in θ for all given z ∈ Z .
Suppose that we runadversarial training with step sizes α t ≤ 1 /L θ for T steps.
Then, adversarial training satisfiesuniform stability with
[Equation]
Remark:
Corollary 5.1 shows that adversarial training with weak attacks (large ∆ ϵ ) have worseadversarial generalization than that with strong attacks.
This is also observed in practice.
However,∆ ϵ is at most 2 ϵ , the upper bound of AT with different attacks have the same order.
It might due tothe weakness of Assumption 4.1 or uniform stabilty framework.
Interpreting Robust Generalization.
If we let ϵ = 0 in Eq.
(5.2) , it reduced to the generalizationbound in [19] for standard training.
Therefore, the additional generalization error of adversarialtraining comes from the additional term L z ϵ .
The global gradient Lipschitz L z with respect to zplays an important role in generalization.
Even though the perturbation ϵ is small, it is amplified bythe Lipschitz L z and finally hurts the robust generalization.
5.2 Further Discussion on the Generalization Bounds
We first provide a lower bound.
Then we compare our bounds with the existing bounds in Table 1.
Theorem 5.2 (Lower Bound) .
There exists functions h ( θ, z ) , s.t.
h is convex, L -Lipschitz, and ηapproximately β -gradient Lipschitz
in θ for all given z ∈ Z .
Exists S and S ′ differ in one sample.
Suppose that we run SGD with step fixed step sizes α ≤ 1 /β for T steps.
Then,
[Equation]
[Equation]
Assumption
[Table]
[14] convex-strongly concave
[Equation]
[46] convex-nonconcave
[Equation]
Ours convex-nonconcave
[Equation]
Comparison with the existing UAS Bounds.
Comparing with the work of [14], they assume thatthe inner problem is strongly-concave.
Thus the bounds are not comparable.
Strongly concave is astrong assumption in practice.
Therefore, the work of [46] and our analysis focus on the nonconcavecases.
Comparing with the bound O ( L √ Tα + LTα/n ) , our bound captures a critical aspect of adversarial generalization bound: ϵ -dependent.
As observed in practice, the adversarial generalizationgap reduces to the standard generalization gap as ϵ → 0 .
Our bound consists with this observation.
On the contrary, the bound O ( L √ Tα + LTα/n ) is very large when ϵ → 0 .
Additionally, we providea matching lower bound w.r.t η .
The comparison is provided in Table 1.
Comparison with the work of [46]. The work of [46] argued that the max function is not smootheven though the standard counterpart is smooth.
Therefore, they followed the bound in non-smoothcases [6].
Then, they aimed to solve the non-smooth issue. They design a noise-injected algorithmand show its effectiveness in tackling the non-smooth issue. Our work focus on providing betterbounds to interpret robust overfitting. 5.3 Non-convex Optimization and Strongly-convex Optimization
Next, we consider the case that the loss function h is general non-convex and strongly convex.
By
Lemma 4.3, we have
Theorem 5.3.
Assuming that h ( θ, z ) is L -Lipschitz, and η -approximately β -gradient Lipschitz in θfor all given z ∈ Z .
Assuming in addition that 0 ≤ h ( θ, z ) ≤ B for all θ and z .
Suppose that werun SGD with diminishing step sizes α t ≤ 1 / ( βt ) for T steps.
Then
[Equation]
Theorem 5.4.
Assuming that h ( θ, z ) is γ -strongly convex, L -Lipschitz, and η -approximately βgradient Lipschitz in θ for all given z ∈ Z .
Suppose that we run SGD with step sizes α t ≤ 1 /β for T steps.
Then
[Equation]
Remark:
The bound in non-convex case provides similar interpretation on robust generalization tothe analysis in the convex case.
In uniform stability analysis, whether the loss function is convex ornon-convex does not give a major difference.
Therefore, we provide the analysis on the non-convexcase in Appendix B. We also provide our convergence analysis of running SGD on a η -approximatesmoothness, non-convex function in Theorem B.1.
Strongly-convex is a strong assumption, we leavethe analysis of the bound in strongly-convex case in Appendix B.2.
Excess Risk Minimization
Based on the risk decomposition, we have Excess Risk ≤ E gen + E opt , we need to minimize E gen +
E opt to achieve better performance.
As our previous discussion, whether the loss function is convexor non-convex does not give a major difference in stability analysis.
We study the convex case inthis section.
We leave the discussion on the non-convex and strongly-convex cases in Appendix B.1and B.2, respectively.
We first introduce the optimization error.
Optimization Analysis.
The convergence analysis of SGD on a L -Lipschitz, convex function isdiscussed in [31].
The convergence rate cannot be improved if we further assume that the functionh is gradient Lipschitz.
Therefore, a weaker condition, approximately gradient Lipschitz, cannotimprove the convergence rate.
We use the following convergence error bound for the optimizationerror of both adversarial training and standard training.
Theorem 6.1 ([31]) .
Assuming that h ( θ, z ) is L-Lipschitz and convex in θ for all given z ∈ Z .
Let
D = ∥ θ 0 − θ ∗ ∥ , where θ 0 is the initialization of SGD.
Suppose that we run SGD with step sizes α tfor T steps.
Then, ∃ k ≤ T , s.t.
[Equation]
If we let α t = 1 / √ T , we have E opt ≤ O (1 / √ T ) , which is the convergence rate of SGD on convexfunction.
Since we need to consider the generalization and optimization errors simultaneously, wekeep the α t in Theorem 6.1.
Generalization-Optimization Trade-off.
Now, we have discussed the optimization error and generalization error of adversarial training.
Our goal is to find the optimal trade-off between generalization and optimization in terms of α t and T .
However, finding the optimal α t and T simultaneouslyis challenging.
We consider the settings we use in practice.
Fixed Step Size.
We first consider the simplest case, the step size α is fixed.
Then, combining Eq.(5.1) and Eq.
(6.1), we have
[Equation]
Interpretation of Robust Overfitting.
In standard training, overfitting is rarely observed in practice.
The optimization error and generalization error are both small.
In Eq.
(6.2), the second to thefourth terms are for standard training.
The second term is controlled by the number of samples n ,which is small if we have sufficient training samples.
The third term is controlled by T , and thelast term is fixed given a small α .
This bound partially explains the good performance of standardtraining.
However, we have an additional term LηTα for excess risk in adversarial training.
Then,after a particular iteration that LηTα dominates Eq.
(6.2), robust overfitting appears.
This is consistent with the training procedure in practice.
Therefore, the η = 2 L z ϵ approximate smoothness ofadversarial loss provides a possible explanation of robust overfitting.
To achieve better performance,we need to stop training the model earlier.
Early Stopping.
It is shown that early stopping is an important training technique for adversarialtraining [35].
In Eq.
(6.2), if we optimize the right-hand-side with respect to T , we have
[Equation]
Therefore, it is best to stop training at T ∗ .
However, T ∗ is unknown in practice.
It is important toselect stopping criteria.
For example, we can use a validation set to determine when to stop.
Varying Step Size.
We discuss one popular varying step size schedule, cyclic learning rate, whichis also called super-converge learning rate for adversarial training.
It is shown that it can speed upadversarial training with fewer epochs [44].
The Super-converge learning rate follows the followingrules.
In the first phase (warm-up), the step size increase from 0 to α ′ linearly.
In the second phase(cold down), the step size decreases back to 0 linearly.
It is unclear (to our knowledge) why thisschedule can speed up convergence in optimization theory.
But the generalization part can partiallybe explained by UAS.
If we set α ′ =
2 α , it is easy to check that E gen ≤ ( Lη + 2 L 2 /n )
Tα in thiscase, which is the same as the bound in the fixed learning rate case.
Notice that the cyclic learningrate usually requires fewer steps T to converge.
Then, the generalization gap is smaller.
Cycliclearning rate can be viewed as another form of early stopping from the perspective of UAS.
Stochastic Weight Averaging.
SWA is also a useful training technique for adversarial training[22].
Instead of using the last checkpoint, SWA suggests using the average of the checkpoints forinference.
It is shown that SWA can find a model with better generalization since it leads to widerminima [23].
Below we study SWA from the perspective of UAS.
Theorem 6.2.
Assuming that h ( θ, z ) is convex, L -Lipschitz, and η -approximately β -gradient Lipschitz in θ for all given z ∈ Z .
Suppose that we run SGD with step sizes α t ≤ 1 /β for T steps.
Let ¯ θbe the average of the trajectory.
Then,
[Equation]
In words, SWA reduces the generalization error bound to one-half of the one without SWA.
But thetraining error bound remains unchanged.
Experiments
Training Settings.
We mainly consider the adversarial training experiments on CIFAR-10 [26],
CIFAR-100, and SVHN [32].
We also provide one experiment on ImageNet [12].
For the first threedatasets, we conduct the experiments on training PreActResNet-18, which follows [35], For theexperiment on ImageNet, we use ResNet-50 [20], following the experiment of [29].
For the innerproblems, we adopt the ℓ ∞ PGD adversarial training in [29], the step size in the inner maximizationis set to be ϵ/ 4 on CIFAR-10 and CIFAR100 and is set to be ϵ/ 8 on SVHN.
Weight decay is set tobe 5 × 10 − 4 .
Additional experiments are provided in Appendix C.
[Figure]
(d
[Figure]
Robust Overfitting on Common Dataset.
In Fig. 2 (a), (b), and (c), we show the experimentson the piece-wise learning rate schedule, which is 0.1 over the first 100 epochs, down to 0.01 overthe following 50 epochs, and finally be 0.001 in the last 50 epochs, on CIFAR-10, CIFAR-100, and SVHN.
Experiments on different ϵ are shown in Appendix C. Robust Overfitting on ImageNet.
Weprovide one experiment on ImageNet in Fig. 2 (d).
We start from a pre-trained model from Madry’s
Lab and keep running 50 more epochs.
Robust overfitting can be observed in these experiments.
After a particular epoch (around the 50th epoch), the robust training accuracy is still increasing, butthe robust test accuracy starts to decrease.
Fixed Step Size To better understand robust overfitting and match the theoretical settings (in Eq.(6.2)), we consider the fixed learning rate schedule.
In Fig. 3, we show the experiments of adversarial training using fixed step size equals to 0 .
The perturbation intensity ϵ is set to be 2, 4, 6, 8,and 10.
respectively.
Fig.
3 (a) and (b) show the experiments on CIFAR-10.
Fig.
3 (c) and (d) showthe experiments on CIFAR-100, respectively.
Generalization Errors dominates Training Errors.
In the robust overfitting phase, the robustgeneralization error dominates the robust training error.
This phenomenon corresponds to the Eq.(6.2) that the first term dominates the other terms with large T .
[Figure]
Robust Test Accuracy Decreases in ϵ .
Comparing different ϵ in Fig. 3, we can see that the robusttest accuracy decreases faster when ϵ is larger.
This corresponds to the robust overfitting rule inthe theoretical setting that the test performance decreases in ϵ .
These phenomena are similar intheoretical and practical settings.
Therefore, the stability analysis provides a different perspectiveon understanding robust overfitting.
Conclusion
Limitations and Future work. Firstly, in Fig 3, we can see that the decrease rate of robust overfiting is closed to O ( T ) when ϵ is small and is closed to O ( √ T ) when ϵ is large.
One possible directionis to figure out the relation.
Secondly, one might improve adversarial training by controlling L z .
L zdepends on the loss, the network architecture, and the dataset.
One possible direction is to design asmoother loss or smooth activation function (e.g., SiLU) for a lower L z .
Notice that L z is uniformfor all θ .
If we view L z ( θ ) locally with respect to θ , we might use an (approximated) second-orderpenalty term on its magnitude to control it.
In this paper, we show that the adversarial loss satisfies η -approximate smoothness
and we derive stability-based generalization bounds on this general class of η -approximate smooth functions.
Our bounds give a different perspective on understanding robust overfitting.
The robust test accuracy decreases in η and experimental results confirm this phenomenon.
We think our work will inspire more theoretical and empirical research to improve adversarial training
A Proof of the Theorem
A.1 Proof of Theorem 3.1
The proof can be found in [19].
We provide the proof in Appendix for reference.
Denoteby S =
( z 1 , . . .
, z n ) and S ′ =
( z ′ 1 , . . .
, z ′ n ) two independent random samples and let S ( i ) =(
z 1 , . .
, z i − 1 , z ′
i , z i +1 , . . .
, z n ) be the sample that is identical to S except in the i ’th example wherewe replace
z i with z ′ i .
With this notation, we get that
[Equation]
where we can express δ as
[Equation]
Furthermore, taking the supremum over any two data sets S, S ′ differing in only one sample, we canbound the difference as
[Equation]
by our assumption on the uniform stability of A.
The claim follows.
A.2 Proof of Lemma 4.
Let the adversarial examples for parameter θ
and θ 2 be
[Equation] [Equation]
then we have
[Equation]
where the first inequality is based on the fact that g ( θ 1 , z 1 ) ≥ g ( θ 1 , z 2 ) and g ( θ 2 , z 2 ) ≥ g ( θ 2 , z 1 ) ,the second inequality is based on Assumption 4.1.
This proves Lemma 4.1.1.
For all subgradient d ( θ, z ) ∈ ∂ θ h ( θ, z ) , we have
[Equation]
where the first and the third inequality is due to triangle inequality, the second inequality is based on
Assumption 4.1.
This proves the second inequality (non-gradient Lipschitz) in Lemma 4.1.
□ A.3 Proof of Lemma 4.
Proof of Lemma 4.2.1 ( η -approximate descent Lemma).
Let ˜ θ be a point in the line segment of θ 1 and θ 2 , ˜ θ ( u )
= θ 2 + u ( θ 1 − θ 2 ) , then
[Equation]
θ
[Equation]
□
Proof of Lemma 4.2.2 ( η -approximate co-coercive).
By Lemma 4.2.1 ( η -approximate descent Lemma), we have
[Equation]
Let θ ∗ be a minimizer of h , then
[Equation]
where t = ∥ θ 1 − θ 2 ∥ and v = ( θ 1 − θ 2 ) / ∥ θ 1 − θ 2 ∥ .
Then
[Equation]
If ∥∇ θ h ( θ 2 ) ∥ − η ≤ 0 , the quaduatic function is optimized when t = 0 .
Thenh
[Equation]
= h [Equation] If ∥∇ θ h ( θ 2 ) ∥ − η ≥ 0 , then
[Equation]
Therefore, we obtain that
[Equation]
Define
[Equation]
and h
[Equation]
Firstly, it is easy to see that
h 1 ( w ) and h 2 ( w ) are both η -approximate β -gradient Lipschitz, whichsatisfies inequatily in Eq.
(A.1).
Secondly, w = θ 1 minimizes h 1 ( w ) .
Then
[Equation]
Similarly, we have
[Equation]
Take the summation of Eq.
(A.2) and Eq.
(A.3), we have
[Equation]
A.4 Proof of Lemma 4.
Proof of Lemma 4.3.1 ( αη -approximately (1 + αβ ) -expansive.).
∥
[Equation]
□
Proof of Lemma 4.3.2 ( αη -approximately non-expansive.)
Let t
= ∥∇ h ( θ 1 ) − ∇ h
( θ 2 ) ∥ .
If t ≤ η , we have
[Equation]
If t ≥ η , we have
[Equation]
Let α ≤ 1 /β
[Equation]
then
[Equation]
By the definition of η -approximate smoothness
[Equation]
[Equation]
Then
[Equation]
Therefore, we obtain that
[Equation]
□
Proof of Lemma 4.3.3 ( αη -approximately (1 − αγ ) -contraction.).
Firstly, if h ( θ ) is a γ -strongly convex, η -approximately β -gradient Lipschitz function, ϕ ( θ )
=h ( θ ) − γ {+
 ∥+} θ ∥ 2 is a convex, η -approximate ( β − γ ) -gradient Lipschitz function.
The proof of convexityfollows the definition.
To see the second claim, since
[Equation]
Therefore, ϕ ( θ ) satisfies the η -approximate ( β − γ ) -descent
Lemma.
Let t = ∥∇ ϕ
( θ 1 ) − ∇ ϕ ( θ 2 ) ∥ .
If t ≤ η , we have
[Equation]
If t ≥ η
[Equation]
we have
[Equation]
Since α ≤ 1 /β
[Equation]
we have (1 − αγ ) / ( β − γ ) ≥ α
[Equation]
then
[Equation]
[Table]
[Equation]
[Table]
[Equation]
Therefore, we obtain that
[Equation]
□
A.5 Proof of Theorem 5.
The proof follows the standard techniques for uniform stability.
We need to replace the nonexpansive property used in standard analysis by the approximately non-expansive property.
Let
S and S ′ be two samples of size n differing in only a single example.
Consider two trajectories θ 11 , . .
, θ T 1 and θ 12 , . .
, θ T 2 induced by running SGD on sample S and S ′ , respectively.
Letδ t = ∥ θ t 1 − θ t 2 ∥ .
Fixing an example z ∈ Z and apply the Lipschitz condition on h ( · ; z ) , we have [Equation]
[Equation]
Observe that at step t, with probability 1 − 1 /n, the example selected by SGD is the same in both
S and S ′ .
With probability 1 /n the selected example is different.
Therefore, by the αη -approximatenon-expansive property, we have
[Equation]
Unraveling the recursion gives
[Equation]
Plugging this back into Eq. (A.4), we obtain
[Equation]
Since this bounds holds for all S, S ′ and z, we obtain the desired bound on the uniform stability. □
A.6 Proof of Theorem
Proof: The construction of function h is adopted from the construction in the work of [6].
[-Theorem 5.
-]Let T ≤ d , and v, K ≥ 0 . Considering Z = { 0 , 1 } , and the objective function [Equation] where r = ( − 1 , · · · , − 1 , 0 , · · · , 0) (i.e., equals to -1 for the first T components).
Function h isη -approximately smoooth since the first case is a piecewise linear function.
For the dataset S and S ′differ in at most one sample, the empirical objective functions are [Equation] and [Equation]
{+Let θ T 1 and θ T 2 be the trajectories runing algorithm on dataset S and S ′ , initialized on θ 
+}
= θ 02 = 0 .
Clearly, θ t 2 = 0 for all t .
It is easy to obtain θ t 1 = − tαrnk − αη n − 1 n (cid:80) t − 1 s =1 e s recursively.
By theorthogonality of the subgradients, we have
[Equation]
On the other hand, the work of [19] provided a lower bound for general non-smooth function
[Equation]
Combining Eq.
(A.7) and Eq.
(A.9), we have
[Equation]
□ A.7 Proof of Theorem 5.
We consider a general form of Theorem 5.3.
Theorem A.1 (Non-convex) .
Assuming that h ( θ, z ) is L -Lipschitz, and η -approximately β -gradient
Lipschitz in θ for all given z ∈ Z .
Assuming in addition that 0 ≤ g ( θ, z ) ≤ B for all θ and z .
Suppose that we run SGD on the adversarial surrogate loss with step sizes α t ≤ c/t for T steps,where c > 0 .
Then, for all t 0 ∈ { 1 , 2 , · · · , n } , adversarial training satisfies uniform stability with
[Equation]
A.10)
Let q = βc .
For small T (s.t. t 0 ≤ n when we set the first term equals to the second term).
we selectt 0 to optimize the right hand side, then
[Equation]
For arbitrary T , the optimal t 0 > n .
We simply let t
= 1 , then
[Equation]
Proof: Let S and S ′ be two samples of size n differing in only a single example.
Consider twotrajectories θ 11 , . .
, θ T 1 and θ 12 , . .
, θ T 2 induced by running SGD on sample S and S ′ , respectively.
Let δ t = ∥ θ t 1 − θ t 2 ∥ .
Let t 0 ∈ { 0 , 1 , . .
. , n } , be the iteration that δ t 0 = 0 , but SGD picks two differentsamples form S and S ′ in iteration t
+ 1 , then
[Equation]
Let ∆ t =
E
[ δ t | δ t 0 = 0] .
Observe that at step t, with probability 1 − 1 /n, the example selected by
SGD is the same in both S and S ′ .
With probability 1 /n the selected example is different.
Therefore,by the αη -approximate ( 1 + αβ )-expansive property, for every t ≥ t 0 ,
[Equation]
Here we used that 1 + x ≤ exp( x ) for all x.
Using the fact that ∆ t 0 = 0 , we can unwind this recurrence relation from T down to t 0 + 1 .
Thisgives
[Equation]
Plugging this bound into (A.13), we get
[Equation]
Let q = βc .
For small T (s.t. t 0 ≤ n when we set the first term equals to the second term).
we selectt 0 to optimize the right hand side, then
[Equation]
For arbitrary T , the optimal t 0 > n .
We simply let t
= 1 , then
[Equation]
Since the bound we just derived holds for all S, S ′ and z, we immediately get the claimed upperbound on the uniform stability.
Let q = 1 , we obtain the result of Theorem 5.3.
□
A.8 Proof of Theorem 5.4
The proof follows the idea in convex case.
By the αη -approximately (1 − αγ ) -contraction, for everyt,
[Equation]
Unraveling the recursion gives
[Equation]
Plugging the above inequality into Eq. (A.4), we obtain
[Equation]
Since this bounds holds for all S, S ′ and z, the Theorem follows.
□
A.9 Proof of Theorem [-6.
-]6.
Let ¯ θ = 1 T
T (cid:88) t
= θ t denote the average of the stochastic gradient iterates.
Sinceθ t = t (cid:88) k
= α k ∇ h
( θ k ; z k ) , we have
[Equation]
Using the αη -approximate non-expansive, we have
[Equation]
which implies
[Equation]
Since f is L -Lipschitz, we have
[Equation]
Here the expectation is taken over the algorithm and hence the claim follows by our definition ofuniform stability.
E opt follows [31].
□
B Discussion on Non-convex and Strongly-convex Case
B.1 Discussion on Non-convex Case
To discuss the generalization-optimization trade-off in the non-convex case.
We first give the optimization error bound.
Theorem B.1.
Assuming that h is η -approximate β -gradient Lipschitz and given 0 < τ < 1 .
Without loss of generality, assume the stochastic gradient ∇ ˜
h ( θ ) be unbiased and have a boundedvariance σ 2 .
Let the stochastic gradient descent (SGD) update be θ t +
=
θ t − α ∇
˜ h
( θ t ) with aconstant step size α = 1 / √ T for number of iterations T ≥ ( β/ 2(1 − τ
[Equation]
[Equation]
∃ t
[Equation]
T , s.t.
[Equation]
Proof:
Assuming that the stochastic gradient ∇ h ( θ ) be unbiased and have a bounded variance σ 2 .
[Equation]
[Equation]
Notice that when θ is a random vector, the above expectation is condition on θ .
Let the stochasticgradient descent (SGD) update be θ t +
=
θ t − α ˜ h
( θ t ) with a constant step size α = 1 / √ T .
Byη -approximately Descent Lemma, we have
[Equation]
Given θ t , take the conditional expectation over the noised introduced by SGD, we have
[Equation]
where the first inequality is the assumption of SGD, the second inequality is the Jensen’s inequality,the third one is the assumption of SGD, the fourth one is the Cauchy-schwarz inequality, and thelast one is because of the size of step size α .
Take the expectation over the trajectory θ 0 , θ 1 , · · · , θ T ,and take the average aver t = 0 , 1 , · · · , T , we have
[Equation]
Let E
[ h ( θ 0 )] − h ( θ ∗ ) =
D and divide α on both side.
∃ t ≤ T , s.t.
[Equation]
Since α = 1 / √ T , we have
[Equation]
Then we remove the absolute value
[Equation]
and obtain
[Equation]
By Cauchy-Schwarz inequality
[Equation]
we have [Equation] □ In words, running SGD on an approximately smooth non-convex function, the algorithm cannotconverge to a stationary point but with an additional constant term.
Notice that this is an error boundfor gradient norm.
If we need an error bound for the optimality gap, we need an additional PLcondition.
Combining the optimization error and generalization error, we have
[Equation]
where the first term is an additional term for adversarial training, which induces robust overfitting.
Therefore, we can see that the analysis of the convex and non-convex cases does not have a majordifference.
To simplify the argument, we only discuss the convex case in the main paper.
B.2 Discussion on Strongly Convex Case
By [31],
[Equation]
in the strongly-convex case.
Whether the function is smooth does not affect the convergence rate.
Therefore,
[Equation]
This result shows that robust overfitting will disappear if the loss function is strongly convex.
Butthe performance of adversarial training is still worse than the performance of standard training in
O ( η ) in this strong assumption.
B.3 Discussion on µ -Strongly-concave Assumption on the Inner Problem
In this subsection, we discuss the case that g ( θ, z ) is µ -strongly-concave in z .
Assumption B.1.
The function g satisfies the following Lipschitzian smoothness conditions:
[Equation]
Assumption B.1 assumes that the loss function is smooth (in zeroth-order and first-order), which arealso used in the stability literature [14, 46], as well as the convergence analysis literature [43, 28].
Comparing to Assumption 4.1, Assumption B.1 requires one more gradient Lipschitz ∥∇ z g ( θ 1 , z ) −∇ θ g ( θ 2 , z )
∥ ≤ L zθ ∥
θ 1 − θ 2 ∥ .
Lemma B.1.
Under Assumption B.1, assume in addition that g ( θ, z ) is µ -strongly-concave in z .∀
θ 1 , θ 2 and ∀ z ∈ Z , the following properties hold.
(Lipschitz
[Equation]
[Equation]
The proof can be found in [38, 43].
Therefore, the adversarial surrogate loss is β 2 -gradient Lipschitz.
The stability generalization bounds follows [19] by replacing β by β 2 (for the choice of step size α ).
C Additional Experiments
In Fig.
4, we show the experiments of adversarial training using a fixed learning rate. In Fig. 5, weshow the experiments of adversarial training using a standard piece-wise learning rate.
In Fig. 6,we show the experiments of adversarial training on SVHN, CIFAR-10, and CIFAR-100 using cycliclearning rate.
Cyclic Learning Rate.
We illustrate the experiments of adversarial training using cyclic learningrate in Appendix C.
The learning rate increases linearly in the first 80 epochs and decreases to zero inthe last 120 epochs.
This learning rate mainly contributes to optimization.
In terms of generalization,as discussed in theoretical settings, the generalization gap is no larger than that of the previous cases.
[Figure]
[Figure] [Figure]