A DVERSARIAL R ADEMACHER C OMPLEXITY OF D EEP N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Deep neural networks are vulnerable to adversarial attacks.
Adversarial training is one of the most effective algorithms to increase the model’s robustness.
However, the trained models cannot generalize well to the adversarial examples on the test set.
In this paper, we study the generalization of adversarial training through the lens of adversarial Rademacher complexity.
Current analysis of adversarial
Rademacher complexity is up to two-layer neural networks.
In adversarial settings, one major difﬁculty of generalizing these results to deep neural networks is that we cannot peel off the layer as the classical analysis for standard training.
We provide a method to overcome this issue and provide upper bounds of adversarial
Rademacher complexity of deep neural networks.
Similar to the existing bounds of standard Rademacher complexity of neural nets, our bound also includes the product of weight norms.
We provide experiments to show that the adversarially trained weight norms are larger than the standard trained weight norms, thus providing an explanation for the bad generalization performance of adversarial training.
I NTRODUCTION
Deep neural networks (DNNs) (Krizhevsky et al.
; Hochreiter & Schmidhuber (1997)) have become successful in many machine learning tasks such as computer vision (CV) and natural language processing (NLP).
But they are shown to be vulnerable to adversarial examples (Szegedy et al.
Goodfellow et al.
A well-trained model can be easily attacked by adding a small perturbation to the original data.
Adversarial training is one of the most effective algorithms to defend against adversarial attacks.
However, generalization is one of the main issues of adversarial training.
An adversarially-trained model will overﬁt the adversarial examples on the training dataset, and it cannot generalize well to the adversarial examples on the testset.
For example, in the experiment of training
ResNet (He et al.
on CIFAR-10 (Krizhevsky et al.
, Projected gradient descent (PGD) adversarial training achieves 100% robust accuracy on the training set, but it only gets 45% robust accuracy on the test set (Madry et al.
Recent works (Gowal et al.
; Rebufﬁ et al.
(2021)) mitigate the overﬁtting issue, but it still has a 20% generalization gap between robust test accuracy (60%) and training accuracy (80%).
On the other hand, a standard trained model can generalize well to the test set with a 5% generalization gap.
To understand why the generalization of adversarial training behaves differently from standard training, we study the generalization issue of adversarial training through the lens of Rademacher complexity.
In classical machine learning theory, Rademacher complexity measures the generalization capacity of machine learning models.
For depth- d neural networks, assuming that the weight matrices
W 1 , W 2 , · · · , W d in each of the d layers have Frobenius norms bounded by M 1 , · · · , M d , and all the data x have (cid:96) 2 -norm bounded by B , given m training samples, the generalization gap between population risk and empirical risk scales as O ( B 2 d (cid:81) dl
=1 M l / √ m ) with high probability (Neyshabur et al.
Another works provide different norm-based complexity, such as (cid:107) · (cid:107) 1 , ∞ -norm (Bartlett & Mendelson (2002)) and spectral norm (Bartlett et al.
The work of (Golowich et al.
(2018)) reduces the dependence on depth- d from 2 d to √ d .
The proofs of the above bounds are based on the induction on layers, which is also called the ‘peeling off’ techniques.
For more details, see section 3.
In adversarial training, adversarial Rademacher complexity was ﬁrst introduced in (Yin et al.
Khim & Loh (2018)) to measure the robust generalization gap.
They prove that the robust generalization gap of linear function ( x → w T x ) scales as O (( B + (cid:15) ) M/ √ m ) , where M is the upper bound of norm of the weights w and (cid:15) is the perturbation intensity of adversarial attacks.
Awasthi et al.
(2020) provides an upper bound in two-layers neural network cases.
For depth-2, width- h neural networks, with high probability, the generalization gap scales as O (( B + (cid:15) ) √ hqM 1 M 2 (cid:112) log m/m ) , where q is the dimension of the data x .
One might think it is straightforward to use the induction methods in (Neyshabur et al.
Golowich et al.
to extend the adversarial Rademacher complexity in (Yin et al.
; Khim & Loh (2018))
to multi-layers cases.
However, it seems challenging to apply their induction methods to adversarial cases.
Let the adversarial loss be max (cid:107)
[Table]
(cid:107)≤ (cid:15) (cid:96) ( f ( x (cid:48) ) , y ) and f is a DNN.
We cannot peel off the layer because of the max operation in the adversarial loss.
The work of (Khim & Loh (2018)) and (Gao & Wang (2021)) also indicate the difﬁculty of analyzing adversarial Rademacher complexity of DNNs.
They analyze other variants of adversarial Rademacher complexity, which are quite different from the original adversarial Rademacher complexity.
See more detailed discussions of these works in section 3.
To our knowledge, direct analysis of the original adversarial Rademacher complexity is largely missing.
In this paper, we analyze the adversarial Rademacher complexity of deep neural networks.
Speciﬁcally, for depth- d , width-
h fully connected neural networks, with high probability, the robust generalization gap scales as
[Equation]
Similar to the existing bounds of standard Rademacher complexity of deep neural nets, the bound includes the product of weight norms (cid:81) dl
=1 (cid:107) W l (cid:107) , but they are trained by different algorithms.
We empirically show that the adversarially trained weight norms are larger than the standard trained weight norms, which provide an explanation why adversarial training did not generalize well.
Our contributions are listed as follow:
We provide a method and give upper and lower bounds for the adversarial Rademacher complexity of deep neural nets.
Compared to standard Rademacher complexity, the bound has a higher-order dependence on the depth and width and an additional factor (cid:15) .
We provide experiments to analyze the relationship between the generalization gap and the adversarial Rademacher complexity.
We show that one of the reasons why adversarial training cannot generalize well is the large weight norms of an adversarially-trained model.
P RELIMINARIES 2. G ENERALIZATION G AP AND R ADEMACHER C OMPLEXITY
Generalization Gap.
We start from the classical machine learning framework.
Let F be the hypothesis class (e.g. Linear functions, Neural networks).
The goal of the learning problem is to ﬁnd f ∈ F to minimize the population risk R ( f ) =
E ( x ,y ) ∼D
[ (cid:96) ( f ( x ) , y )] , where D is the true distribution, (cid:96) ( · ) is the loss function.
Since D is unknown, we minimize the empirical risk in practice.
Given m i.i.d samples S
= { ( x 1 , y 1 ) , · · · , ( x m , y m ) } , the empirical risk is R m ( f ) =
1 m (cid:80) mi
[Equation] [Equation]
i
[Equation]
, y i
[Equation]
The generalization gap or generalization error is deﬁned as follow:
[Equation]
Rademacher Complexity.
A classical measure of the generalization error is Rademacher complexity (Bartlett & Mendelson (2002)).
Given the hypothesis class H , the (empirical) Rademacher complexity is deﬁned as
[Equation]
where σ i are i.i.d Rademacher random variables, i.e. σ i equals to 1 or − 1 with equal probability.
Deﬁne the function class (cid:96) F = { (cid:96) ( f ( x ) , y ) | f ∈ F} , we have the following generalization bound.
Proposition 1.
(Mohri et al. ; Bartlett & Mendelson (2002)) Suppose that the range of the loss function (cid:96) ( f ( x ) , y ) is [0 , C ] .
Then, for any δ ∈ (0 , 1) , with probability at least 1 − δ , the following holds for all f ∈ F , [Equation]
[Equation]
R OBUST G ENERALIZATION G AP AND A DVERSARIAL R ADEMACHER C OMPLEXITY
Robust Generalization Gap.
Let ˜ (cid:96) ( f ( x ) , y ) :
= max (cid:107) x [Equation]
− x (cid:107) p ≤ (cid:15) (cid:96) ( f ( x (cid:48) ) , y ) be the adversarial loss.
The adversarial population risk and the adversarial empirical risk are
[Equation]
and
[Equation]
respectively.
In this paper we consider general (cid:96) p attacks for p ≥ 1 .
The robust generalization gap is deﬁned as follow:
[Equation]
Let the adversarial hypothesis class be ˜ (cid:96) F = { ˜ (cid:96) ( f ( x ) , y ) | f ∈ F} , according to Proposition 1, we have the following adversarial generalization bound.
Proposition 2.
(Yin et al.
Suppose that the range of the loss function ˜ (cid:96) ( f ( x ) , y ) is [0 , C ] .
Then, for any δ ∈ (0 , 1) , with probability at least 1 − δ , the following holds for all f ∈ F ,
[Equation]
Binary Classiﬁcation.
We ﬁrst discuss the binary classiﬁcation case, then we discuss the extension to the multi-class classiﬁcation case in section 5.
Following (Yin et al.
; Awasthi et al.
, we assume that the loss function can be written as (cid:96) ( f ( x ) , y ) =
φ ( yf ( x )) where φ is a non-increasing function.
Then
[Equation]
Assume that the function φ is L φ -Lipschitz, by Talagrand’s Lemma (Ledoux & Talagrand (2013)), we have R S (˜ (cid:96) F ) ≤ L φ R S ( ˜ F ) , where we deﬁne the adversarial function class as
[Equation]
Adversarial Rademacher Complexity.
We deﬁne R S ( ˜ F ) as adversarial Rademacher complexity.
Our goal is to give upper bounds for adversarial Rademacher complexity.
Then, it induces the guarantee of the robust generalization gap.
Hypothesis Class.
We consider depth- d , width-
h fully-connected neural networks,
[Equation]
where ρ ( · ) is an element-wise L ρ -Lipschitz activation function, W l are h l × h l − 1 matrices, for l = 1 , · · · , d .
We have h d = 1 and h
= q is the dimension of the data x .
Let h = max { h 0 , · · · , h d } be the width of the neural networks.
Convolution neural networks are also included in this hypothesis class because convolution layer can be viewed as a special form of fully-connected layer.
Denote the ( a, b )-group norm (cid:107) W (cid:107) a,b as the a -norm of the b -norm of the rows of W .
We consider two cases,
Frobenius norm and (cid:107) · (cid:107) 1 , ∞ -norm in equation (2).
Additionally, we assume that (cid:107) X (cid:107) p, ∞ =
B .
2. R ADEMACHER C OMPLEXITY AND C OVERING N UMBER
Covering Number.
Our solution for adversarial Rademacher complexity is based on the covering number.
We ﬁrst provide the deﬁnition of covering number.
Deﬁnition 1 ( ε -cover) .
Let ε > 0 and ( V , d ( · , · ) ) be a metric space, where d ( · , · ) is a (pseudo)-metric.
C ⊂ V is an ε -cover of V , if for any v ∈ V , there exists v (cid:48)
∈ C s.t.
d ( v, v (cid:48) )
≤ ε .
Deﬁne the smallest |C| as ε -covering number of V and denote as N ( V, d ( · , · ) , ε ) .
Next, we deﬁne the ε -covering number of a function class F .
Given the sample dataset S = { ( x 1 , y 1 ) , · · · , ( x m , y m ) } with x i ∈ R d , let (cid:107) f (cid:107) 2 S
= 1 m (cid:80) mi =1 f ( x i , y i ) 2 be a pseudometric of
F .
Deﬁne the ε -covering number of F be N ( F , (cid:107) · (cid:107) S , ε ) .
Let D be the diameter of F with
D = 2 max f ∈F (cid:107) f
(cid:107) S .
Proposition 3 (Dudley’s integral) .
The Rademacher complexity R ( F ) satisﬁy
[Equation]
The proof of Dudley’s integral can be found in statistic textbooks (e.g. (Wainwright (2019))).
Based on this, we can bound the covering number of the function class F to give an upper bound of the
Rademacher complexity.
R ELATED W ORK
Adversarial Attacks and Defense.
Starting from the work of (Szegedy et al.
, it has now been well known that deep neural networks trained via standard gradient descent based algorithms are highly susceptible to imperceptible corruptions to the input data (Goodfellow et al.
; Chen et al.
; Carlini & Wagner (2017); Madry et al.
This has led to a series of work aimed at training neural networks robust to such perturbations (Wu et al.
; Gowal et al.
; Wu et al.
(2020)) and works aimed at designing more sophisticated attacks to attack the classiﬁers (Athalye et al.
; Tramer et al.
; Chen et al.
Adversarial Generalization.
The work of (Schmidt et al.
; Raghunathan et al.
; Zhai et al.
(2019)) have shown that in some scenarios achieving adversarial generalization requires more data.
The work of (Attias et al.
; Montasser et al.
(2019)) explains generalization in adversarial settings using VC-dimension.
Cullina et al.
(2018) studies PAC-learning guarantees in the adversarial setting via VC-dimension.
VC-dimension usually depends on the number of parameters in the model, while Rademacher complexity usually depends on the weight matrices.
Rademacher complexity usually provides tighter generalization bounds (Bartlett (1998)).
Neyshabur et al.
(2017b) uses a pac-bayesian approach to provide a generalization bound for neural networks.
Sinha et al.
(2017) study the generalization of an adversarial training algorithm in terms of distributional robustness.
The work of (Xing et al.
(2021a;b); Javanmard et al.
(2020)) study the generalization properties in the setting of linear regression.
Gaussian mixture models are used to analyze adversarial generalization (Taheri et al.
; Javanmard et al.
Dan et al.
The work of (Allen-Zhu & Li (2020)) explains adversarial generalization through the lens of feature puriﬁcation.
Adversarial Rademacher Complexity.
Researchers have analyzed adversarial Rademacher complexity in linear and two-layers neural networks cases.
In linear cases, the upper bounds can be directly derived by deﬁnition (Khim & Loh (2018); Yin et al.
In two-layers neural networks cases, an upper bound is derived using Massart’s Lemma (Awasthi et al.
It seems that these proofs cannot be extended to multi-layers cases.
Moreover, based on the deﬁnition of adversarial function class ˜ F in equation (1), the candidate functions are not composition functions, but with an inf operation in front of the neural networks.
Then, the induction on layers seems not applicable in calculating adversarial Rademacher complexity for deep neural networks.
The works of (Khim &
Loh (2018)) and (Gao & Wang (2021)) indicate the difﬁculty of analyzing adversarial Rademacher complexity.
They analyze other variants of adversarial Rademacher complexity of DNNs.
The ﬁrst one introduce tree transformation, but it overestimates the adversarial loss.
The second one considers fast gradient sign methods (FGSM) adversarial examples, but it also requires additional assumption on the gradient.
In Appendix B, we provide the details of the above bounds and discuss why these methods seem not applicable in multi-layers cases.
We also provide a comparison of adversarial generalization in Rademacher complexity framework and other frameworks.
In the next section, we provide our solution of adversarial Rademacher complexity based on covering number and analyze each factor in the upper bound.
O UR S OLUTION OF A DVERSARIAL R ADEMACHER C OMPLEXITY
The following Theorem states an upper bound of adversarial Rademacher complexity in the Frobenius norm cases.
Theorem 1 (Frobenius Norm Bound) .
Given the function class F in equation (2) under Frobbenius Norm, and the corresponding adversarial function class ˜ F in equation (1).
The adversarial
Rademacher complexity of deep neural networks R S ( ˜ F ) satisﬁes
[Equation]
By assuming that L ρ = 1 , p ≤ 2 , (cid:107) X (cid:107) p, ∞ =
B , and h = max { h 0 , · · · , h d } , we have
[Equation]
Because of the inf operation of the function in ˜ F , we cannot peel off the layers or calculate the covering number by induction on layers.
Our proof is based on calculating the covering number of ˜ F directly.
Below we sketch the proof.
The completed proof is provided in Appendix A.
Step 1: Diameter of ˜ F .
We ﬁrst calculate the diameter of ˜ F .
We have
[Equation]
Step 2: Distance to ˜ F c .
Let C l be δ l -covers of {(cid:107) W l (cid:107) F ≤ M l } , l = 1 , 2 , · · · , d .
Let
F c = { f c : x → W cd ρ ( W cd − 1 ρ ( · · · ρ ( W c 1 x ) · · · )) , W cl ∈ C l , l = 1 , 2 · · · , d } and ˜ F c = { ˜ f : ( x , y ) → inf (cid:107) x − x (cid:48) (cid:107) p ≤ (cid:15) yf ( x (cid:48) )
| f ∈ F c } .
For all ˜ f ∈ ˜ F , we need to ﬁnd the smallest distance to ˜ F c , i.e. we need to calculate the
[Equation]
∀ ( x , y ) ∈ D , given f and f c , let x ∗ = arg inf (cid:107) x − x (cid:48) (cid:107) p yf ( x (cid:48) ) and x c = arg inf (cid:107) x − x (cid:48) (cid:107) p yf c ( x (cid:48) ) .
Let z = (cid:26) x c if f ( x ∗ ) ≥ f c ( x c ) x if f ( x ∗ ) <
f c ( x c ) and g ab ( z ) =
W b ρ ( · · · W a +1 ρ ( W ca · · · ρ ( W c 1 z ) · · · ))) .
Then | f ( z ) − f c ( z ) | =
| g 0 d ( z ) − g dd ( z ) | ≤ | g 0 d ( z ) − g 1 d ( z ) | + · · · + | g d − 1 d ( z ) − g dd ( z ) | ≤ d (cid:88) l =
Dδ l 2 M l .
Let δ l = 2 M l ε/dD , l = 1 , · · · , d , we have max ˜ f
[Equation]
Step 3: Covering Number of ˜ F .
Then, We can calculate the ε -covering number N ( ˜ F , (cid:107) · (cid:107) S , ε ) .
Because ˜ F c is a ε -cover of ˜ F .
The cardinality of ˜ F c is
[Equation]
Step 4: Integration.
By Dudley’s integral, we obtain the bound in Theorem 1.
Remark: Step 2 is the critical step in the proof.
In words, if we calculate the covering number of the class of d -layers neural nets directly, we only need to deﬁne the optimal adversarial example one time.
Then we can calculate the other things using this adversarial example.
In contrast, if we want to do it layer by layer, the optimal adversarial example will be changed when we add or peel off a layer.
This is why the induction methods fail in adversarial settings.
Theorem 2 ( (cid:107) · (cid:107) 1 , ∞ -Norm Bound) .
Given the function class F in equation (2) under (cid:107) · (cid:107) 1 , ∞ -norm, and the corresponding adversarial function class ˜ F in equation (1).
The adversarial Rademacher complexity of deep neural networks R S ( ˜ F ) satisﬁes
[Equation]
In the case of (cid:107) · (cid:107) 1 , ∞ -norm, the bound is similar to the bound in the Frobenius norm case except the term max { 1 , q 1 / 2 − 1 /p } .
Therefore, for all p ≥ 1 , the (cid:107) · (cid:107) 1 , ∞ -norm bound have the same order in equation (4).
Theorem 3 (Lower Bound) .
Given the function class of DNNs F in equation (2), and the corresponding adversarial function class ˜ F in equation (1).
Exist sample dataset S , s.t.
the adversarial
Rademacher complexity of deep neural networks R S ( ˜ F ) satisﬁes
[Equation]
[Equation]
The proof of the above Theorem is based on constructing a scalar network and is provided in Appendix
A.
The gap between the upper bound and the lower bound is the dependence on depth- d and width- h , h √ d log d .
In the next section, we extend the adversarial Rademacher complexity to the Multi-class classiﬁcation cases.
[Equation]
ARGIN B OUNDS FOR M ULTI -C LASS C LASSIFICATION 5. S ETTING FOR M ULTI -C LASS C LASSIFICATION
The setting for multi-class classiﬁcation follows (Bartlett & Mendelson (2002)).
In a K-class classiﬁcation problem, let Y =
1 , 2 , · · · , K } .
The functions in the hypothesis class F map X to R K , the k -th output of f is the score of f ( x ) assigned to the k -th class.
Deﬁne the margin operator M ( f ( x ) , y ) =
[ f ( x )] y − max y (cid:48)
(cid:54) =
y [ f ( x )] y (cid:48) .
The function makes a correct prediction if and only if M ( f ( x ) , y ) > 0 .
We consider a particular loss function (cid:96) ( f ( x ) , y ) =
φ γ ( M ( f ( x ) , y )) , where γ > 0 and φ γ : R → [0 , 1] is the ramp loss:
[Equation]
The loss function (cid:96) ( f ( x ) , y ) satisﬁes:
[Equation]
Deﬁne the function class (cid:96) F := { ( x , y )
(cid:55)→ φ γ ( M ( f ( x ) , y )) : f ∈ F} .
Since φ γ ( t ) ∈ [0 , 1] and φ γ ( · ) is 1 /γ -Lipschitz, by combining (8) with Theorem 1, we can obtain the following direct corollary as the generalization bound in the multi-class classiﬁcation.
Corollary 1 (Mohri et al.
Consider the above multi-class classiﬁcation setting.
For any ﬁxed γ > 0 , we have with probability at least 1 − δ , for all f ∈ F ,
[Equation]
[Equation]
In adversarial training, let B p x ( (cid:15) )
= { x (cid:48) :
(cid:107) x (cid:48) − x (cid:107) p ≤ (cid:15) } and we deﬁne the adversarial function class ˜ (cid:96) F := { ( x , y )
(cid:55)→ max x (cid:48) ∈
B p x ( (cid:15) ) (cid:96) ( f ( x (cid:48) ) , y ) : f ∈ F} .
We have
Corollary 2 (Yin et al.
Consider the above adversarial multi-class classiﬁcation setting.
For any ﬁxed γ > 0 , we have with probability at least 1 − δ , for all f ∈ F ,
[Equation]
5. A DVERSARIAL R ADEMACHER C OMPLEXITY
Under the multi-class setting, we have the following bound for adversarial Rademacher complexity.
Theorem 4.
Given the function class F in equation (2) under Frobbenius Norm, and the corresponding adversarial function class ˜ F in equation (1).
The adversarial Rademacher complexity of deep neural networks R S (˜ (cid:96) F ) satisﬁes
[Equation]
The (cid:107) · (cid:107) 1 , ∞ -norm bound is similar, except the term max { 1 , q 12 − 1 p } .
Below we sketch the proof.
Step 1: Let ˜ F k = { ( x , y ) → inf (cid:107) x (cid:48)
− x (cid:107)≤ (cid:15) ([ f ( x (cid:48) )
y − [ f ( x (cid:48) )] k ) , f ∈ F} , then R S (˜ (cid:96) F ) ≤
K R S (˜ (cid:96) F k ) .
Step 2:
By the Lipschitz property of φ γ ( · ) , R S (˜ (cid:96) F k ) ≤ 1 γ R S ( F k ) .
Step 3: The calculation of R S ( ˜ F k ) follows the binary case.
5. C OMPARISON OF THE B OUNDS
Now, we compare the difference between the bounds for (standard) Rademacher complexity and adversarial Rademacher complexity.
We have shown that
[Equation]
R S
[Equation]
˜
[Equation]
F
[Equation]
O
[Equation] [Equation]
where we use the upper bound of R S ( F ) in (Golowich et al.
Algorithm-Independent Factors.
In the two bounds, the algorithm-independent factors include
Sample size B , perturbation intensity (cid:15) , depth- d , and width- h .
To simplify the notations, we let
C std = B √ d and C adv
=
( B + (cid:15) )
h √ d log d be the constants in standard and adversarial Rademacher complexity, respectively.
We simply have C adv > C std .
Algorithm-Dependent Factors.
In the two bounds, the margins γ and the product of the matrix norms (cid:81) d l =1 (cid:107) W l (cid:107) depend on the training algorithms.
To simplify the notations, we deﬁne W std :=
(cid:81)
dl
=1 (cid:107) W l (cid:107) /γ
if the training algorithm is standard training.
Correspondingly, let
W adv :=
(cid:81)
dl
=1 (cid:107) W l (cid:107) /γ
if the training algorithm is adversarial training.
In the next section, we conduct experiments to show that W adv > W std .
Notation of generalization gaps.
In the next section, we use E ( · ) and ˜ E ( · ) to denote the standard and robust generalization gap.
We use f std and f adv to denote the standard- and adversarially-trained model.
Our goal is to understand why the robust generalization gap of an adversarial training model is large, which is quite different from the standard generalization gap of a standard-trained model, i.e., we want to analyze why ˜ E ( f adv ) >
E ( f std ) .
Based on the standard and adversarial Rademacher complexity bounds, it is suggested that
[Equation]
To analyze the individual effect of factors C adv and W adv , we further introduce two kinds of generalization gaps, the robust generalization gap of a standard-trained model ( ˜ E ( f std ) ) and the standard generalization gap of an adversarially-trained model ( E ( f adv ) ).
The standard and adversarial
Rademacher complexity suggest that
[Equation]
E XPERIMENT
As we discuss in the previous section, the product of weight norm (cid:81) dl =1 (cid:107) W l (cid:107) and the margin γ are algorithm-dependent factors controlling the generalization gap.
We provide experiments comparing the difference between these terms in standard and adversarial settings.
Since the bounds also hold for convolution neural networks, we consider the experiments of training VGG networks (Simonyan & Zisserman (2014)) on CIFAR-10 (Krizhevsky et al.
We use the experiments on VGGto illustrate the results.
Other experiments are provided in Appendix C.
[Figure]
Training Settings.
For both standard and adversarial training, we use the stochastic gradient descent (SGD) optimizer, along with a learning rate schedule, which is 0.1 over the ﬁrst 100 epochs, down to 0.01 over the following 50 epochs, and ﬁnally be 0.001 in the last 50 epochs.
For adversarial settings, we adopt the (cid:96) ∞
PGD adversarial training (Madry et al.
The perturbation intensity is set to be 8 / 255 .
We set the number of steps as 20 and further increase it to 40 in the testing phase.
For the stepsize in the inner maximization, we set it as 2/255.
In Corollary 2, we need to use the optimal adversarial examples to calculate the margin and the robust generalization gap, but it is unknown in practice.
We use the PGD adversarial examples as substitutes.
Calculation of Margins.
We adopt the setting in (Neyshabur et al.
(2017a)).
In standard training, we set the margin over training set to be 5 th -percentile of the margins of the data points in S .
i.e.
Prc 5 { f ( x i )
[ y i ] − max y (cid:54) =
y
i f ( x )[ y ] | ( x i , y i ) ∈ S } .
In adversarial settings, we set the margin over training set to be 5 th -percentile of the margins of the PGD-adversarial examples of S .
The choice of 5 th -percentile is because the training accuracy is 100% in all the experiments.
We provide ablation studies about the percentile in the Appendix C.
Standard and Robust Generalization Gap.
In Figure 3 (a) and (b), we plot the standard and robust generalization gap of both standard-trained and adversarially-trained models.
We use the results using 50000 training samples to discuss the experiments.
Firstly, in Figure 3 (a), we can see that E ( f std ) is small (=10.45%).
On the other hand, an adversarial-trained model has a larger standard generalization gap ( E ( f adv ) =26.34%).
It is a widely observed phenomenon that adversarial training hurts standard generalization.
One reason is that adversarial training overﬁts the adversarial examples and performs worse on the original examples.
Secondly, in Figure 3 (b), the robust generalization gap of a standard-trained model is very small ( ˜ E ( f std ) = 0 ).
It is because the standard-trained model can easily be attacked on both the training set and test set.
Then, the robust training accuracy and test accuracy are closed to 0%.
Therefore, the robust generalization gap is also 0.
On the contrary, ˜
E ( f adv ) =58.90%, i.e. the adversarial generalization is bad.
This is also observed in the previous studies, and we aim to discuss the reasons.
Adversarially-trained Models Have Larger Weight Norms, i.e. W adv > W std .
In Figure 3 (c), we can see that the (cid:81) dl
=1 (cid:107) W l (cid:107) F of adversarial training is much larger than the (cid:81) dl
=1 (cid:107) W l (cid:107) F of standard training.
In (d), the (cid:81) dl
=1 (cid:107) W l (cid:107) F is divided by γ , we can see that the Figures is similar to the Figures in (c), W adv is larger than W std .
One of the reasons why W adv > W std is neural networks need more capacity to ﬁt the adversarial examples.
[Table] [Table]
˜ E ( f std ) =
0% is a degenerated case.
In Table 1, we show the training and test errors for all kinds of generalization gaps.
We can see that the robust training error for a standard-trained model is equal to 100%.
Since the model does not ﬁt any adversarial examples in the training set, there is nothing to generalize to the adversarial examples in the test set.
The generalization gap becomes meaningless.
And the Rademacher complexity bound ˜ E
( f std ) ≤ O ( C adv W std / √ m ) becomes a trivial bound.
In the other three cases, the training errors are all ≈ 0%.
The generalization gaps are meaningful.
We aim to analyze why ˜ E ( f adv ) >
E ( f std ) by analyzing ˜ E ( f adv ) >
E ( f adv ) >
E ( f std ) .
The effect of C adv .
We ﬁrst compare the difference between ˜ E ( f adv ) and E ( f adv ) .
We can see that ˜ E ( f adv ) = 58 .
90% > E ( f adv )
= 26 .
For an adversarially-trained model, the robust generalization gap is larger than the standard generalization gap.
If we use the bounds of adversarial and standard Rademacher complexity as approximations of the robust and standard generalization gap, i.e., ˜ E ( f adv ) ∝ C adv W adv and E ( f adv ) ∝ C std W adv , ˜ E ( f adv ) > E ( f adv ) can be explained by
C adv > C std since W adv are the same in the two bounds.
The effect of W adv .
Similarly, we compare the difference between E ( f adv ) and E ( f std ) .
We can see that E ( f adv ) = 26 .
34% > E ( f std ) = 10 .
This is a widely observed phenomenon that adversarial training hurts standard generalization.
It can also be explained by the Rademacher bounds.
If we use the bounds of standard Rademacher complexity as approximations, i.e., E ( f adv ) ∝
C std W adv and E ( f std ) ∝ C std W std , ˜ E ( f adv ) > E ( f adv ) can be explained by W adv > W std .
In summary, we can use a simple formula to explain why ˜ E ( f adv ) > E ( f adv ) >
E ( f std ) through the lens of Rademacher complexity.
That is
[Equation]
The difﬁculty of adversarial generalization comes from two parts, the constant C adv and the weight norms W adv .
The ﬁrst part C adv is independent of the algorithms.
It comes from the minimax problem of adversarial training itself, and it cannot be avoided.
The second part W adv depends on the algorithms.
Therefore, the product of the weight norms is an important factor for the robust generalization of adversarial training.
Ablation Studies.
We provide other ablation studies in Appendix C. First, we consider different
VGG architecture and give the experiments on VGG-11, 13, and 16.
Secondly, We consider different percentile of the margins of the training dataset.
Thirdly, we provide the experiments on (cid:107)·(cid:107) 1 , ∞ -norm.
We can see that the (cid:107) · (cid:107) 1 , ∞ -norm bound has a larger magnitude than the Frobenius norm bound.
Then, we provide the experiments on CIFAR-100.
Finally, the large weight norms suggest adding a regularization term on the weights during training.
We provide experiments with and without weight decay and see that the one with weight decay has a smaller generalization gap and a smaller (cid:81) dl
=1 (cid:107) W l (cid:107) F /γ .
These experiments suggest the strong relationship between robust generalization gap and the product of weight norms.
C ONCLUSION
In this paper, we ﬁrst provide upper bounds for the adversarial Rademacher complexity of deep neural networks.
Then, we experimentally investigate these bounds and show that the product of weight norms is a key factor explaining why adversarial training cannot generalize well.
We think our results will motivate more theoretical research to understand adversarial training and empirical research to improve the generalization of adversarial training.
A P ROOF OF THE T HEOREMS A. P ROOF OF T HEOREM
Theorem 1 (Frobenius Norm Bound) .
Given the function class F in equation (2) under Frobbenius Norm, and the corresponding adversarial function class ˜ F in equation (1).
The adversarial
Rademacher complexity of deep neural networks R S ( ˜ F ) satisﬁes
[Equation]
By assuming that L ρ = 1 , p ≤ 2 , (cid:107) X (cid:107) p, ∞ ≤ B , and h = max { h 0 , · · · , h d } , we have
[Equation]
Because of the inf operation of the function in ˜ F , we cannot peel off the layers or calculate the covering number by induction on layers.
Our proof is based on calculating the covering number of ˜ F directly.
Before we provide the proof, we ﬁrst introduce the following lemma.
Lemma 1 (Covering number of norm-balls) .
Let B be a (cid:96) p norm ball with radius W .
Let d ( x 1 , x 2 )
= (cid:107) x 1 − x 2 (cid:107) p .
Deﬁne the ε -covering number of B as N ( B , d ( · , · ) , ε ) , we have
[Equation]
In the case of Frobenius norm ball of m × n matrices, we have the dimension q = m × n and
N ( B , (cid:107) · (cid:107) F ,
[Equation] [Equation]
Lemma 2.
if x ∗ i ∈ { x (cid:48)
i |(cid:107) x i − x (cid:48)
i (cid:107) p ≤ (cid:15) } , we have
[Equation]
Proof: If p ≥ r ∗ , by Holder’s inequality with 1 /r ∗ = 1 /p + 1 /s ,
[Equation]
Equality holds when all the entries are equal.
If p < r ∗ , we have
[Equation]
Equality holds when one of the entries of θ equals to one and the others equal to zero.
Then
[Equation]
Lemma 3.
Let A be a m × n matrix and b be a n -dimension vector, we have
[Equation]
Proof: let A i be the rows of A , i = 1 · · · m , we have
[Equation]
Step 1: Diameter of ˜ F .
We ﬁrst calculate the diameter of ˜ F .
∀ f ∈ F , given ( x i , y i ) , let x ∗ i =
inf
(cid:107) x
i − x (cid:48)
i (cid:107) p ≤ (cid:15)
p yf ( x (cid:48) i ) , and we let x li be the output of x ∗ i pass through the ﬁrst to the l − 1 layer, we have
[Equation]
where inequality (i) is because of Lemma 3, inequality (ii) is because of the Lipschitz propertiy of activation function ρ ( · ) , inequality (iii) is because of Lemma 2.
Therefore, we have
[Equation]
=
[Equation]
m
[Equation]
Step 2: Distance to ˜ F c .
Let C l be δ l -covers of {(cid:107) W l (cid:107) F ≤ M l } , l = 1 , 2 , · · · , d .
Let
F c = { f c : x
[Equation]
W cd
[Equation]
( W cd
[Equation]
( · · · ρ ( W
[Equation]
1 x ) · · ·
[Equation]
) , W cl ∈ C l , l = 1 , 2 · · · , d } and ˜ F c = { ˜ f : ( x , y ) → inf
[Equation]
For all ˜ f ∈ ˜ F , we need to ﬁnd the smallest distance to ˜ F c , i.e. we need to calculate the
[Equation]
∀ ( x i , y i ) , i = 1 , · · · , n , given ˜ f and ˜ f
c with (cid:107) W l − W cl (cid:107) F ≤ δ l , l = 1 , · · · , d , consider | ˜ f
( x i , y i ) − ˜ f
c ( x i , y i ) | = |
[Equation]
i
[Equation]
y i
[Equation]
− inf
[Equation]
) |
Let x ∗ i = arg inf (cid:107) x
i − x (cid:48)
i (cid:107) p y i f
( x (cid:48) i ) , and x ci = arg inf (cid:107) x
i − x (cid:48)
i (cid:107) p y i f c ( x (cid:48) i ) , we have | ˜ f
( x i , y i )
[Equation]
˜ f
c ( x i , y i )
[Equation]
= | y
i f ( x ∗ i )
[Equation]
y i f c ( x ci ) |
[Equation]
| f
( x ∗ i )
[Equation]
f c ( x
[Equation]
| .
Let z i =
[Equation]
if
[Equation]
x
[Equation]
if
[Equation]
Then,
[Equation]
Deﬁne g ab ( · ) as
[Equation]
In words, for the layers b ≥ l > a in g ab ( · ) , the weight is W l , for the layers a ≥ l ≥ 1 in g ab ( · ) , the weight is W cl .
Then we have f ( z i ) =
g 0 d
( z i ) , f ( z i ) =
g Ld ( z i ) .
We can decompose
[Equation]
To bound the gap | f
( z i ) − f c
( z i ) | , we ﬁrst calculate | g l − 1 d
[Equation]
z i ) − g ld
( z i ) | for l = 1 , · · · , d .
[Equation]
where (i) is due to Lemma 3, (ii) is due to the bound of (cid:107) W L (cid:107) and the Lipschitz of ρ ( · ) , (iii) is because of the deﬁnition of g ab ( z ) .
Then
[Equation]
where inequality (i) is due to Lemma 3, inequality (ii) is due to Lemma 3 the assumption that (cid:107) W l − W cl (cid:107) F ≤ δ l .
It is lefted to bound (cid:107) ρ ( g l − 1 l − 1 ( z i )))
(cid:107) ∞ , we have
[Equation]
combining inequalities (12) and (13), we have
[Equation]
Therefore, combining inequalities (15) and (14), we have
[Equation]
Then
[Equation]
Let δ l = 2 M l ε/dD , l = 1 , · · · , d , we have
[Equation]
Step 3: Covering Number of ˜ F .
We then calculate the ε -covering number N ( ˜ F , (cid:107)·(cid:107) S , ε ) .
Because ˜ F c is a ε -cover of ˜ F .
The cardinality of ˜ F c is
[Equation]
where inequality (i)) is due to Lemma 1.
Step 4, Integration.
By Dudley’s integral, we have
[Equation]
m (cid:90) 1 / (cid:112) log(3 d/ 2 ε )
dε.
Integration by part, we have
[Equation]
Plug equation (17) to equation (16), we have
[Equation]
A. P ROOF OF T HEOREM
Theorem 2 ( ( (cid:107)·(cid:107) 1 , ∞ -Norm Bound) .
Given the function class F in equation (1) under (cid:107)·(cid:107) 1 , ∞ -norm, and the corresponding adversarial function class ˜ F in equation (2).
The adversarial Rademacher complexity of deep neural networks R S ( ˜ F ) satisﬁes
[Equation]
The proof is mimilar to the proof of the Frobenius norm bound.
We ﬁrst introduce the following inequality.
Lemma 4.
Let A be a m × n matrix and b be a n -dimension vector, we have
[Equation]
Proof: let A i be the rows of A , i = 1 · · · m , we have
[Equation]
Step 1: Diameter of ˜ F .
We ﬁrst calculate the diameter of ˜ F .
∀ f ∈ F , given ( x i , y i ) , let x ∗ i =
inf
(cid:107) x
i − x (cid:48)
i (cid:107) p ≤ (cid:15)
p yf ( x (cid:48) i ) , and we let x li be the output of x ∗ i pass through the ﬁrst to the l − 1 layer, we have
[Equation]
where inequality (i) is because of Lemma 4, inequality (ii) is because of the Lipschitz propertiy of activation function ρ ( · ) , inequality (iii) is because of Lemma 2.
Therefore, we have
[Equation]
Step 2: Distance to ˜ F c .
Let C il be δ l -covers of {(cid:107) W il (cid:107) 1 ≤ M l } , l = 1 , 2 , · · · , d , i = 1 , · · · , h l , where W il is the i th row of W il .
Let
F
[Equation]
, d } and ˜ F c = { ˜ f
[Equation]
( x , y ) → inf
[Equation]
x
[Equation]
f ∈ F c } .
For all ˜ f ∈ ˜ F , we need to ﬁnd the smallest distance to ˜ F c , i.e. we need to calculate the
[Equation]
∀ ( x i , y i ) , i = 1 , · · · , n , given ˜ f and ˜ f c with | W il − W cil | ≤ δ l , i = 1 , · · · , h l , l = 1 , · · · , d , we have (cid:107) W l − W cl (cid:107) 1 , ∞ ≤ δ l .
By the same argument as the step 2 of the proof o Theorem 3, we have
[Equation]
Let δ l = 2 M l ε/dD , l = 1 , · · · , d , we have
[Equation]
Step 3: Covering Number of ˜ F .
We then calculate the ε -covering number N ( ˜ F , (cid:107)·(cid:107) S , ε ) .
Because ˜ F c is a ε -cover of ˜ F .
The cardinality of ˜ F c is
[Equation]
where inequality (i)) is due to Lemma 1.
Step 4, Integration.
By the same argument as the step 4 of the proof o Theorem 1, integration by part, we have
[Equation]
A. P ROOF OF T HEOREM
Theorem 3 (Lower Bound) .
Given the function class F in equation (2), and the coressponding adversarial function class ˜ F in equation (1).
Exist sample dataset S , s
[Equation]
t. the adversarial Rademacher complexity of deep neural networks R S ( ˜ F ) satisﬁes
[Equation]
The proof of the above Theorem is based on constructing a scalar network.
By the deﬁnition of
Rademacher complexity, if H (cid:48) is a subset of H , we have
[Equation]
Therefore
[Equation]
it is enough to lower bound the complexity of ˜ F (cid:48) in a particular distribution D , where ˜ F (cid:48) is a subset of ˜ F .
Let
[Equation]
We ﬁrst prove that ˜ F (cid:48) is a subset of ˜ F .
In ˜ F , we let the activation function ρ ( · ) be a identity mapping.
Let
[Equation]
Then, we have (cid:107) W l (cid:107) ≤ M l and ˜ F with additional constraint in equation (18) reduce to ˜ F (cid:48) .
In other words, ˜ F (cid:48) is a subset of ˜ F .
It turns out that we need to lower bound the adversarial Rademacher complexity of linear hypothesis.
The results are given by the work of (Yin et al.
; Awasthi et al.
Below we state the result.
Proposition 4.
Given the function class G = { x → yw T x | w ∈ R q , (cid:107) w (cid:107) r ≤ W } and ˜ G = { x → inf
[Equation] [Equation]
≤ W } , the adversarial Rademacher complexity R S ( ˜ G ) satisﬁes
[Equation]
Since the standard Rademacher complexity
[Equation]
let (cid:107) x
i (cid:107) =
B with equal entries for i = 1 , · · · , m , by Lemma 2 we have
[Equation]
By Khintchine’s inequality, we know that there exists a universal constant c > 0 such that
[Equation]
Then, we have
[Equation]
Therefore,
[Equation]
Let W = (cid:81) dl
=1 M l
[Equation]
we have
[Equation]
where r = 2 for frobenius norm bound and r = 1 for (cid:107) · (cid:107) 1 , ∞ -norm bound.
A
[Equation]
P ROOF OF T HEOREM
Theorem 4.
Given the function class F in equation (2) under Frobbenius Norm, and the corresponding adversarial function class ˜ F in equation (1).
The adversarial Rademacher complexity of deep neural networks R S (˜ (cid:96) F ) satisﬁes
[Equation]
The (1 , ∞ ) -norm bound is similar, except the term max { 1 , q 12 − 1 p } .
Proof: Firstly
[Equation]
we have
[Equation]
Deﬁne
[Equation]
we now prove that
[Equation]
If
[Equation]
we have
[Equation]
If
[Equation]
we have
[Equation]
Therefore
[Equation]
we have
[Equation]
Deﬁne
[Equation]
, we have
[Equation]
19) where inequality (i) is the Lemma 9.1 of (Mohri et al.
[Equation]
inequality (ii) is due to the Lipschitz property of φ γ ( · ) .
Now
[Equation]
deﬁne f k ( x , y )
=
inf
[Equation]
x )] k ) , we have h k ( x , y )
=
f k ( x , y )
+ γ 1 ( y = k ) .
Deﬁne the function class
[Equation]
We have
[Equation]
R
[Equation]
Finally, we need to bound the Rademacher complexity of R ( F k ) .
Notice that [ f ( x )]
y − [
[Equation]
x )]
[Equation]
W
[Equation]
−
[Equation] [Equation]
x
[Equation]
· · ))) , and we have (cid:107) W y d − W kd (cid:107) F ≤ 2 M l .
By Theorem 3 (the results in binary classiﬁcation case), we have
[Equation] [Equation]
Combining inequalities (20) and (19), we obtain that
[Equation]
B D ISCUSSION ON E XISTING M ETHODS FOR R ADEMACHER C OMPLEXITY
In this section, we discuss the related work, discuss the existing methods in calculating Rademacher complexity, and identify the difﬁculty of analyzing adversarial Rademacher complexity.
B. E XISTING M ETHODS
‘Layer Peeling’ Bounds.
The main idea of calculating the Rademacher complexity of multi-layers neural networks is the ‘peeling off’ technique (Neyshabur et al.
We denote g ◦ F as the function class { g ◦ f | f ∈ F} .
By Talagrand’s Lemma, we have R S ( g ◦ f ) ≤ L g R S ( F ) .
Based on this property, we can obtain R S ( F l ) ≤ 2 L ρ M l R S ( F l − 1 ) , where F l is the function class of l -layers neural networks.
Since the Rdemacher complexity of linear function class is bounded by
O ( BM 1 / √ m ) , we can get the upper bound O ( B 2 d L d − 1 ρ
[Equation]
√ m ) by induction.
We can remove the L ρ by assuming that L ρ = 1 (e.g. Relu activation function).
Golowich et al.
(2018) improves the dependence on depth- d from 2 d to √ d .
The main idea is to rewrite the Rademacher complexity E σ [ · ] as E σ exp ln
Then, we can peel off the layer inside the ln( · ) function and the 2 d now appears inside the ln( · ) .
Covering Number Bounds.
Bartlett et al.
(2017) uses a covering numbers argument to show that the generalization gap scale as
[Equation]
where (cid:107) · (cid:107) is the spectral norm.
The proof is based on the induction on layers.
Let W l be the weight matrix of the present layer and X l be the output of X pass through the ﬁrst to the l − 1 layer.
Then, one can compute the matrix covering number N ( { W l X l } , (cid:107) · (cid:107) 2 , (cid:15) ) by induction.
Adversarial Generalization Bounds.
Researchers have analyzed adversarial Rademacher complexity in linear and two-layers neural networks cases.
In linear cases, the upper bounds can be directly derived by deﬁnition (Khim & Loh (2018); Yin et al.
In two-layers neural networks cases, an upper bound is derived using Massart’s Lemma (Awasthi et al.
These proofs cannot be extended to multi-layers cases.
Moreover, based on the deﬁnition of adversarial function class ˜
F in equation (1), the candidate functions are not composition functions, but with an inf operation in front of the neural networks.
Then, the induction on layers seems not applicable in calculating adversarial Rademacher complexity for deep neural networks.
The works of (Khim & Loh (2018)) and (Gao & Wang (2021)) indicate the difﬁculty of analyzing adversarial Rademacher complexity.
They analyze other variants of adversarial Rademacher complexity of DNNs.
‘Tree Transformation’ Bound.
Khim & Loh (2018) introduces a tree transformation T and shows that max (cid:107) x − x (cid:48) [Equation]
f [Equation] x ) , y ) ≤ [Equation]
( Tf ( x ) , y ) .
Then, we have the following upper bound for the adversarial population risk.
For δ ∈ (0 , 1) ,
[Equation]
It gives an upper bound of the robust population risk by the empirical risk and the standard Rademacher complexity of T ◦ f .
R S
( T ◦ F ) can be viewed as an approximation of adversarial Rademacher complexity.
However, the empirical risk R m ( Tf ) is not the objective in practice.
This analysis does not provide a guarantee for robust generalization gaps.
FGSM Attack Bound.
The work of (Gao & Wang (2021)) tries to provide an upper bound for adversarial Rademacher complexity.
To deal with the max operation in the adversarial loss, they consider FGSM adversarial examples.
Then, the adversarial loss max
[Table]
(cid:96) ( x , y ) becomes (cid:96) ( f ( x FGSM ) , y ) .
By some assumptions on the gradient, they provide an upper bound for the
Rademacher complexity of (cid:96) ( f ( x FGSM ) , y ) .
However, the bound includes some parameters of the assumptions on the gradients, and FGSM underestimates the adversarial examples.
It is hard to use this bound to analyze adversarial generalization.
Therefore, the existing bounds give limited interpretations in understanding the generalization of adversarial training.
B. W HY L AYER P EELING IS NOT A PPLICABLE IN A DVERSARIAL S ETTING ?
We ﬁrst take a look at the layer peeling technique.
[Equation]
In adversarial settings
[Equation]
if we directly apply the layer peeling technique
[Equation]
we have
[Equation]
[Table]
[Equation]
[Table]
[Equation] [Table] [Equation] where x ∗ i ( h ) is the optimal adversarial example given a d -layers neural networks,
x ∗ i ( h (cid:48) ) is the optimal adversarial example given a d − 1 -layers neural networks.
x ∗ i ( h ) (cid:54) =
x ∗ i ( h (cid:48) ) is the main reason why layer peeling cannot be directly extended to the adversarial settings.
This is the main reason why the work we introduce above studied the variants of adversarial
Rademacher complexity.
Once they take off the max operation by some approximation (e.g., let max (cid:107) x − x (cid:48)
(cid:107)≤ (cid:15) (cid:96) ( f ( x ) , y ) ≤ (cid:96) (
Tf ( x ) , y ) ), they don’t have the issue x ∗ i ( h ) (cid:54)
= x ∗ i ( h (cid:48) ) and they can use the layer peeling technique to bound the variants of adversarial Rademacher complexity.
The main drop back is that they change the deﬁnition of adversarial Rademacher complexity.
These bounds cannot provide theoretical guarantee on the robust generalization gap.
B. W HY COVERING NUMBER CAN HELP AVOIDING THIS ISSUE ?
In our opinion, it is hard to modify the procedure of layer peeling such that it is applicable in adversarial settings.
Therefore, we try to bound the adversarial Rademacher complexity in a different way, using the covering number.
In the proof of Theorem 1, we can see that we can avoid the issue of x ∗ i ( h ) (cid:54) =
x ∗ i ( h (cid:48) ) .
Speciﬁcally, when we calculate the covering number of the whole function class ˜ F directly, we only need to deﬁne the optimal adversarial examples x ∗ i for a d -layer neural networks.
We don’t need to consider the optimal adversarial examples of neural networks with fewer layers.
This is the beneﬁt of covering numbers.
B. C OMPARISON OF D IFFERENT A DVERSARIAL G ENERALIZATION B OUNDS
VC-Dimension Bounds.
A classical approach in statistical learning is to use VC dimension to bound the generalization gap.
It is thus natural to apply the VC-dim framework to adversarial setting, as (Cullina et al.
; Montasser et al.
; Attias et al. (2021)) did. However, these works did not provide a computable bound on the adversarial generalization gap, as explained next. let H be the hypothesis class (e.g. the set of neural networks with a given architecture).
In the work of (Cullina et al.
, the authors deﬁned adversarial VC-dim (AVC) and gave an bound on adversarial generalization gap with respect to AV C ( H ) . However, they did not show how to calculate AVC of neural works. Therefore, their paper did not provide a computable bound for adversarial generalization gap. In the work of(Montasser
et al.
, the authors deﬁned the adversarial function class as L UH , where L is the loss and U is the uncertainty set.
They bound the adversarial generalization gap by L UH , which is different from AV C ( H ) of (Cullina et al.
However, the authors did not provide a computable bound of as well, which means that their paper did not provide a computable bound of the adversarial adversarial generalization gap.
In the work of (Attias et al.
, the authors assume that the perturbation set U ( x ) is ﬁnite, i.e., for each sample x , there are only k adversarial examples that can be chosen.
They showed that the adversarial generalization gap can be bounded by
[Equation]
Note that there is a computable bound of V C ( H ) , which is the number of parameters, thus in terms of ”computable”, this bound is stronger than the previous two.
However, this comes at a price: their bound depends on k , the number of allowed examined perturbed samples.
This is a deviation from the original notion of adversarial generalization, where U ( x ) is assumed to be an inﬁnite set ( k (cid:54) = + ∞ ).
In contrast, our bound is for the ”original” adversarial generalization gap, which allows k = + ∞ .
Adversarial Generalization Bounds in Other Settings.
The work of (Xing et al.
(2021a;b); Javanmard et al.
(2020)) study the generalization properties in the setting of linear regression.
Gaussian mixture models are used to analyze adversarial generalization (Taheri et al.
; Javanmard et al.
Dan et al.
Certiﬁed robustness. A series of works study the certiﬁed robustness within the norm constraint around the original data.
Cohen et al.
(2019) privides an analysis on certiﬁed robustness via random smoothing.
Lecuyer et al.
(2019) studies certiﬁed robustness through the lens of differential privacy. Other Theoretical Studies on Adversarial Examples.
A series of works (Gilmer et al.
Khoury & Hadﬁeld-Menell (2018)) study the geometry of adversarial examples.
The off-manifold assumption tells us that the adversarial examples leave the underlying data manifold (Szegedy et al.
Pixeldefends (Song et al.
(2017)) uses a generative model to show that adversarial examples lie in a low probability region of the data distribution.
The work of (Ma et al.
(2018)) uses Local Intrinsic Dimensionality (LID) to argues that the adversarial subspaces are of low probability, and lie off the data submanifold.
C A DDITIONAL E XPERIMENTS
In this section, we provide additional experiments.
C
E XPERIMENTS ON VGG-11 AND VGG-
In Figure 2, we show the experiments on VGG-11 and VGG-13.
As we can see, the results are the same as the results in Figure 3, the gap of product of Frobenius norm between standard and adversarial training is large, which yields bad generalization. [Figure] [Figure] VGG
[Figure] [Figure] [Figure] (cid:107)·(cid:107) 1 , ∞
-Norm Bounds.
The (cid:107)·(cid:107) 1 , ∞ -norm bounds are shown in Figure 4.
Similar the the Frobenius norm bounds,The gap of (cid:81) dl
=1 (cid:107) W l (cid:107) 1 , ∞ between adversarial training and standard training are large.
But the magnitude of (cid:81) dl =1 (cid:107) W l (cid:107) 1 , ∞ is larger than the magnitude of (cid:81) dl
=1 (cid:107) W l (cid:107) F .
[Figure]
C. A BLATION S TUDY OF M ARGINS
In Figure 5, we show the results of the margins in 1 th , 3 th , and, 5 th -percentile of the training dataset.
Since the (robust) training accuracy is 100%, the choice of percentile will not affect the results.
As we can see in the Figure, in all the cases, the margins of standard training are larger than the margins of adversarial training.
Since the margins appear in the divider in the upper bound of Rademacher complexity, the margins of the training dataset have some small effects on the bad generalization of adversarial training.
C. E XPERIMENTS ON CIFAR-
Perfomance.
In Table 2, we show the performance of standard training and adversarial training on CIFAR-100 using VGG-16 and 19 networks.
We can see that using smaller number of training samples is unable to train an acceptable VGG-networks on CIFAR-100.
Therefore it is hard use only 50000 training samples to study the trends of the weight norm using the experiments on CIFAR-100.
We compare the product of weight norm between standard and adversarial training.
Product of Weight Norms.
In Figure 6, we show the results of on training VGG-19-16 and VGG19 on CIFAR-100.
Similar to the experiments on CIFAR-10, we can see that the adversarially trained models have larger weight norm that that of the standard trained model.
For standard training model, we shows the clean accuracy.
For adversarial training model, we show the robust accuracy against PGD
[Table]
C. W EIGHT D ECAY
The upper bounds of adversarial Rademacher complexity suggest adding a regularization term on the weights to improve generalization, which is essentially weight decay.
In Figure 7, we provide the experiments of adversarial training with and without weight decay.
In Figure 7 (a) and (c), we can see that adversarial training with weight decay has a smaller robust generalization gap.
In Figure(b) and (d), adversarial training with weight decay have a smaller product of weight norms.
These experiments show the relationship between the robust generalization gap and the product of weight norms.
1 t h - p
e r ce n til e (a) (b) (c) (d)
3 t h - p
e r ce n til e (e) (f) (g) (h)
5 t h - p e r ce n til e ( [Figure] ) (j) (k) (l)
[Figure]
[Figure]
[Figure]
D O PEN P ROBLEM
In this section, we list some open problems.
How to bridge the gap between the upper bound and the lower bound?
There are two ways: one way is to show a depth/width-dependent lower bound as the reviewer suggested (increase lower bound); another way is to show a depth/width-independent upper bound (reduce upper bound). We brieﬂy discuss which way is possible, and then discuss the technical challenges in both ways. Which way is more likely to be true? If the upper bound can be improved to be depth-widthindependent (thus matching our lower bound), then fundamentally the lower bound cannot be improved.
Such a possibility exists. Actually, we are more inclined to this possibility, i.e., we tend to believe it is more promising to reduce the upper bound to be depth-width-independent, rather than increasing the lower bound. Anyhow, we don’t have strong evidence of this possibility. Technical challenge on increasing the lower bound (obtain a depth/width-dependent lower bound).
In the current analysis, we construct a class of scalar networks to provide the lower bound.
We obtain a closed-form expression of the adversarial examples.
To obtain a depth/width-dependent lower bound, we need to: i) construct a more general function class of neural networks, and ii) then calculate the optimal adversarial examples in this class of neural networks.
Currently, the challenge lies in the ﬁrst step (construction).
We have not tried hard to construct the function class so far, and we leave it to future work.
Can we reduce the upper bound (remove the dependence on depth/width in the upper bound)?
This seems quite difﬁcult by the current analysis. More speciﬁcally, the dependence h √ dlogd is probably unavoidable by our current approach of calculating the covering number.
Despite the technical difﬁculty, we suspect that reducing the upper bound is doable by using a new tool other than covering number and layer peeling. This is surely nontrivial.
Why adversarial training yields larger weight norms?
In our opinion, it is because we require the additional capacity of the neural networks to ﬁt the adversarial examples.
As in the discussion of the work of (Neyshabur et al.
(2017a))
, we require more capacity of the model to ﬁt random labels.
A model with larger weight norms has a better ability to ﬁt the training data.
There might be other reasons, for example, the loss landscape of the minimax problem of adversarial training, the implicit bias of PGD attacks, or the implicit bias of adversarial training.
Is the widely used regularization techniques essentially reducing weight norms?
In adversarial training, there are many training tricks to reduce overﬁtting and yield better generalization, for example, stochastic weight averaging, early stopping, adversarial weight perturbation, and cyclic learning.
It is an open problem that whether these techniques are related to the weight norms.
How to design better algorithms to improve generalization?
Our analysis suggests that adding regularization to the weight norms could improve generalization.
The explicit regularization to control the weight norms is weight decay, which is widely used.
How to design implicit control on the weight norms is an open problem.