Implicit Bias of Adversarial Training for Deep Neural Networks

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved great success in many fields such as computer vision ( Krizhevsky et al. , 2012 ; He et al. , 2015 ) and natural language processing ( Collobert & Weston , 2008 ) and other applications . These breakthroughs also lead to the importance of research on the robustness and security issues of DNNs , among which those about adversarial examples are especially prevalent . Adversarial examples are obtained by adding craftily designed imperceptible perturbations to the original examples , which can sharply change the predictions of the DNNs with high confidence ( Szegedy et al. , 2014 ; Nguyen et al. , 2015 ) . Such vulnerability of DNNs renders security concerns about deploying them in security-critical systems including vision for autonomous cars and face recognition . It is therefore crucial to develop defense mechanisms against these adversarial examples for deep learning . To this end , many defense strategies have been proposed to make DNNs resistant to adversarial examples , such as adding a randomization layer before the input to the classifier ( Xie et al. , 2018 ) , input transformations ( Guo et al. , 2018 ) , adversarial training ( Madry et al. , 2018 ) , etc . However , Athalye et al . ( 2018 ) pointed out that most of the defense techniques are ineffective and give a false sense of security due to obfuscated gradients except for adversarial training—a method which has not been comprehensively attacked yet . Given a C-class training dataset { ( xi , yi ) } ni=1 with natural example xi ∈ Rd and corresponding label yi ∈ { 1 , ... , C } , adversarial training is formulated as solving the following minimax optimization problem min W L̃ ( W ) = min W 1 n n∑ i=1 max δi∈B ( 0 , ) ` ( xi + δi ( W ) , yi ; W ) , ( 1 ) where f is the DNN function , ` is the loss function and δi ( W ) is the adversarial perturbation generated by some adversary A ( xi , yi ; W ) typically depending on model parameters W , within the set B ( 0 , ) = { δ : ‖δ‖p ≤ } around the natural example xi . Commonly , adversarial training conducts a two-player game at each iteration : the inner maximization is to attack the model and find the corresponding perturbation of the original example that maximizes the classification loss ` ; the outer minimization , on the other hand , is to update the model parameters W with the gradient descent method such that the loss ` is minimized on adversarial examples generated by the inner maximization ( see Algorithm 1 for details ) . To have a better theoretical understanding of the fact that adversarial training empirically improves the robustness of DNNs against adversarial examples , Zhang et al . ( 2019 ) decomposed the prediction error for adversarial examples and identified a trade-off between robustness and accuracy while Li et al . ( 2020 ) studied the inductive bias of gradient descent based adversarial training for logistic regression on linearly separable data . Faghri et al . ( 2021 ) studied adversarial robustness of linear neural networks by exploring the optimization bias of different methods . Yu et al . ( 2021 ) studied adversarial training through the bias-variance decomposition and showed that its generalization error on clean examples mainly comes from the bias . However , even for the simplest DNN—the deep linear network , we notice that there exists no work to theoretically understand such robustness achieved by adversarial training through exploring its implicit bias . On the other hand , for the standard training , recent works extensively explored the implicit bias imposed by gradient descent or its variants for DNNs in different settings . For a simple setting of linear logistic regression on linearly separable data , Soudry et al . ( 2018 ) ; Ji & Telgarsky ( 2018 ) ; Nacson et al . ( 2019b ) derived that the direction of the model parameter converges to that of the max- ` 2-margin with divergent norm . Shamir ( 2021 ) concluded that gradient methods never overfit when training linear predictors over separable dataset . Viewing the above model as a single-layer network , a natural but more complicated extension is that for the deep linear network on linearly separable data : Ji & Telgarsky ( 2019 ) proved that the gradient descent aligns the weight matrices across layers and the product of weight matrices also converges to the direction of the max- ` 2-margin solution ; Gunasekar et al . ( 2018 ) showed the implicit bias of gradient descent on linear convolution networks . Nacson et al . ( 2019a ) ; Lyu & Li ( 2020 ) further promoted the study of implicit bias of gradient descent for standard training to the case of more general homogeneous non-linear neural networks and proved that the limit point of the optimization path is along the direction of a KKT point of the max-margin problem . Wei et al . ( 2019 ) studied the regularization path for homogeneous DNNs and also proved the convergence to the max-margin direction in this setting . Banburski et al . ( 2019 ) showed that gradient descent induces a dynamics of normalized weights converging to an equilibrium which corresponds to a minimal norm solution . It is therefore our goal in this paper to theoretically understand the resistance to adversarial examples for adversarially trained DNNs , linear and non-linear ones , through the lens of implicit bias imposed by adversarial training . Due to the inner maximization , adversarial training differs a lot from standard training and one should be careful to analyze the perturbed training dynamics . For the adversarial training objective Eq . ( 1 ) , various approaches have been proposed to solve the inner maximization , such as fast gradient sign method ( FGSM ( Goodfellow et al. , 2015 ) ) and its stronger version projected gradient descent ( PGD ( Madry et al. , 2018 ) ) . The widely used adversarial training adopts PGD to attack the model , while recent work ( Wong et al. , 2020 ) also suggested that a weaker adversary can also surprisingly yield a model with satisfying robustness . Thus to conduct a comprehensive study about the implicit bias of adversarial training for DNNs , we will use ` 2-fast gradient method ( FGM ( Miyato et al. , 2016 ) ) , ` ∞ FGSM , ` 2-PGD and ` ∞-PGD to solve the inner maximization of the adversarial training objective . 1.1 OUR CONTRIBUTION . In this paper , we devote to answering two questions . First , is there any implicit bias imposed by adversarial training for DNNs without explicit regularization ? Second , if there exists such an implicit bias , what are the convergence properties of the model parameters along the adversarial training trajectory ? To this end , we first investigate the adversarial training with ` 2 adversarial perturbations for deep linear networks on linear separable data where the allowed Euclidean distances from the adversarial examples to their corresponding original examples ‖x′i − xi‖2 are less than the max- ` 2-margin of the original dataset . Despite the simplicity of this setting , this problem is meaningful due to its non-convexity and the introduction of adversarial examples , which heavily depend on the model parameters , during the training process . We prove that gradient descent for adversarial training implicitly aligns weight matrices across layers and the direction of the product of weight matrices also surprisingly converges to that of the max- ` 2-margin solution of the original dataset—similar to that of standard training for deep linear network ( Ji & Telgarsky , 2019 ) . Our results significantly generalize those in Li et al . ( 2020 ) for adversarial training of logistic regression . This simple yet insightful case positively answers our first question but partially answers the second one because it still remains unclear why such convergence property can improve robustness considering its similarity to that for the standard training . In fact , adversarial training differs from standard training in the way how they impose such convergence property for parameters : the first one is to maximize the margin of adversarial examples while the latter is maximizing that of the original dataset , and these two optimization problems happen to possess solutions along the same direction . We then move forward to explore a more general situation , adversarial training for homogeneous non-linear DNNs without the linear separability of the dataset . We study the limit point of the normalized model parameters along the adversarial training trajectory and show that Theorem 1 ( Informal ) . When the deep neural network is adversarially trained with one of the ` 2-FGM , FGSM , ` 2-PGD and ` ∞-PGD perturbations , the limit point of the normalized model parameters is along the direction of a KKT point of a constrained optimization problem which aims to maximize the margin of adversarial examples . This indicates that adversarial training is implicitly maximizing the margin of adversarial examples rather than that of original dataset . Thus Theorem 1 provides another view for the high bias error on clean examples of adversarial training in Yu et al . ( 2021 ) since distributions of adversarial and clean examples are different . To the best of knowledge , these results are the first attempt to analyze the implicit bias of adversarial training for DNNs . We believe our results provide a theoretical understanding on the effectiveness of adversarial training for improving robustness against adversarial examples . It could potentially shed light on how to enhance the robustness of adversarially trained models or even further inspire more effective defense mechanisms . Organization This paper is organized as follows . Section 2 is about notations and settings . Section 3 presents our main results on the implicit bias of adversarial training for DNNs . Section 4 provides numerical experiments to support our claims . We conclude this work in Section 5 and discuss future directions . Some technical proofs are deferred to supplementary materials . Algorithm 1 Adversarial Training Input : Training set S = { ( xi , yi ) } ni=1 , Adversary A to solve the inner maximization , learning rate η , initialization Wk for k ∈ { 1 , . . . , L } for t = 0 to T − 1 do S′ ( t ) = ∅ for i = 1 to n do x′i ( t ) = A ( xi , yi , W ( t ) ) S′ ( t ) = S′ ( t ) ⋃ ( x′i ( t ) , yi ) end for for k = 1 to L do Wk ( t+ 1 ) = Wk ( t ) − η ( t ) ∂L̃ ( S ′ ( t ) ; W ) ∂Wk end for end for 2 PRELIMINARIES . Notations . For any matrix A ∈ Rm×n , we denote its i-th row j-th column entry by Aij . Let AT denote the transpose of A . ‖ · ‖F represents the Frobenius norm and ‖ · ‖p is the ` p norm . The training set is { ( xi , yi ) } ni=1 where xi ∈ Rd , ‖xi‖2 ≤ 1 and yi ∈ { 1 , −1 } . For a scalar function f : Rd 7→ R , we denote its gradient by∇f . Furthermore , tr ( A ) = ∑ iAii denotes the trace for the matrix A . We study the adversarial training for the L-layer positively homogeneous deep neural network f ( x ; W ) = WLϕL ( WL−1 · · ·ϕ3 ( W2ϕ2 ( W1x ) ) · · · ) ( 2 ) where Wk is the k-th layer weight matrix and ϕk is the activation function of the k-th layer 1 . The multi-c-homogeneity of the network is defined by f ( x ; a1W1 , · · · , aLWL ) = L∏ k=1 ackf ( x ; W ) ( 3 ) for any positive constants ak ’ s and c ≥ 1 , where W = ( W1 , · · · , WL ) is the collection of the parameters of the network . For example , deep ReLU networks are multi-1-homogeneous . For convenience , we also adopt the following notations for a multi-ck-homogeneous DNN : ρk = ‖Wk‖F , ρ = ρc1 · · · ρcL and f ( x ; W ) = ρf ( x ; Ŵ ) , ( 4 ) where Ŵk = Wk/‖Wk‖F with ‖Ŵk‖F = 1 for k ∈ { 1 , . . . , L } . We use δi ( W ) to represent the adversarial perturbation of the original example xi within the perturbation set B ( 0 , ) : { δ : ‖δ‖p ≤ } around the original example xi for f ( x ; W ) . Furthermore , we use the scale invariant adversarial perturbations defined as follows for adversarial training in this paper . Definition 1 ( Scale invariant adversarial perturbation ) . An adversarial perturbation is said to be a scale invariant adversarial perturbation for f ( xi ; W1 , . . . , WL ) and loss function ` if it satisfies δi ( a1W1 , . . . , aLWL ) = δi ( W1 , . . . , WL ) ( 5 ) for any positive constants ak ’ s . We will show in Section 3.2 that FGSM , ` 2-FGM , ` 2-PGD and ` ∞-PGD perturbations for homogeneous DNNs are all scale invariant perturbations , which are important for analyzing different types of perturbation in a unified manner . The empirical adversarial training loss with the perturbation δi ( W ) is given by L̃ ( W ) = 1 n n∑ i=1 ˜̀ ( yi , xi ; W ) = 1 n n∑ i=1 ` ( yif ( xi + δi ( W ) ; W ) ) . ( 6 ) For ease of notation , we denote ˜̀i ( W ) = ˜̀ ( yi , xi ; W ) . The loss function ` is continuously differentiable and satisfies Assumption 1 ( Loss function ) . ` > 0 , ` ′ < 0 , limx→∞ ` ( x ) = 0 and ` x→−∞ ( x ) =∞ . Many widely used loss functions satisfy the above assumption such as ` ( x ) = e−x and the logistic loss ` ( x ) = ln ( 1 + e−x ) . Furthermore , we make the following common assumptions about the smoothness of f ( x ; W ) and the adversarial perturbation δ ( W ) : Assumption 2 ( Smoothness ) . With respect to W , yf ( x ; W ) is locally Lipschitz for any fixed x ; yif ( xi ; W ) further have locally Lipschitz gradients and δi ( W ) are locally Lipschitz for all training examples xi . Remark . Our results can also be generalized to non-smooth homogeneous neural networks straightforwardly ( Appendix B.4 ) . Assuming Lipschitzness about perturbations is because we focus on popular perturbations such as ` 2-FGM and PGD perturbations which have explicit forms and depend on gradients of the network , whose Lipschitzness assumptions are quite common .

Summary: This paper aims to understand the training results of adversarial training, and proves that under certain conditions, adversarial training results maximize the margin for the adversarial training samples. Similar results have been observed in the cleaning training of DNN, this paper's contribution is to extend them to the adversarial training settings. The paper seems technically sound (although I don't have the time to go over all the appendix). The results, to be honest, are not surprising, given previous works on standard training. But I believe the rigorous justification presented in the paper is of importance.