THE EFFECT OF ADVERSARIAL TRAINING: A THEORETICAL CHARACTERIZATION

1 INTRODUCTION . Despite the impressive performance of deep neural networks on various learning tasks , the widely existing adversarial examples ( Goodfellow et al. , 2014 ; Szegedy et al. , 2017 ) has thwarted its application in the safety-sensitive scenarios ( Kurakin et al. , 2016 ; Chen et al. , 2015 ) . A well trained neural network can be vulnerable to certain small adversarial perturbation added to the original data , despite the perturbations is almost imperceptible to human . Adversarial training ( Madry et al. , 2018 ) is an effective method to train a robust deep neural network that can resist adversarial samples to some extent . However , the theoretical understanding of adversarial training is quite limited . In this paper , we try to unveil the mystery of adversarial training . Specifically , in this paper for adversarial training , we consider the attack in l2 norm , i.e . the following objective min θ 1 N N∑ i=1 max ‖x−xi‖2≤ε ` ( x , θ ) . ( 1 ) This is in contrast with the standard training objective minθ 1N ∑N i=1 ` ( x , θ ) . To obtain a clear theoretical characterization , we focus on the linear classifier . In fact , for linearly separable data , Soudry et al . ( 2017 ) ; Ji & Telgarsky ( 2018 ) have proven that the linear classifier trained by gradient descent ( GD ) with logistic loss converges to the hard margin solution of SVM with a rate of O ( ( log t ) −1 ) . However , we find things become much different for adversarial training . We first prove that for ” ε-strongly linearly separable ” data ( Definition 1 ) , GD can find the hard margin classifier with a faster rate of O ( ( log t ) − ( 1+ε ∗ ) ) , with the same exponential tail loss used in Soudry et al . ( 2017 ) . Here ε∗ = ( |S|ε ( ε1−ε ) ) /N , where ε1 is the distance between the support vectors and the hard margin solution of SVM , and |S| is the number of support vectors corresponding to hard margin classifier . This result shows that we can find a robust solution much faster by adversarial training if the data themselves are more separable than the adversarial radius . When the data are not ε-strongly linearly separable , adversarial training gives significantly different solutions from the standard training . To further illustration , we consider the following case , most of the data are ε-strongly linearly separable while there several outliers are not or even not linearly separable . Then the classifier returned by standard training is heavily affected by these outliers , since the classifier returned by standard training converges to the hard margin classifier while it can be sensitive to outliers . However , we can show the stability of adversarial training to outliers , i.e . the classifier returned by adversarial training is slightly affected by outliers . Next , we also show that adversarial training leads to a classifier with relatively lower confidence in each data point than that of standard training . We then give a formal characterization for this phenomenon under the case of a large ε . The low confidence in each training data naturally induces a high training loss . A simple generalization error bound informs the high loss on test set , which interprets the widely observed poor test performance of adversarial training . 1.1 RELATED WORK . Plenty of work trying to obtain a large margin solution to promote model robustness . Cisse et al . ( 2017 ) ; Hein & Andriushchenko ( 2017 ) regularize the training with the Lipschitz constant of the model to enhance robustness . Another line of work ( Elsayed et al. , 2018 ; Lee et al. , 2019a ; Serra et al. , 2018 ; Croce et al. , 2019 ) uses a first order approximation to compute the margin of deep neural network , and then find a large margin solution by setting the approximation to be the optimization objective . However , none of these works come up with a theoretical guarantee . Our paper is also related with Soudry et al . ( 2017 ) ; Ji & Telgarsky ( 2018 ) who prove that gradient descent with logistic loss converge to the hard margin classifier for linearly separable data but with a rate of O ( ( log t ) −1 ) or on non-linearly separable data with rate of O ( log log t/ log t ) . Nacson et al . ( 2018 ) extend this result to stochastic gradient descent . Gunasekar et al . ( 2018 ) study the convergence under other optimization methods . Lyu & Li ( 2019 ) provides a similar result for the homogeneous neural networks . However , they all target the standard optimization problem rather than the adversarial training . Ilyas et al . ( 2019 ) studies adversarial training via the lens of robust/non-robust features . They claim that adversarial attacks are attributed to the presence of non-robust features and adversarial training can be viewed as explicitly preventing the classifier from learning useful but non-robust features . Their theory is presented under the parameter estimation framework , which is different from the conventional adversarial training . Moreover , they do not discuss the convergence direction of adversarial training . 2 NOTATIONS AND ASSUMPTIONS . In this section , we introduce the notations and assumptions we used in this paper . We consider a binary classification problem . Notations . Dataset is represented as { xi , yi } Ni=1 , where xi ∈ Rd and yi ∈ { −1 , 1 } . The loss function is represented by ` ( · ) . ‖ · ‖ means l2 norm in this paper . The objective of standard training can be represented as L0 ( w ) = 1 N N∑ i=1 ` ( wT yixi ) , ( 2 ) where w ∈ Rd is a linear classifier . Adversarial training has the objective of L ( w ) = 1 N N∑ i=1 max ‖x−xi‖≤ε ` ( wT yix ) , ( 3 ) where ε is the adversarial radius . An intuitive explanation of adversarial training is that it requires the classifier perform well in the ε-ball centered at each data xi . In this paper , wt is the iterate at step t of gradient descent ( GD ) under adversarial training . We use ŵ to represent the solution of SVM ( 8 ) . Now , we give the definition of ε-strongly linearly separable data . Definition 1 ( ε-strongly linearly separable ) . 1 The dataset is ε-strongly linearly separable , if there exists w∗ such that ∀xi , w∗T yixi > 0 and w∗T yixi > ε‖w∗‖ . The ε-strongly linearly separable means there exists a linear classifier that does not only give a right prediction to each data point but also ensures all data are away from the linear classifier larger than 1Please notice that ε-strongly linearly separable is a stronger condition compared with linearly separable . It distinguishes with the separability under soft margin which allows some w∗T yixi < 0 but w∗T yixi > ε‖w∗‖ . ε . In addition , linearly separable refers to choosing ε = 0 in Definition 1 . We now define confidence of classifier . Definition 2 ( Confidence ) . For a given linear classifier w of binary classification problem , the logits is 1/ ( 1 + e−w Txi ) and e−w Txi/ ( 1 + e−w Txi ) for category 1 and −1 . Then max { 1/ ( 1 + e−w Txi ) , e−w Txi/ ( 1 + e−w Txi ) } is the confidence of w on data xi . The confidence measures how confident the classifier is about the prediction . In the sequel , we use xi to represent yixi for simplify the notations . We introduce the assumptions used in this paper . Assumption 1 . The loss function ` ( · ) 2 satisfies that ∀u , ` ( u ) > 0 , ` ′ ( u ) < 0 , l′′ ( u ) ≥ 0 , limu→∞ ` ( u ) = limu→∞ ` ′ ( u ) = 0 , and the ` ( u ) has Lipschitz gradient.3 Assumption 2 . The l ( · ) and l′ ( · ) have exponential tail , which means there exists some constant u0 and C1 , C2 satisfies that ∀u > u0 , C1e−u ≤ l ( u ) ≤ C2e−u as well as l′ ( u ) .4 Assumption 3 . ThewTxi−ε‖w‖ has the range of [ c1 , ∞ ) for eachxi , where c1 > −∞ . ‖w‖ ≥ c2 for some constant c2.5 . Due to the properties of loss function , the adversarial training objective ( 3 ) with radius ε has an explicit formula L ( w ) = 1 N N∑ i=1 ` ( wTxi − ε‖w‖ ) . ( 4 ) The adversarial training objective equation 4 is optimized by the gradient descent , i.e. , wt+1 = wt − η∇L ( wt ) = wt − η N∑ i=1 ` ′ ( wTt xi − ε‖wt‖ ) ( xi − ε wt ‖wt‖ ) , ( 5 ) where η is the learning rate . 3 ADVERSARIAL TRAINING CONVERGES FASTER TO HARD MARGIN FOR . ε-STRONGLY LINEARLY SEPARABLE DATA In this section , we theoretically characterize where adversarial training converges when the data themselves are ε-strongly linearly separable . We first have the following key lemma which ensures the loss of adversarial training with radius ε can converge to zero on ε-strongly linearly separable data . The proof of this lemma is delegated to Appendix A. Lemma 1 . The adversarial training objective equation 4 is convex and L-smooth for some positive constant L. Then if the data are ε-strongly linearly separable ( 1 ) , the iterates { wt } returned by GD under adversarial training satisfy 1 ) : limt→∞ L ( wt ) = 0 , 2 ) : limt→∞ ‖wt‖ = ∞ . and 3 ) : limt→∞w T t xi − ε‖wt‖ =∞ The distance of xi away from linear classifier wt is |wTt xi|/‖wt‖ . Then , the convergence result in the Lemma 1 does not inform us the robustness of trained classifier . Fortunately , the results in Soudry et al . ( 2017 ) ; Ji & Telgarsky ( 2018 ) claim that the iterates of standard training updated by GD can converge to hard margin classifier with the rate of O ( ( log t ) −1 ) . Now , we give the main result of this section . The conclusion is similar to Theorem 3 in Soudry et al . ( 2017 ) while the convergence is much sharper . Theorem 1 . For any ε-strongly linearly separable data ( Definition 1 ) , and loss function ` ( · ) satisfies Assumption 1 and 2 . If the Assumption 3 holds , then the gradient flow iterates w ( t ) , dw ( t ) dt = −∇L ( w ( t ) ) ( 6 ) satisfies w ( t ) = ŵ ·O ( log t ) + h ( t ) , ( 7 ) 2The loss is set to be positive , differentiable , monotonically decreasing convex function . Lots of general used loss functions are satisfied , such as e−u , log ( 1 + e−u ) etc . 3It means that | ` ′ ( u ) − ` ′ ( v ) | ≤ L|u− v| for some positive constant L. 4Lots of loss functions we used such as log ( 1 + e−u ) , e−u are satisfied with this assumption . 5This assumption can be attained by re-scale the norm of w if necessary . for a large t. Here ŵ is the hard margin solution of SVM : ŵ = arg min w∈Rd ‖w‖ s.t.wTxi ≥ 1 . ( 8 ) ‖h ( t ) ‖ is in the order of o ( log t ) . Then lim t→∞ ∥∥∥∥ w ( t ) ‖w ( t ) ‖ − ŵ‖ŵ‖ ∥∥∥∥ ≤ O ( ( log t ) − ( 1+ε∗ ) ) , ( 9 ) where ε∗ = |S|ε ( ε1−ε ) N , { εi } is the sorted distance of data away from the hard margin solution of SVM , and S = { i : ŵTxi = ε1‖ŵ‖ } ; |S| is the number of elements in set S. Finally , iterates { wt } of adversarial training updated by GD with step size η satisfies ‖wk −w ( kη ) ‖ ≤ O ( η ) , ( 10 ) for k ∈ N+ . Then we can conclude lim t→∞ ∥∥∥∥ wt‖wt‖ − ŵ‖ŵ‖ ∥∥∥∥ ≤ limt→∞ ∥∥∥∥ w ( tη ) ‖w ( tη ) ‖ − ŵ‖ŵ‖ ∥∥∥∥+O ( η ) . ( 11 ) We give a brief discussion to hard margin solution of SVM . Since the dataset is ε-strongly linearly separable , by KKT condition , we have ε < 1/‖ŵ‖ which informs us all the distance between xi and ŵ are larger than ε . Then , we present that perturbations smaller than ε can be defended by ŵ . Hence , the linear classifier wt becomes robust for a large t , since it has the same direction with ŵ . We then conclude adversarial training on ε-strongly linearly separable data helps iterates { wt } converge to a robust solution with a fast rate . A detailed proof of this Theorem is delegated to Appendix C. For the theorem , we have several remarks about the adversarial training on ε-strongly linearly separable data . • Adversarial training converges to a robust solution , the same as standard training , with a faster rate . The acceleration is determined by the constant ε∗ = ( |S|ε ( ε1 − ε ) ) /N which related with the true margin ε1 and the number of support vectors |S| . • Large proportion of support vectors |S|/N , and an appropriate choice of ε ( ε1/2 is the best ) increase the convergence rate . We next study where adversarial training converges when the data are not ε-strongly linearly separable . 4 ADVERSARIAL TRAINING WHEN DATA ARE NOT ε-STRONGLY LINEARLY SEPARABLE It ’ s hard to obtain the exact separability of data without verifying the hard margin solution of SVM before adversarial training . Hence , understanding adversarial training on non-ε-strongly linearly separable data is also crucial .

Summary: This paper provides some analyses of the difference between adversarial training and standard training for linear classification problem. In particular, it proves that when the data is \eps linearly separable, adversarial training converges faster than standard trading. It also argues that when the data is not \eps linearly separable, adversarial training is more robust to outlier. Simulations are constructed to verify the arguments in the paper but there is no experiments on real dataset.