On the Algorithmic Stability of Adversarial Training

1 Introduction . Successful machine learning algorithms require not only a good empirical performance but also generalizing well to unseen data . For the robustness towards unseen data , empirical experiments show that deep learning models can be fragile and vulnerable against adversarial input ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ) . To set an example , in image recognition problems , a deep neural network will predict a wrong label when the testing image is slightly altered , while the change is barely recognizable by human eyes ( Papernot et al. , 2016a ) . Related research efforts in adversarial learning include designing adversarial attacks in various applications ( Papernot et al. , 2016a , b ; Moosavi-Dezfooli et al. , 2016 ) , detecting attacked samples ( Tao et al. , 2018 ; Ma and Liu , 2019 ) , and modifications on the training process to obtain adversarially robust models , i.e. , adversarial training ( Shaham et al. , 2015 ; Madry et al. , 2017 ; Jalal et al. , 2017 ) . However , although adversarial training improves the adversarial robustness during testing , its generalization performance is still poor . While Yin et al . ( 2018 ) presented that the adversarial Rademacher complexity is never smaller than its natural counterpart , Schmidt et al . ( 2018 ) ; Zhai et al . ( 2019 ) argued that a better adversarial generalization requires more labeled/unlabeled data . In the literature of natural DNN optimization via iterative gradient moves , the empirical loss at each iteration can be characterized by convergence rate analysis , yet generalization properties are not well understood . To characterize the generalization error , one popular way is to study the algorithmic stability . Algorithmic stability is first considered by Kearns and Ron ( 1999 ) ; Bousquet and Elisseeff ( 2001 , 2002 ) . Later , Hardt et al . ( 2016 ) explored the connection between algorithmic stability and generalization performance of gradient-type optimization . Some follow-up research studies the stability for different classes of algorithms , or relax the definition of stability to generalize its usage , see Ramezani-Kebrya et al . ( 2018 ) ; Charles and Papailiopoulos ( 2018 ) ; Kuzborskij and Lampert ( 2018 ) ; Zhou et al . ( 2018 ) ; Lei and Ying ( 2020 ) ; Ho et al . ( 2020 ) ; Madden et al . ( 2020 ) . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In general , there are two ways to utilize algorithmic stability . On the one hand , as showed by Hardt et al . ( 2016 ) , the algorithmic stability provides an upper bound for the generalization error ; hence it will be useful when establishing the convergence of generalization error . On the other hand , the algorithmic stability itself is also a measure that evaluates the performance of an algorithm . Our work extends algorithmic stability analysis to adversarial training . Our contributions are : • Through figuring out the stability upper bound and lower bound , we argue that adversarial training leads to poor algorithmic stability even the clean loss is smooth . To solve this problem , we propose to inject noise into the adversarial training process . Although some existing works proposed the usage of noise injection , we highlight that such a method is more meaningful in adversarial training than its natural counterpart . Theoretical justification of the noise injection method is provided for a wide range of data-generating models in several tasks , including both linear regression and logistic classification . • Noticing that , in practice , adversarial attacks are mostly approximated via numerical methods , e.g. , fast gradient method ( FGM ) or projected gradient method ( PGD ) , our theory investigates the role of accuracy of attack approximation for the stability of adversarial training algorithms . In short , a more accurate attack leads to better stability upper bound . • The effectiveness of noise-injected adversarial training is further generalized to the L1 attack . Compared with L2 , L1 training algorithm is generally less stable . • Beyond the theoretical analysis under simple models , we provide a theory in two-layer ReLU network with lazy training ( training the hidden layer ) and observe the effectiveness of the noise injection method . We also obtain empirical evidence that for deep neural networks model , proper forms of noise injection and more accurate attack calculation ( e.g. , PGD-k over FGM ) improve the generalization error . 2 Related Works . Theories in adversarial training To theoretically understand how adversarial training works , Sinha et al . ( 2018 ) ; Wang et al . ( 2019a ) investigated the convergence of adversarial training when the loss is strongly convex w.r.t . data attributes . In this case , it can be shown that the gradient of adversarial loss w.r.t . model parameters is Lipschitz , leading to good stability . However , when the loss is not strongly concave in data attributes , Xing et al . ( 2021a ) figured out that adversarial training does not have a Lipschitz gradient even for linear regression . Some studies in deep neural networks ( Gao et al. , 2019 ; Zhang et al. , 2020 ; Allen-Zhu and Li , 2020 ) studied the convergence of adversarial training loss , and Allen-Zhu and Li ( 2020 ) also provided a theoretical guarantee of the adversarial testing loss when attack strength is small enough . Some other literature in the generalization of adversarial training can be found in Khim and Loh ( 2018 ) ; Awasthi et al . ( 2020 ) ; Pinot et al . ( 2021 ) ; Xing et al . ( 2021b ) . Observations in deep learning In terms of empirical studies , He et al . ( 2019 ) ; Zheltonozhskii et al . ( 2020 ) ; Xie et al . ( 2020 ) ; Lee and Chandrakasan ( 2020 ) ; Wu et al . ( 2020 ) focused on improving the performance of attack/adversarial robustness in deep learning . In the literature , there are several ways to improve the adversarial training , including modifying the objective function to help the convergence of the training process ( Zhang et al. , 2019 ; Wang et al. , 2019b ) , regularization ( He et al. , 2019 ; Zheltonozhskii et al. , 2020 ; Wu et al. , 2020 ) , replacing nonsmooth components ( Lee and Chandrakasan , 2020 ; Xie et al. , 2020 ) , and handling over-fitting issue ( Lee and Chandrakasan , 2020 ) . Stability for non-smooth loss and min-max problem Besides works in the algorithmic stability of first-order optimization methods on smooth loss , Bassily et al . ( 2020 ) studied the case when loss is convex but not smooth . In this scenario , the minimax lower bound and convergence upper bound of stability together imply that stochastic gradient descent1 ( SGD ) and gradient descent ( GD ) have poor stability . It is recommended to run SGD/GD with an extremely small learning rate for a vast number of iterations , which implies that it is not practical to train a non-smooth model with good stability . Consequently , adaptations are essential for non-smooth models to improve the training process . 1We are considering sample-with-replacement SGD . Another recent work , Farnia and Ozdaglar ( 2020 ) , considered the algorithmic stability in the minmax problems for generative adversarial networks ( GAN ) to argue that simultaneous training in generator and discriminator leads to good stability . However , besides the strongly-convex-concave assumption in their loss , the “ min-max ” problem considered in GAN and adversarial training are not the same . These two differences lead to different conclusions between GAN and adversarial training . 3 Stability of adversarial training . In this section , we study the uniform argument stability ( UAS ) of adversarial training . Utilizing the notations introduced in Section 3.1 , we present in Section 3.2 the upper and lower bounds of UAS . Section 3.3 studies the effect of attack error on stability . 3.1 Notations . Adversarial training To introduce adversarial training , let l denote the loss function and f✓ ( x ) be the model with parameter ✓ . The ( population ) adversarial loss is defined as R ( ✓ , ✏ ) : = E [ l ( f✓ [ x+A✏ ( f✓ , x , y ) ] , y ) ] , where A✏ is an attack of strength ✏ > 0 and intends to deteriorate the loss in the following way A✏ ( f✓ , x , y ) : = argmax z2Bp ( 0 , ✏ ) { l ( f✓ ( x+ z ) , y ) } , ( 1 ) where Bp ( x , r ) is a Lp ball centering at x with radius r. Given n i.i.d . samples S = { ( xi , yi ) } ni=1 , the adversarial training minimizes the sample version of R ( ✓ , ✏ ) w.r.t . ✓ : RS ( ✓ , ✏ ) = 1 n nX i=1 l ( f✓ [ xi +A✏ ( f✓ , xi , yi ) ] , yi ) , ( 2 ) and the estimator b✓ aims to minimize RS ( ✓ , ✏ ) . We rewrite RS ( ✓ , ✏ ) as RS ( ✓ ) for simplicity when there is no confusion . The minimization in ( 2 ) is often implemented through an iterative two-step ( min-max ) update . In the t-th iteration , we calculate the adversarial sample ex ( t ) i = xi + A✏ ( f✓ ( t ) , xi , yi ) based on the current ✓ ( t ) , and then update ✓ ( t+1 ) based on the gradient of the adversarial training loss while fixing ex ( t ) i ’ s with learning rate ⌘t . The algorithm runs for T iterations . A more detailed pseudocode is postponed to Algorithm 1 when introducing our adaptations . Note that for some loss function l or model f✓ , there may not be an analytic form for A✏ ( e.g . deep neural networks ) , and numerical methods , e.g . FGM and PGD , are utilized to approximate A✏ . Risk decomposition Define ✓0 and ✓̄ as the minimizer of R and RS respectively . Then for the algorithm output b✓ , the excess risk can be decomposed into four parts as below : R ( b✓ ) R ( ✓0 ) = R ( b✓ ) RS ( b✓ ) | { z } Egen +RS ( b✓ ) RS ( ✓̄ ) | { z } Eopt +RS ( ✓̄ ) RS ( ✓0 ) | { z } 0 +RS ( ✓0 ) R ( ✓0 ) | { z } E=0 , Since the last two parts are either negative or with zero expectation , we mainly focus on the first two parts , namely , generalization error R ( b✓ ) RS ( b✓ ) and optimization error RS ( b✓ ) RS ( ✓̄ ) . Based on Hardt et al . ( 2016 ) , Egen is upper bounded by algorithmic stability . Uniform argument stability ( UAS ) UAS aims to quantify the output sensitivity in L2 norm w.r.t an arbitrary change in a single data point . An algorithm is -UAS if for neighboring datasets S1 ⇠ S2 ( i.e. , S1 and S2 differ only in a single data point ) , it satisfies that sup S1⇠S2 kb✓ ( S1 ) b✓ ( S2 ) k : = sup S1⇠S2 ( S1 , S2 ) . where k·k represents the L2 norm . Under proper conditions , the UAS bound implies a generalization error bound ( Bassily et al. , 2020 ) : if P ( k ( S1 , S2 ) k ) 0 for any neighboring ( S1 , S2 ) , then for any , P `` |Egen| c ( log n ) ( log ( n/ ) ) + r log ( 1/ ) n ! # + 0 . ( 3 )

Summary: This paper studies the theory of adversarial examples from the angle of algorithmic stability. More precisely, the authors characterize the algorithmic stability of the now famous adversarial training scheme. Their main findings can be summarized as follows.  1) Even if the loss function utilized during adversarial training is smooth, the adversarial perturbation applied to the input image causes some non-differentiability issues which impede guaranteeing a good algorithmic stability in the worst-case scenario. 2) The numerical approximation error of adversarial attacks used during training can have an impact on the algorithmic stability of the scheme.  3) Injecting noise to the learning procedure helps to make the loss function smooth again; hence restoring the algorithmic stability with high probability.