Do Wider Neural Networks Really Help Adversarial Robustness?

Adversarial training is a powerful type of defense against adversarial examples . Previous empirical results suggest that adversarial training requires wider networks for better performances . However , it remains elusive how does neural network width affect model robustness . In this paper , we carefully examine the relationship between network width and model robustness . Specifically , we show that the model robustness is closely related to the tradeoff between natural accuracy and perturbation stability , which is controlled by the robust regularization parameter λ . With the same λ , wider networks can achieve better natural accuracy but worse perturbation stability , leading to a potentially worse overall model robustness . To understand the origin of this phenomenon , we further relate the perturbation stability with the network ’ s local Lipschitzness . By leveraging recent results on neural tangent kernels , we theoretically show that wider networks tend to have worse perturbation stability . Our analyses suggest that : 1 ) the common strategy of first fine-tuning λ on small networks and then directly use it for wide model training could lead to deteriorated model robustness ; 2 ) one needs to properly enlarge λ to unleash the robustness potential of wider models fully . Finally , we propose a new Width Adjusted Regularization ( WAR ) method that adaptively enlarges λ on wide models and significantly saves the tuning time . 1 Introduction . Researchers have found that Deep Neural Networks ( DNNs ) suffer badly from adversarial examples [ 59 ] . By perturbing the original inputs with an intentionally computed , undetectable noise , one can deceive DNNs and even arbitrarily modify their predictions on purpose . To defend against adversarial examples and further improve model robustness , various defense approaches have been proposed [ 48 , 42 , 18 , 40 , 67 , 26 , 58 , 55 ] . Among them , adversarial training [ 23 , 41 ] has been shown to be the most effective type of defenses [ 5 ] . Adversarial training can be seen as a form of data augmentation by first finding the adversarial examples and then training DNN models on those examples . Specifically , given a DNN classifier f parameterized by θ , a general form of adversarial training with loss function ∗Equal contribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . L can be defined as : argmin θ 1 N N∑ i=1 [ L ( θ ; xi , yi ) ︸ ︷︷ ︸ natural risk +λ · max x̂i∈B ( xi , ) [ L ( θ ; x̂i , yi ) − L ( θ ; xi , yi ) ] ︸ ︷︷ ︸ robust regularization ] , ( 1.1 ) where { ( xi , yi ) ni=1 } are training data , B ( x , ) = { x̂ | ‖x̂ − x‖p ≤ } denotes the ` p norm ball with radius centered at x , and p ≥ 1 , and λ > 0 is the regularization parameter . Compared with standard empirical risk minimization , the extra robust regularization term encourages the data points within B ( x , ) to be classified as the same class , i.e. , encourages the predictions to be stable . The regularization parameter λ adjusts the strength of robust regularization . When λ = 1 , it recovers the formulation in [ 41 ] , and when λ = 0.5 , it recovers the formulation in [ 23 ] . Furthermore , replacing the loss difference in robust regularization term with the KL-divergence based regularization recovers the formulation in [ 70 ] . One common belief in the practice of adversarial training is that , compared with the standard empirical risk minimization , adversarial training requires much wider neural networks to achieve better robustness . [ 41 ] provided an intuitive explanation : robust classification requires a much more complicated decision boundary , as it needs to handle the presence of possible adversarial examples . However , it remains elusive how the network width affects model robustness . To answer this question , we first examine whether the larger network width contributes to both the natural risk term and the robust regularization term in ( 1.1 ) . Interestingly , when tracing the value changes in ( 1.1 ) during adversarial training , we observe that the value of the robust regulariza- tion part actually gets worse on wider models , suggesting that larger network width does not lead to better stability in predictions . In Figure 1 , we show the loss value comparison of two different wide models trained by TRADES [ 70 ] with λ = 6 as suggested in the original paper . We can see that the wider model ( i.e. , WideResNet-34-10 ) achieves better natural risk but incurs a larger value on robust regularization . This motivates us to find out the cause of this phenomenon . In this paper , we study the relationship between neural network width and model robustness for adversarially trained neural networks . Our contributions can be summarized as follows : 1 . We show that the model robustness is closely related to both natural accuracy and perturbation stability , a new metric we proposed to characterize the strength of robust regularization . The balance between the two is controlled by the robust regularization parameter λ . With the same value of λ , the natural accuracy is improved on wider models while the perturbation stability often worsens , leading to a possible decrease in the overall model robustness . This suggests that proper tuning of λ on wide models is necessary despite being extremely time-consuming , while directly using the fine-tuned λ on small networks to train wider ones , as many people did in practice [ 41 , 70 ] , may lead to deteriorated model robustness . 2 . Unlike previous understandings that there exists a trade-off between natural accuracy and robust accuracy , we show that the real trade-off should between natural accuracy and perturbation stability . And the robust accuracy is actually the consequence of this trade-off . 3 . To understand the origin of the lower perturbation stability of wider networks , we further relate perturbation stability with the network ’ s local Lipschitznesss . By leveraging recent results on neural tangent kernels [ 36 , 3 , 73 , 8 , 21 ] , we show that with the same value of λ , larger network width naturally leads to worse perturbation stability , which explains our empirical findings . 4 . Our analyses suggest that to unleash the potential of wider model architectures fully , one should mitigate the perturbation stability deterioration and enlarge robust regularization parameter λ for training wider models . Empirical results verified the effectiveness of this strategy on benchmark datasets . In order to alleviate the heavy burden for tuning λ on wide models , we develop the Width Adjusted Regularization ( WAR ) method to transfer the knowledge we gain from fine-tuning smaller networks into the training of wider networks and significantly save the tuning time . Notation . For a d-dimensional vector x = [ x1 , ... , xd ] > , we use ‖x‖p = ( ∑d i=1 |xi|p ) 1/p with p ≥ 1 to denote its ` p norm . 1 ( · ) represents the indicator function and ∀ represents the universal quantifier . 2 Related Work . Adversarial attacks : Adversarial examples were first found in [ 59 ] . Since then , tremendous work have been done exploring the origins of this intriguing property of deep learning [ 25 , 38 , 20 , 60 , 22 , 72 ] as well as designing more powerful attacks [ 23 , 47 , 43 , 41 , 9 , 11 ] under various attack settings . [ 5 ] identified the gradient masking problem and showed that many defense methods could be broken with a few changes on the attacker . [ 13 ] proposed gradient-free black-box attacks and [ 32 , 33 , 12 ] further improved its efficiency . Recently , [ 34 , 35 ] pointed out that adversarial examples are generated from the non-robust or invariant features hidden in the training data . Defensive adversarial learning : Many defense approaches have been proposed to directly learn a robust model that can defend against adversarial attacks . [ 41 ] proposed a general framework of robust training by solving a min-max optimization problem . [ 62 ] proposed a new criterion to evaluate the convergence quality quantitatively . [ 70 ] theoretically studied the trade-off between natural accuracy and robust accuracy for adversarially trained models . [ 63 ] followed this framework and further improved its robustness by differentiating correctly classified and misclassified examples . [ 14 ] solve the problem by restricting the variation of outputs with respect to the inputs . [ 15 , 54 , 39 ] developed provably robust adversarial learning methods that have the theoretical guarantees on robustness . Recent works in [ 65 , 50 ] focus on creating adversarial robust networks with faster training protocol . Another line of works focuses on increasing the effective size of the training data , either by pre-trained models [ 30 ] or by semi-supervised learning [ 10 , 1 , 44 ] . Very recently , [ 66 ] proposed to conduct adversarial weight perturbation aside from input perturbation to obtain more robust models . [ 24 ] achieves further robust models by practical techniques like weight averaging . Robustness and generalization : Earlier works like [ 23 ] found that adversarial learning can reduce overfitting and help generalization . However , as the arms race between attackers and defenses keeps going , it is observed that strong adversarial attacks can cause severe damage to the model ’ s natural accuracy [ 41 , 70 ] . Many works [ 70 , 61 , 52 , 19 ] attempt to explain this trade-off between robustness and natural generalization , while some other works proposed different perspectives . [ 56 ] confirmed that more training data has the potential to close this gap . [ 6 ] suggested that a robust model is computationally difficult to learn and optimize . [ 72 ] showed that there is still a large gap between the currently achieved model robustness and the theoretically achievable robustness limit on natural image distributions . [ 2 ] showed that the adversarial examples stem from the accumulation of small dense mixtures in the hidden weights during training and adversarial training works by removing such mixtures . Very recently , [ 51 ] showed that this trade-off stems from overparameterization and insufficient data in the linear regression setting . [ 68 ] proved that both accuracy and robustness are achievable through locally Lipschitz functions with separated data , and the gap between theory and practice is due to either failure to impose local Lipschitzness or insufficient generalization . [ 7 ] also studied the relationship between robustness and network size . In particular , [ 7 ] shows that overparametrization is necessary for robustness on two-layer neural networks , while we show that when networks get wider , they will have worse perturbation stability , and therefore larger regularization is needed to achieve better robustness . 3 Empirical Study on Network Width and Adversarial Robustness . In this section , we empirically study the relation between network width and robustness by first taking a closer look at the robust accuracy and the associated robust examples . 3.1 Characterization of Robust Examples . Robust accuracy is the standard evaluation metric of robustness , which measures the ratio of robust examples , i.e. , examples that can still be correctly classified after adversarial attacks . Previous empirical results suggest that wide models enjoy both better generalization ability and model robustness . Specifically , [ 41 ] proposed to extend ResNet [ 28 ] architecture to WideResNet [ 69 ] with a widen factor 10 for adversarial training on the CIFAR10 dataset and found that the increased model capacity significantly improves both robust accuracy and natural accuracy . Later works [ 70 , 63 ] follow this finding and report their best result using the wide networks . However , as shown by our findings in Figure 1 , wider models actually lead to worse robust regularization effects , suggesting that wider models are not better in all aspects , and the relation between model robustness and network width may be more intricate than what people understood previously . To understand the intrinsic relationship between model robustness and network width , let us first take a closer look at the robust examples . Mathematically , robust examples can be defined as Srob : = { x : ∀x̂ ∈ B ( x , ) , f ( θ ; x̂ ) = y } . Note that by definition of robust examples , we have the following equation holds : { x : ∀x̂ ∈ B ( x , ) , f ( θ ; x̂ ) = y } ︸ ︷︷ ︸ robust examples : Srob = { x : f ( θ ; x ) = y } ︸ ︷︷ ︸ correctly classified examples : Scorrect ∧ { x : ∀x̂ ∈ B ( x , ) , f ( θ ; x ) = f ( θ ; x̂ ) } , ︸ ︷︷ ︸ stable examples : Sstable ( 3.1 ) Figure 2 : An illustration of robust , correctly classified , and stable examples in ( 3.1 ) . where ∧ is the logical conjunction operator . ( 3.1 ) suggests that the robust examples are the intersection of two other sets : the correctly classified examples ( examples whose predictions are the correct labels ) and the stable examples ( examples whose predictions are the same within the ` p norm ball ) . A more direct illustration of this relationship can be found in Figure 2 . While the natural accuracy measures the ratio of correctly classified examples |Scorrect| against the whole sample set , to our knowledge , there does not exist a metric measuring the ratio of stable examples |Sstable| against whole the sample set . Here we formally define this ratio as the perturbation stability , which measures the fraction of examples whose predictions can not be perturbed as reflected in the robust regularization term in ( 1.1 ) .

Summary: The paper studies the connection between adversarial robustness and model capacity in the form of width. The key idea is to decompose robust accuracy into standard accuracy and stability---i.e., on what fraction of test inputs the prediction of the model cannot be changed by an adversary.  The authors find that while increased width tends to improve robust accuracy (as has also been previously observed) the gain are mostly coming from an improvement in the standard (test) accuracy of the model, while the stability of wider models is actually _decreased._ To counteract this phenomenon, they suggest increasing the regularization term corresponding to stability as the width of the model increases and indeed observe empirical performance improvements. The authors further support their hypothesis by theoretically analyzing the effect of width on robustness in an NTK setting and observing similar results. Finally, they propose a heuristic scheme for adjusting the stability regularization parameter during training and demonstrate promising results.