Formalizing Generalization and Adversarial Robustness of Neural Networks to Weight Perturbations

1 Introduction . Neural network is currently the state-of-the-art machine learning model in a variety of tasks , including computer vision , natural language processing , and game-playing , to name a few . In particular , feedforward neural networks consists of layers of trainable model weights and activation functions with the premise of learning informative data representations and the complex mapping between data samples and the associated labels . Albeit attaining superior performance , the need for studying the sensitivity of neural networks to weight perturbations is also intensifying owing to several practical motivations . For instance , in model compression , the robustness to weight quantization is crucial for designing energy-efficient hardware accelerator [ 20 ] and for reducing memory storage while retaining model performance [ 9 , 26 ] . The notion of weight perturbation sensitivity is also used as a property to reflect the generalization gap at local minima [ 10 , 15 ] . In adversarial robustness and security , weight sensitivity can be leveraged as a vulnerability for fault injection and causing erroneous prediction [ 11 , 32 ] . However , while weight sensitivity plays an important role in many machine learning tasks and problem setups , theoretical characterization of its impacts on generalization and robustness of neural networks remains elusive . This paper bridges this gap by developing a novel theoretical framework for understanding the generalization gap ( through Rademacher complexity ) and the robustness ( through classification margin ) of neural networks against norm-bounded weight perturbations . Specifically , we consider 1Work partly done in TCFSH 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . the multi-class classification problem setup and multi-layer feed-forward neural networks with non-negative monotonic activation functions . Our analysis offers fundamental insights into how weight perturbation affects the generalization gap and the pairwise class margin . To the best of our knowledge , this study is the first work that provides a comprehensive theoretical characterization of the interplay between weight perturbation , robustness in classification margin , and generalization gap . Moreover , based on our analysis , we propose a theory-driven loss function for training generalizable and robust neural networks against norm-bounded weight perturbations . We validate its effectiveness via empirical experiments . Our main contributions are summarized as follows . • We study the robustness ( worst-case bound ) of the pairwise class margin function against weight perturbations in neural networks , including the analysis of single-layer ( Theorem 1 ) , all-layer ( Theorem 2 ) , and selected-layer ( Theorem 3 ) weight perturbations . • We characterize the generalization behavior of robust surrogate loss for neural networks under weight perturbations ( Section 3.4 ) through Rademacher complexity ( Theorem 4 ) . • We propose a theory-driven loss design for training generalizable and robust neural networks ( Section 3.5 ) . The empirical results in Section 4 validate our theoretical analysis and demonstrate the effectiveness of improving generalization and robustness against weight perturbations . We also show that in our studied setting the associated generalization bounds are non-vacuous . 2 Related Works . 2.1 Generalization in Standard Settings . In model compression , the robustness to weight quantization is critical to reducing memory size and accesses for low-precision inference and training [ 9 ] . [ 20 ] showed that neural models that are robust to random bit flipping errors are beneficial for operating in low-voltage regime and thus improving energy efficiency for hardware implementation . [ 26 ] showed that incorporating weight perturbation sensitivity into training can better retain model performance ( standard accuracy ) after quantization . For studying the generalization of neural networks , [ 10 ] proposed a metric called sharpness ( or weight sensitivity ) by perturbing the learned model weights around the local minima of the loss landscape for generalization assessment . Similar concepts of considering nearby parameter loss is as well presented in [ 18 ] where this concept is to circumvent from falling into sharp empirical loss pit with high generalization gap . [ 1 ] introduced weight noise into the training process and concluded that random noise training improves the overall generalization . [ 27 ] empirically showed that using adversarial weight perturbation during training can improve generalization against input perturbations . [ 5 ] obtained non-vacuous generalization bound on deep ( stochastic ) neural networks by directly optimizing the PAC-Bayes bound while [ 16 ] as well combined weight perturbation with PAC-Bayes analysis to derive the generalization bound . [ 15 ] made a connection between sharpness and PAC-Bayes theory and found that some combination of sharpness and norms on the model weights may capture the generalization behavior of neural networks . Additionally , [ 4 ] discovered normalized margin measure to be useful towards quantifying generalization property and a bound was therefore constructed to give an quantitative description on the generalization gap . Moreover , [ 7 ] incorporated additional assumptions to offer tighter and size-independent bounds from the setting of [ 17 ] and [ 4 ] respectively . We note that previous methods use random weight perturbations to evaluate the generalization ability ( e.g . sharpness ) , which we call a standard generalization setting . In contrast , we study a different problem setup : generalization of models trained under worst-case ( adversarial ) weight perturbation . Despite development of various generalization bounds , empirical observations in [ 14 ] showed that once the size of the training dataset grows , generalization bounds proposed in [ 15 ] and [ 4 ] will enlarge and thus become vacuous . Discussions on the relation between [ 14 ] and our works can be found at Appendix E , where we show that in our studied setting the associated generalization bounds are non-vacuous . On the other hand , [ 3 ] and [ 22 ] applied several techniques in tandem with a probabilistic method named path sampling to construct a representing set of given neural networks for approximating and studying the generalization property . Another approach considered by [ 19 ] consists of segmenting the neural networks into two functions , predictor and feature selection respectively , where two measures ( representativeness and feature robustness ) concerning these aforementioned functions were later combined to offer a meaningful generalization bound . However , these works only focused on the generalization behavior of the local minima and did not consider the generalization and robustness under weight perturbations . [ 26 ] proposed a certification method for weight perturbation retaining consistent model prediction . While the certification bound can be used to train robust models with interval bound propagation [ 8 ] , it requires additional optimization subroutine and computation costs when comparing to our approach . Moreover , the convoluted nature of certification bound complicates the analysis when studying generalization , whereas our theoretical characterization is an intuitive and tight worst-case analysis . 2.2 Adversarial Robustness and Input/Weight Perturbation . In adversarial robustness , fault-injection attacks are known to inject errors to model weights at the inference phase and causing erroneous model prediction [ 11 , 32 ] , which can be realized at the hardware level by changing or flipping the logic values of the corresponding bits and thus modifying the model parameters saved in memory [ 2 , 24 ] . [ 31 ] proposed to use the mode connectivity of the model parameters in the loss landscape for mitigating such weight-perturbation-based adversarial attacks . Although , to the best of our knowledge , theoretical characterization of generalization and robustness for neural networks against weight perturbations remains elusive , recent works have studied these properties under another scenario — input perturbations . Both empirical and theoretical evidence suggests the existence of a fundamental trade-off between generalization and robustness against norm-bounded input perturbations [ 28 , 21 , 30 , 23 ] . The adversarial training proposed in [ 12 ] is a popular training strategy for training robust models against input perturbations , where a minmax optimization principle is used to minimize the worst-case input perturbations of a data batch during model parameter updates . For adversarial training with input perturbations , [ 25 ] proved its convergence and [ 29 ] derived bounds on its Rademacher complexity for generalization . Different from the case of input perturbation , we note that min-max optimization on neural network training subject to weight perturbation is not straightforward , as the minimization and maximization steps are both taken on the model parameters . In this paper , we disentangle the min-max formulation for weight perturbation by developing bounds for the inner maximization step and provide quantifiable metrics for training generalizable and robust neural networks against weight perturbations . 2.3 Comparison against Related Generalization Bound . In the following subsection below we will provide the discussion to our characterization of generalization bound against similar notions in other related works . For the convenience of comparison , we formulated it as a table in Appendix H. In what follows we offer the detailed comparison on different settings , equation setup between ours and these related notions . In [ 6 ] , the authors extended the result of sharpness [ 15 ] and integrated this concept as a part of the training process , namely the sharpness-aware minimization . In the paper , the author instead of directly solving the inner maximization loop , uses linear approximation to resolve the issue . Furthermore , the generalization bound given in [ 6 ] considers only the standard setting , in which the generalization gap is measured between population risk and the sharpness-aware sample risk , differing from our adversarial setting where the generalization gap is contrast between both adversarial population and sample risk . On the other hand , the authors in [ 27 ] propose to utilize weight perturbation to mitigate the widening robust generalization gap ( based on input robustness ) . In [ 27 ] , the author trains the model by injecting the so-called “ double perturbation ” into the training process . We note that this is different from our approach of constructing adversarial robustness on weight space . Specifically , the weight perturbation in [ 27 ] ( second maximization step in equation ( 7 ) , [ 27 ] ) is determined after every corresponding adversarial example is calculated , while our approach focuses on the worst-case weight perturbation given every sample data . Moreover , the generalization gap extends from [ 15 ] where it is measured against the population risk and the sample risk along with perturbation direction sampled from zero mean spherical Gaussian distribution . On the contrary , our setting considers a generalization gap from both the adversarial population risk and sample risk . 3 Main Results . We provide an overview of the presentation flow for our main results as follows . First , we introduce the mathematical notations and preliminary information in Section 3.1 . In Section 3.2 , we establish our weight perturbation analysis on a simplified case of single-layer perturbation . We then use the single-layer analysis as a building block and extend the results to the multi-layer perturbation setting in Section 3.3 . Here we would like to note that similar reasoning could be applied to obtain results in convolutional neural networks and we relay the corresponding results and experiments in Appendix A.4 , respectively . In Section 3.4 , we define the framework of robust training with surrogate loss and study the generalization property using Rademacher complexity . Finally , we propose a theory-driven loss toward training robust and generalizable neural networks in Section 3.5 . 3.1 Notation and Preliminaries . Notation We start by introducing the mathematical notations used in this paper . We define the set [ L ] : = { 1 , 2 , ... , L } . For any two non-empty sets A , B , FA 7→B denotes the set of all functions from A to B . We mark the indicator function of an event E as 1 ( E ) , which is 1 if E holds and 0 otherwise . We use sgn ( · ) to denote element-wise sign function that outputs 1 when input is nonnegative and -1 otherwise . Boldface lowercase letters are used to denote vectors ( e.g. , x ) , and the i-th element is denoted as [ x ] i. Matrices are presented as boldface uppercase letters , say W. Given a matrix W ∈ Rk×d , we write its i-th row , j-th column and ( i , j ) element as Wi , : , W : ,j , and Wi , j respectively . Moreover , we write its transpose matrix as ( W ) T . The matrix ( p , q ) norm is defined as ‖W‖p , q : = ∥∥∥ [ ‖W : ,1‖p , ‖W : ,2‖p , ... , ‖W : ,d‖p ] ∥∥∥ q for any p , q ≥ 1 . For convenience , we have ‖W‖p = ‖W‖p , p and write the spectral norm and Frobenius norm as ‖W‖σ and ‖W‖F respectively . We mark one matrix norm commonly used in this paper – the matrix ( 1 , ∞ ) norm . With a matrix W , we express its matrix ( 1 , ∞ ) norm as ‖W‖1 , ∞ , which is defined as ‖W‖1 , ∞ = maxj ‖W : ,j‖1 and ∥∥WT∥∥ 1 , ∞ = maxi ‖Wi , :‖1 . We use IB ∞ W ( ) to express an element-wise ` ∞ norm ball of matrix W within radius , i.e. , IB∞W ( ) = { Ŵ| |Ŵi , j −Wi , j | ≤ , ∀i ∈ [ k ] , j ∈ [ d ] } . Preliminaries In order to formally explain our theoretical results , we introduce the considered learning problem , neural network model , and complexity definition . Let X and Y be the feature space and label space , respectively . We place the assumption that all data are drawn from an unknown distribution D over X × Y and each data point is generated under i.i.d condition . In this paper , we specifically consider the feature space X as a subset of d-dimensional Euclidean space , i.e. , X ⊆ Rd . We denote the symbol F ⊆ FX 7→Y to be the hypothesis class which we use to make predictions . Furthermore , we consider a loss function ` : X × Y −→ [ 0 , 1 ] and compose it with the hypothesis class to make a function family written as ` F : = { ( x , y ) 7→ ` ( f ( x ) , y ) | f ∈ F } . The optimal solution of this learning problem is a function f∗ ∈ F such that it minimizes the population risk R ( f ) = E ( x , y ) ∼D [ ` ( f ( x ) , y ) ] . However , since the underlying data distribution is generally unknown , one typically aims at reducing the empirical risk evaluated by a set of training data { ( xi , yi ) } ni=1 , which can be expressed as Rn ( f ) = 1n ∑n i=1 ` ( f ( xi ) , yi ) . The generalization error is the gap between population and empirical risk , which could serve as an indicator of model ’ s performance under unseen data from identical distribution D. To study the generalization error , one would explore the learning capacity of a certain hypothesis class . In this paper , we adopt the notion of Rademacher complexity as a measure of learning capacity , which is widely used in statistical machine learning literature [ 13 ] . The empirical Rademacher complexity of a function class F given a set of samples S = { ( xi , yi ) } ni=1 is RS ( ` F ) = Eν [ supf∈F 1n ∑n i=1 νi ` ( f ( xi ) , yi ) ] where { νi } ni=1 is a set of i.i.d Rademacher random variables with P { νi = −1 } = P { νi = +1 } = 12 . The empirical Rademacher complexity measures on average how well a function class F correlates with random noises on dataset S. Thus , a richer or more complex family could better correlate with random noise on average . With Rademacher complexity as a toolkit , one can develop the following relationship between generalization error and complexity measure . Specifically , it is shown in [ 13 ] that given a set of training samples S and assume that the range of loss function ` ( f ( x ) , y ) is [ 0 , 1 ] . Then for any δ ∈ ( 0 , 1 ) , with at least probability 1 − δ we have ∀f ∈ F , R ( f ) ≤ Rn ( f ) + 2RS ( ` F ) + 3 √ log 2δ 2n Note that when the Rademacher complexity is small , it is then viable to learn the hypothesis class F by minimizing the empirical risk and thus effectively reducing the generalization gap . Finally , we define the structure of neural networks and introduce a few related quantities . The problem studied in this paper is a multi-class classification task with the number of classes being K. Consider an input vector x ∈ X ⊆ Rd , an L-layer neural network is defined as fW ( x ) = WL ( ... ρ ( W1x ) ... ) ∈ FX 7→R with W being the set containing all weight matrices , i.e. , W : = { Wk| ∀k ∈ [ L ] } , and the notation ρ ( · ) is used to express any non-negative monotone activation function and we further assume that ρ ( · ) is 1-Lipschitz , which includes popular activation functions such as ReLU applied elementwise on a vector . Moreover , the i-th component of neural networks ’ output is written as f iW ( x ) = [ fW ( x ) ] i and a pairwise margin between i-th and j-th class , denoted as f ijW ( x ) : = f j W ( x ) −f j W ( x ) , is said to be the difference between two classes in output of the neural network . Lastly , we use the notion of zk and ẑk to represent the output vector of the k-th layer ( k ∈ [ L− 1 ] ) under natural and weight perturbed settings respectively , which are zk = ρ ( Wk ( ... ρ ( W1x ) ... ) ) and ẑk = ρ ( Ŵ k ( ... ρ ( Ŵ 1 x ) ... ) ) , where Ŵ i ∈ IB∞Wi ( i ) denotes the perturbed weight matrix bounded by its element-wise ` ∞-norm with radius i for some i ∈ [ k ] . Throughout this paper , we study the weight perturbation setting that each layer has an independent layer-wise perturbation budget , denoted by k , ∀k ∈ [ L ] . In the case of single-layer perturbation , we may omit the layer index and simply use .

Summary: The paper studies the worst-case perturbations in the weight-space for multi-layer networks. The authors first derive a robust surrogate loss which is an upper bound on the loss under norm-bounded weight perturbations based on a product of the 1-inf matrix norm of layer weight matrices. Then the authors derive a generalization bound based on the Rademacher complexity which motivates their new loss function which is evaluated empirically on MNIST.