Formalizing Generalization and Adversarial Robustness of Neural Networks to Weight Perturbations
Authors: Yu-Lin Tsai
Chia-Yi Hsu
Chia-Mu Yu
Pin-Yu Chen
Venue: NeurIPS 2021 Poster (2021)
Keywords: Generalization under weight perturbation
Robustness
Neural Network
Weight Perturbation
Rademacher complexity

Abstract: Studying the sensitivity of weight perturbation in neural networks and its impacts on model performance, including generalization and robustness, is an active research topic due to its implications on a wide range of machine learning tasks such as model compression, generalization gap assessment, and adversarial attacks. In this paper, we provide the first integral study and analysis for feed-forward neural networks in terms of the robustness in pairwise class margin and its generalization behavior under weight perturbation. We further design a new theory-driven loss function for training generalizable and robust neural networks against weight perturbations. Empirical experiments are conducted to validate our theoretical analysis. Our results offer fundamental insights for characterizing the generalization and robustness of neural networks against weight perturbations.

Formalizing Generalization and Adversarial Robustness of Neural Networks to Weight Perturbations
Yu-Lin Tsai
Chia-Yi Yi Hsu
National Yang Ming Chiao Tung University uriah1001 @ gmmil.com"
National Yang Ming Chiao Tung University
chiayihsu83 @ @enyyissuB15@@mail.comm
Chia-Mu Yu
Pin- Yu Chen
National Yang Ming Chiao Tung University Siamuuu@ggaill.com
IBM Research
pin-yu.chen www..cheennimmmcmmmmmmm
Abstract
Studying the sensitivity of weight perturbation in neural networks and its impacts model performance, including generalization and robustness, is active re- search topic due its implications on wide range of machine learning tasks such model compression, generalization gap assessment, and adversarial attacks. In this paper, we provide the first integral study and analysis for feed-forward neural networks in terms of the robustness in pairwise class margin and its generalization behavior under weight perturbation.
further design new theory-driven loss function for training generalizable and robust neural networks against weight per- turbations. Empirical experiments are conducted to validate our thcoretical analysis. Our results offer fundamental insights for characterizing the generalization and robustness of neural networks against weight perturbations.
Introduction
Neural network is currently the state-of-the-art machine learning model in variety of tasks, including computer vision, natural language processing, and game-playing, to name a few. In particular, feed- forward neural networks consists of layers of trainable model weights and activation functions with premise of learning informative data representations and the complex mapping between data samples and the associated labels. Albeit attaining superior performance, the need for studying the sensitivity of neural networks to weight perturbations is also intensifying owing to several practical motivations. For instance, in model compression, the robustness weight quantization is crucial for designing energy-efficient hardware accelerator (20 and for reducing memory storage while retaining model performance 9 26 The notion of weight perturbation sensitivity also used a property reflect the generalization gap at local minima 10 15]. In adversarial robustness and security, weight sensitivity can be leveraged
vulnerability for fault injection and causing erroneous prediction 11 32 However, while weight sensitivity plays an important role in many machine learning tasks and problem setups, theoretical characterization of its impacts on generalization and robustness of neural networks remains elusive.
This paper bridges this gap by developing novel theoretical framework for understanding the generalization gap (through Rademacher complexity) and the robustness (through classification margin) of neural networks against norm-hounded weight perturbations. Specifically, consider
Work partly done in TCFSH
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

the multi-class classification problem setup and multi-layer feed-forward neural networks with non-negative monotonic activation functions. Our analysis offers fundamental insights into how weight perturbation affects the generalization gap and the pairwise class margin. To the best of our knowledge, this study is the first work that provides a comprehensive theoretical characterization of the interplay between weight perturbation, robustness in classification margin, and generalization gap. Moreover, based on our analysis, we propose a theory-driven loss function for training generalizable and robust neural networks against norm-bounded weight perturbations. We validate its effectiveness via empirical experiments. Our main contributions are summarized as follows.
We study the robustness (worst-case bound) of the pairwise class margin function against weight perturbations in neural networks, including the analysis of single-layer (Theorem [ all-layer (Theorem 2 and selected-layer (Theorem 3) weight perturbations.
We characterize the generalization behavior of robust surrogate loss for neural networks under weight perturbations (Section 3.4) through Rademacher complexity (Theorem 4
We propose a theory-driven loss design for training generalizable and robust neural networks (Section 3.5). The empirical results in Section validate our theoretical analysis and demonstrate the effectiveness of improving generalization and robustness against weight perturbations. We also show that in our studied setting the associated generalization bounds are non-vacuous.
2 Related Works
2.1 Generalization in Standard Settings
In model compression, the robustness to weight quantization is critical to reducing memory size and accesses for low-precision inference and training 19. 20 showed that neural models that are robust to random bit flipping errors are beneficial for operating in low-voltage regime and thus improving energy efficiency for hardware implementation. (26) showed that incorporating weight perturbation sensitivity into training can better retain model performance (standard accuracy) after quantization. For studying the generalization of neural networks, [10] proposed metric called sharpness (or weight sensitivity) by perturbing the learned model weights around the local minima of the loss landscape for generalization assessment. Similar concepts of considering nearby parameter loss is as well presented in [18] where this concept is to circumvent from falling into sharp empirical loss pit with high generalization gap. [1] introduced weight noise into the training process and concluded that random noise training improves the overall generalization. (27] empirically showed that using adversarial weight perturbation during training can improve generalization against input perturbations. (5) obtained non-vacuous generalization bound on deep (stochastic) neural networks by directly optimizing the PAC-Bayes bound while (16) well combined weight perturbation with PAC-Bayes analysis to derive the generalization bound. [15] made a connection between sharpness and PAC-Bayes theory and found that some combination of sharpness and norms on the model weights may capture the generalization behavior of neural networks. Additionally, 44 discovered normalized margin measure to be useful towards quantifying generalization property and bound was therefore constructed to give an quantitative description on the generalization gap. Moreover, [7] incorporated additional assumptions to offer tighter and size-independent bounds from the setting of 177 and 4 respectively. We note that previous methods use random weight perturbations to evaluate the generalization ability (e.g. sharpness), which we call a standard generalization setting. In contrast, we study different problem setup: generalization of models trained under worst-case (adversarial) weight perturbation.
Despite development of various generalization bounds, empirical observations in 14 showed that once the size of the training dataset grows, generalization bounds proposed in [15 and [4] will enlarge and thus become vacuous. Discussions on the relation between [14] and our works can be found at Appendix D where we show that in our studied setting the associated generalization bounds are non-vacuous. On the other hand, [3] and (22) applied several techniques in tandem with a probabilistic method named path sampling to construct a representing set of given neural networks for approximating and studying the generalization property. Another approach considered by 19 consists of segmenting the neural networks into two functions, predictor and feature selection respectively, where two measures (representativeness. and feature robustness) concerning these aforementioned functions were later combined to offer meaningful generalization bound. However, these works only focused on the generalization behavior of the local minima and did not consider
2

---

the generalization and robustness under weight perturbations. [26 proposed a certification method for weight perturbation retaining consistent model prediction. While the certification bound can be used to train robust models with interval bound propagation 8 it requires additional optimization subroutine and computation costs when comparing to our approach. Moreover, the convoluted nature of certification bound complicates the analysis when studying generalization, whereas our theoretical characterization is an intuitive and tight worst-case analysis.
2.2 Adversarial Robustness and Input/Weight Perturbation
In adversarial robustness, fault-injection attacks are known to inject errors to model weights at the inference phase and causing erroneous model prediction 32), which can be realized at the hardware level by changing or flipping the logic values of the corresponding bits and thus modifying
the model parameters saved in memory 2 24] [31 proposed to use the mode connectivity of the
model parameters in the loss landscape for mitigating such weight-perturbatioon based adversarial
attacks.
Although, to the best of our knowledge, theoretical characterization of generalization and robustness for neural networks against weight perturbations remains elusive, recent works have studied these properties under another scenario - imput perturbations. Both empirical and theoretical evidence suggests the existence of fundamental trade-off between generalization and robustness against norm-bounded input perturbations [28
The adversarial training proposed in [12] is popular training strategy for training robust models against input perturbations, where a min- max optimization principle is used to minimize the worst-case input perturbations of data batch during model parameter updates. For adversarial training with input perturbations, [25] proved its convergence and [2] derived bounds on its Rademacher complexity for generalization. Different from the case of input perturbation, we note that min-max optimization subject to weight perturbation is not straightforward, as the minimization and maximization steps both taken on the model parameters. In this paper, we disentangle the min-max formulation for weight perturbation by developing bounds for the inner maximization step and provide quantifiable metrics for training generalizable and robust neural networks against weight perturbations.
21
3D 2
neural network training
2.3 Comparison against Related Generalization Bound
In the following subsection below we will provide the discussion to our characterization of general- ization bound against similar notions in other related works. For the convenience of comparison, we formulated it as table in Appendix H In what follows we offer the detailed comparison on different settings, equation setup between ours and these related notions.
In (6) the authors extended the result of sharpness 15) and integrated this concept as part of the training process, namely the sharpness-awart minimization. In the paper, the author instead of directly solving the inner maximization loop. uses linear approximation to resolve the issue. Furthermore, the generalization bound given in (6] considers only the standard setting, in which the generalization gap is measured between population risk and the sharpness-awwe aware sample risk, differing from our adversarial setting where the generalization gap contrast between both adversarial population and sample risk.
On the other hand, the authors in 227 propose to utilize weight perturbation to mitigate the widening robust generalization gap (based on input robustness). In (27]. the author trains the model by injecting the so-called "double perturbation" into the training process. We note that this is different from our approach of constructing adversarial robustness on weight space. Specifically, the weight perturbation
in [27] (second maximization step in equation (7), (27) is determined after every corresponding adversarial example is calculated, while our approach focuses on the worst-case weight perturbation
given every sample data. Moreover, the generalization gap extends from [[5] where it is measured against the population risk and the sample risk along with perturbation direction sampled from zero mean spherical Gaussian distribution. On the contrary, our setting considers generalization gap from both the adversarial population risk and sample risk.
3

---

3 Main Results
We provide an overview of the presentation flow for our main results as follows. First, we introduce the mathematical notations and preliminary information Section 3.1 In Section 3.2 we establish our weight perturbation analysis on simplified case of single-layer perturbation. We then use the single-layer analysis as building block and extend the results to the multi-layer perturbation setting in Section 3.3 Here we would like to note that similar reasoning could be applied to obtain results in convolutional neural networks and we relay the corresponding results and experiments in Appendix A.4, respectively. In Section 3.4 we define the framework of robust training with surrogate loss and study the generalization property using Rademacher complexity. Finally, we propose a theory-driven loss toward training robust and generalizable neural networks in Section 3,5
3.1 Notation and Preliminaries
Notation We start by introducing the mathematical notations used in this paper. We define the set [4] == {1, ..., L). For any two non-empty sets A. B, IF A-+B denotes the set of all functions from A to B. We mark the indicator function of an event E as 1(E), which is 1 if E holds and 0 otherwise. We use sgn(-) to denote element- wise sign function that outputs 1 when input is nonnegative and -1 otherwise. Boldface lowercase letters are used to denote vectors (e.g., x), and the i-th element is denoted as [x];- Matrices are presented as boldface uppercase letters, say W. Given a matrix W E RRxdd we write its i-th row, j-th column and (i,j) element as W.,,, W.g. and Wa respectively. Moreover, we write its transpose matrix as (W)². The matrix (p, 4) norm is defined as || p.q == [II...llp ||W.2llp ..... || for any p.q 2 1. For convenience, p.
we have || > = ||],, and write the spectral norm and Frobenius norm as ||Wll, and [| respectively. We mark one matrix norm commonly used in this paper - the matrix (1, o0) norm.
With matrix W, we express its matrix (1,x) norm as [ 1, which is defined as [[
maxy W.,ll, and ||
1,50 = max; || [Wa: We use B (c) to express an element- foo
norm ball of matrix W within radius c, i... IB (c) = (W] W - Wayl I c, Vi E [k].] E [d]).
Preliminaries In order to formally explain our theoretical results, we introduce the considered learning problem, neural network model, and complexity definition. Let X' and y be the feature space and label space, respectively. We place the assumption that all data are drawn from unknown distribution D over X' x y and each data point is generated under i.idd condition. In this paper, we specifically consider the feature space X'
subset of d-dimensional Euclidean space, i.c., X C Rdd We denote the symbol F C F Fx-y to be the hypothesis class which we use to make predictions. Furthermore, the hypothesis class to make a function family written as E == (x, y) - {(f(x), y)1 f E F). The optimal solution of this learning problem is function f E F such that it minimizes the population risk R(f)= = (((.,))((((( [x),,,,, y)]- However, since the underlying data distribution is generally unknown, one typically aims at reducing the empirical risk evaluated by a set of training data {(x., })) which can be expressed as R,, = + > {(f(xx;, v:). The generalization error the gap between population and empirical risk, which could serve an indicator of model's performance under unseen data from identical distribution D.
consider a loss function é: X x y
[0, 1] and compose it with
To study the generalization error, one would explore the learning capacity of certain hypothe- sis class. In this paper, we adopt the notion of Rademacher complexity as a measure of learn- ing capacity, which is widely used in statistical machine learning literature [13]. The empiri- cal Rademacher complexity of a function class F given a set of samples S = (x is Rs({=) = E, [sup]er 1. E.= (xx()((x),, 2:)] where {ur is a set of i.i.d Rademacher random variables with P(y = P = +1} +1]-- =
3.2 The empirical Rademacher complexity measures on average how well a function class F correlates with random noises on dataset S. Thus, richer or more complex family could better correlate with random noise on average. With Rademacher complexity as a toolkit, one can develop the following relationship between generalization error and complexity measure. Specifically. it is shown in [1] that given a set of training samples S and assume that the range of loss function E(f(x) y) is [0, 1). Then for any E (0, 1), with at least
1 Note that when the probability 1 1- d we have Vf F. R(f) < R, (f) + 2Rs(EF) 3v

---

Rademacher complexity is small, it is then viable to learn the hypothesis class F by minimizing the empirical risk and thus effectively reducing the generalization gap.
Finally, we define the structure of neural networks and introduce a few related quantities. The problem studied in this paper is multi-class classification task with the number of classes being K. Consider an input vector E X' C Rd L-layer neural network is defined fw(x) W (...p(W''x
EX-R with W being the set containing all weight matrices, i.e., W == ( vk [Z.],, and the notation p(.) is used to express any non-negative monotone activation function and we further assume that p(-) is I-Lipschitz, which includes popular activation functionn such as ReLU applied element- wise on a vector. Moreover, the i-th component of neural networks' output is written as fw (x)
[Sw (x)), and a pairwise margin between i-th and j-th class, denoted as S (x) (x) - said to be the difference between two classes in output of the neural network. Lastly, we use the notion of z and gf to represent the output vector of the k-th layer (A: E [I. 1)) under natural and weight perturbed settings respectively, which are zz = p p(W* (...p((''x)... and zt = p(W (.-p(WW x)...)) where W B (e)) denotes the perturbed weight matrix bounded by its element- wise for norm with radius for some E [k.. Throughout this paper, we study the weight perturbation setting that each layer has an independent layer-wise perturbation budget, denoted by Ek. Ak E [L]. In the case of single-layer perturbation, we may omit the layer index and simply use
Sw (x), is
3.2 Building Block: Single-Layer Weight Perturbation
We study the sensitivity of neural network weight perturbations through the pairwise margin bound fu (x). Specifically, when i and corresponds to the top-1 1 and the second-top class prediction of respectively, the margin can be used as an indicator of robust prediction under weight perturbation to have provided simple example of three-layer neural network and explain the bound through the error propagation incurred by weight perturbation in Appendix A.I Here we directly introduce and analyze our theorem below.
W. For ease of understanding.
Theorem (N-th layer weight perturbation (N * L)) Let fw(x)= = W (...p(''xx..... denote
an L-layer neural network and let I (x) = W". (..W.... ...P(( W x)...). with W E BB (EN), N & L. denote the corresponding network subject to N-th layer perturbation. For any set of perturbed and umperturbed pairwise margin f2 (x) and fW (x), we have
12 w (x) S (x) eN | W 11
11 114-N-1
(W kTT
1,00
= (x) + A(EN:z"~1:f))
where zt = (...p x)x))..))
Proof:
See Appendix A.2
Since the final layer does not have any activation function, the margin bound on the margin difference when only perturbing the final layer can be simply derived, which is given in the following lemma.
Lemma (Final-layer weight perturbation) Consider the case N = L in Theorem 7 we have
f (x) S f + 2EL 1
-11
where 2--1 is the output of the (L 1)-th layer.
Proof: See Appendix A.2
3.3 General Setting: Multi-Layer Weight Perturbation
With the developed single-layer analysis in Section 3.2
building block, we now extend our analysis to the general setting of multi-layer weight perturbation, which further divided into two cases: (i) the case of perturbing all L layers; and (ii) the case of perturbing I out of L layers.
5

---

3.3.1 Perturbing All Layers
Once equipped with the concept of error propagation over subsequent layers, we consider the scenario where every layer in a neural network is subject to weight perturbation.
For simplicity and understanding, we will introduce an notation previously used but not fully explained in Theorem and Lemma [ Specifically, the notion A(-) can be described as the error caused by certain layer. Moreover, as we will see later, the function requires three inputs, the perturbation of that certain layer, the previous input and finally the neural network. Explicitly, for an L-layer neural network f()). perturbation radius of the k-th layer eg, and a prior input z, the delta function can be described as the following,
A (e z; ek W 11
[ 1,,
We denote the model under this case as the all-perturbed setting, The following theorem states an upper bound on the pairwise margin between the natural (unperturbed) and all-perturbed settings.
Theorem (all-layer perturbation) Let fw(x) W. (...p(('xx)..) denote an L-layer (natural) neural network and let f (x) = -w" (..W". (.p(W x)...) with W E BB (cx), Vk [L], denote its perturbed version. For any set of pairwise margin 12 (x) and fWd (xx), we have
S (x) (x)+
A(e1;x;
A(en;z==1';,
2
Input Layer Error
Final Layer Error
Intermediate Layer Ervor
where zx p(W* ...p( P(W x) with W defined as
W = W + ee, Vi,j and Vk E [b] \ {I} W15 Who + sgn((x))) e1, Vi,j
Proof:
See Appendix A.3.2
Here, provide some intuition deriving the upper bound of the margin in the all-perturbed setting. The scheme behind this all-perturbed scenario can be viewed as an inductive layer-wise error propagation. Specifically, we can choose any perturbed layer the commencement point of propagation, then fix any other weight matrices' values and further calculate the propagation of error from that layer using the concept Section 3.2, In such manner, after iterating through all these weight matrices subject to weight perturbation, one could obtain the final change in output value and therefore establish the pairwise margin bound. close inspection of the bound shows that the propagation of eгror causes the first term since the input layer and the rest of the terms are errors propagating since the k-th layer in the neural network, where k [L].
3.3.2 Perturbing Multiple Layers
The all- aertuueddd setting is a special case perturbing layers from an index set I when = [L]. We extend our analysis to the general multi-layer weight perturbation setting with [M]], which includes the single-layer (T (N}) and all-perturbed (I = [L]) settings as special cases.
(.. (...(. denote L-layer neural network. Given an index set 1 C [L]. we define the perturbed neural network as
Theorem (multiple-layer perturbation) Let fw
W = w* Vk I
(w(x)) = W (... p. x)..) with
W = wh, w e B (Ex)
Then, for any pairwise margin between 2 (x) and (x),
2 (x) < (x
Σ A(Exiz=11" + 1(L E 1)2EL ---1"
KEIVLI
Final Layer Error
Perturbed Layer Error

---

JW (x) +
(x
Error Weight Perturbation
Why + Vi.j v4: E (T)
Wha Vi,j (TXUU
x p W ...p(W' x) with w
defined as
Why + (gg((=))) Vi.j iflET
W
W Vi,j otherwise
and zd
=
Proof: See Appendix A.3.3
3.4
Surrogate Loss and Generalization Bound
3.4.1 Construction of Robust Surrogate Loss on Pairwise Margin
We aim to construct a surrogate loss function based on a standard loss function and study its behavior against weight perturbations. Here we study the single-layer perturbation case and elucidate the multi-layer perturbation case in Appendix B Specifically, given a perturbation budget for the k-th layer and the original loss function (x),)x))), y), robust training aims to minimize the following objective function: ((Jw(xx2)
maxw eIB@ (e) {( v)) which we call it as the robustness (worst-case) loss. Even for single data point (x, y). it is hard to assess the exact robustness loss since it requires the maximization of non-concave function of robust training against weight perturbations more computationally tractable, we aim to design a surrogate loss an upper bound on the worst-case loss.
norm ball. To make the problem
We focus on constructing a surrogate loss by means of pairwise margin bounds Section 3.3 We first define two popular loss functions in the classification problem, ramp loss and cross entropy, and derive their surrogate versions. Define the margin function M (fw (x).)) as
M(Jw (x), = min [Sw (x)), [fw (x)ly/ [/w (x)), yy y'øy
max
[/w (x)ly* 8'
The ramp loss for a given data point (x, y) and neural network fw (-) is written as l Camp (fw (x), σ, (fw (x), v)). where the function o., R ++ [0, 1] is defined as (t) = 1 1f S 0, oy if 2 7, and @, (4)
& if E [0, 7) Since the ramp loss is piece-wise linear function, its surrogate loss can be directly obtained with the pairwise margin bound Section 3.3 The cross entropy is written as C'(fw((x) y) = - [I[[ (x)ly), where fw (x) represents a neural network with its output passing through a softmax layer. That is, [ff Exe(K] exp /waal For ease of demonstration, we will be using ramp loss and its pairwise margin under single-layer perturbation in the following lemma. The surrogate loss analysis for cross entropy and robust surrogate loss for multiple-layer perturbation is given in Appendix
expl/w (x),
Lemma (robust surrogate ramp loss) Let E [T.] denote the perturbed layer index and let
V v(w(xx) = 2 max eN
||W2,lll
k.:l
N-1
1,,00 - [[(WL-k)2]] 11,00 ||x||
kE(K]
be the worst case error and
((fw(x).v) o.f ((f((x).)) (x)(
marginn
worst-case ervor
Then we have upper and lower bounds of l in terms 0-1 losses expressed as
max W ems x (c) By o arg max ['E[]]
Ww (x)),/ < ((S (x), < 1(M - S {.).
Proof: Please see Appendix B.I

---

One could observe in the formula that the margin function M(fw (x). y) serves as an accuracy objective similar the standard training process while the latter term could be conceived as the worst-case error caused by weight perturbation that should be suppressed. Therefore, by training under such an objective, we can simulate the scenario of robust training. Another intuition on the surrogate loss function is that the surrogate loss also implies the difficulty of training robust and generalizable models against large weight perturbations. Since error caused by perturbations would be surging rapidly through layers, only small perturbations can be applied in training and practice, permitting the worst-case error term to be smaller than the margin term. However, one follow-up question that naturally arises is whether not the generalization gap will be widened when training with the robust surrogate loss. The following section investigates the generalization property while conducting robust training and provides some theoretical insights training toward generalizable and robust model under weight perturbation.
3.4.2 Generalization Gap for Robust Surrogate Loss
We consider the robust surrogate loss established in Lemma 2 and study its generalization bound via Rademacher complexity in Theorem 4 where S {x, V/ lim! denotes the set of i..d training samples, X := X2, "2........ Xn E Rd denotes the matrix composed of training data samples, and dmax max { (d, ..,..., d denotes the maximum dimension among all weight matrices. Firstly, we define the adversarial population risk and empirical risk as
R(f) ((..))/D (3W E BBy (c) s.t. o arg max [fw(x)), (x)),, E|K]
Rv ()) (fw(x)))os -+ max [fw (x:)), + (fw
y'ou
Secondly, denote the following empirical Rademacher complexity of both margin function and worst-case error as
( ES ( 2 3/2
Rs(MF
GD log(n) log (dmmall
||]| (1
3/2
2eN sup II I 1l, 1 ||w -apl--ky 11. X 1
Rs(VF))
Theorem 4 (generalization gap for robust surrogate loss) With Lemmal 2 consider the neural net-
( = (W2 W ..... Wh), W E IBWx (eN) w S work hypothesis class f = (x)
Sh. (( 12,1 bn. h [L]]. For any 7 0, with probability at least 5, have for
all fw(())
log 2
R() & Ro()+
(s(
Rs Rs(VF)) VF)
+ +31
goneralization gop
complexiiy torm of robust training
Proof: Please see Appendix
As highlighted in the bracket term of Theorem 4 if the product of multiple weight norm bounds in
the additional complexity term R.s ( (v)) caused by robust training is not well confined, the model can suffer from a notable generalization gap. Consequently, our analysis suggests solution to reduce the generalization gap by imposing norm penalty functions on all weight matrices for training generalizable neural networks subject to weight perturbations.
3.5 -hroey-driven Loss toward Robustness and Generalization
With our theoretical insights,
propose robust and generalizable loss function. Standard neural network classifier training uses classification loss Eat, (x), that aims to widen the pairwise margin so as to raise accuracy, but won't necessarily be able to curb the error in output once weight perturbation is imposed. To address this issue, we propose to train under a mixed and regularized objective given data sample (x, y). which would, in turn, balance the tradeoff between

---

Cemanal werreitzatio Gat (N3)
Freet 0.001, Eran 0.01
pnù, aac-1847%
Freat 0.001, Cpan
0.0 0.002,
Accuracy (96)
1-0.0. 2-0 aMco25
A=0.0115, pm0.045, auc
Seatt 0. 0022
Eran 0.05
A=0.0135, po0.04% sur-33
A.0.0125, poc.ort suciat 38%,
Aw0.034 pw0.045, auC-29.455
0.00
0.002
0.004
0CG 0.005
10 E.000.30m 3000 wo to a.2za zoan Does 0.22 pommonoooooooooee EiEz
Regulatication le)
Weight perturbattin level (Peest
(a) Empirical generalization gap
(b) Robust accuracy under adversarial perturbation
Figure 1: (a) Empirical generalization gaps when varying the matrix norm regularization coefficient in (2) under 200-step weight PGD attack with perturbation level Gest- Consistent with the theoretical results, the gap reduces μ increases for every érain value used for training. (b) Comparison of test accuracy of neural networks trained with different X and under weight PGD attack (200 steps) with perturbation level Erest- AUC refers to the under curve score. Joint training with the two theory-driven terms in indeed yields more generalizable and robust neural networks against weight
perturbations.
standard accuracy, robustness, and generalization. The designed loss takes the form:
E" (x).)) standard
(as (x)
A
max(n
+
([(W")) +||W"|| 11.00)
m=
robustness from
geseralization pap regularization Thm.
The first term in (2) originates from standard classification problem for task-specific accuracy, while the second term results from the maximum еггоr on pairwise margin (the term T/ (x: defined in Theorem 3 induced by weight perturbation. Finally, we contribute the last term to the theoretical findings in Theorem 4 where imposing norm constraints on weight matrices could benefit generalization and prevent the generalization gap from widening. Based on our analysis, we also note that the standard technique of L1 penalty function on weight applies as well.
Numerical Validation
We validate our theoretical results and the designed loss function in 2 through two sets of experi- ments: empirical generalization gap with matrix norm regularization and robust accuracy against adversarial weight perturbations.
Experiment setup We used the MNIST dataset comprised of gray-scale images of
digits with ten categories. We trained neural network models as in (Section 3.1) with four dense layers (number of neurons are 6---33--10) and the ReLU activation function without the bias term. We used the loss function e in (2) with all-layer perturbation bound (i.e., I = [Z]),, identical weight perturbation radius (or étainn), cross entropy as the standard classification loss fats, and a batch size of 32 with 20 epochs. Stochastic gradient descent with momentum is used for training, with the learning rate set to 0.01. For the generalization experiment, we follow same setting as [29], which uses 1000 data samples train the neural network. In all settings assign identical weight perturbation budget e for every layer. The comparison of weight distribution each layer using standard and our proposed robust training is given in Appendix [F] All experiments were conducted using an Intel Xcon 25-2620v4 CPU, 125 GB RAM, and an NVIDIA TITAN Xp GPU with 12 GB RAM. For reproducibility, our codes given in the supplementary material.
Weight PGD attack To evaluate the robustness against weight perturbations, we modified the projected gradient descent (PGD) attack originally designed for input perturbation [12] which call as weight PGD attack. Starting from a trained neural network weight W, the perturbed weight W is crafted by iterative gradient ascent using the signed gradient the standard loss denoted as egn(Vwe lats ww (x), v)), followed by element-wise (or éwst) clipping centered W. The attack iteration with the step size a is expressed as
( W, (+++1) Clipw. (1() a Sgn(Vwlede (w (x).
(3)

---

I (a) shows the empirical generalization gap (training accuracy test accuracy) with respect to the matrix norm regularization coefficient μ under 200-step weight PGD attack with the perturbation level fist defined in 2 and we supplement test accuracies in Appendix D.2 As indicated in Theorem 4 increasing μ effectively suppresses the Rademacher complexity and thus reduces the generalization gaps, which are consistently observed on neural networks trained with different weight perturbation level Eqainn We also conduct different settings of generalization gap analysis with éxst = 0 and different émain values in Appendix D.1 To demonstrate the necessity of our regularization loss function, in Appendix D.3 we run the same experiments of Figure I (a) but replace regularization loss with L1 and L2 weight decay, respectively. It shows that only our loss function can aid reducing generalization gap and improving robustness.
Empirical generalization gap Figure
Robust accuracy against adversarial weight perturbation We trained neural networks with differ- ent combinations of the coefficients À and μ in ( using frriin 0.01. Figure (b) shows the test accuracy under different weight perturbation level Gest (i.e., the robust accuracy) with 200 attack steps. The standard model (X = f 0) is fragile to weight PGD attack. On the other hand, neural networks trained only with the robustness loss (A > 0 and μ 0) or the generalization gap regularization (A (A=0 0 and μ > 0) can improve the robust accuracy due to improved generalization and classification margin. Moreover, joint training using the proposed loss with proper coefficients can further boost model performance (e.g., (A, μ4) = (0.0125, 0.045)), as seen by the significantly improved area under curve (AUC) score of robust accuracy over all tested Eest values. The AUC of the best model is about 2x larger than that of the standard model. Similar results can be concluded for the attack with 100 steps (see Appendix D.5). Appendix D.4 shows additional experiments on the extension to convolutional neural networks trained on CIFAR-TO. Similar conclusion holds. In Appendix [D.6] we conduct additional experiments the coefficients À and μ and discuss their tradeoffs. In summary, the results suggest the effectiveness of our theory-driven loss function for improving generalization against weight perturbation.
Run-time analysis and weight quantization The run-time analysis in Appendix D.8| shows the training costs for robust and standard models are comparable. In Appendix D.9. we also show that neural networks trained with our proposed loss are more robust to weight quantization. Under bit quantization, the decrease in accuracy of the robust model is 3x less than the standard model.
Non-vacuous generalization bound under weight perturbation In the standard generalization setting, [4] showed that when increasing the size of the training data, the error bound in 4 grows rapidly, loosing the ability to describe generalization gap and thus becomes vacuous. In Appendix 3.5 we show that the error bounds of the models trained using our proposed loss function (2) in Section 3.5 would not grow a polynomial rate and instead shows a decreasing trend, concluding the non-vacuity of the associated generalization bounds in our robust generalization setting.
Comparison to naive adversarial weight training (AWT) We also present comparisons of test accuracy between models trained with AWT, our loss, and the standard model under weight PGD attack with different perturbation level ees in Appendix G shows that directly applying weight PGD attack into adversarial training is not effective in training robust models.
5 Conclusion
In this paper, we developed a formal analysis of the robustness associated with the pairwise class margin for neural networks against weight perturbations. We also characterized its generalization gap through Rademacher complexity. A theory-driven loss function for robust generalization was proposed, and the empirical results showed significantly improved performance in generalization and robustness. Our analysis offers theoretical insights and informs the principles training loss design toward generalizable and robust neural networks subject to weight perturbations.
Acknowledgments
Part of this work was done during Chia-Y Yi Hsu's visit at IBM Research. Yu-Lin Tsai, Chia-Yi Hsu and Chia-Mu Yu were supported by MOST 1 110-2636-F-009-01 8 and we also thank to National Center for High-performance Computing (NCHC) of National Applied Research Laboratories (NARLabs) in Taiwan for providing computational and storage resources.
10

---

References
[1] Guozhong An. The effects of adding noise during backpropagation training on a generalization performance. Neural computation, 8(3):643 674, 1996.
[2] Alessandro Barenghi, Luca Breveglieri, Israel Koren, and David Naccache. Fault injection attacks on cryptographic devices: Theory, practice, and countermeasures. Proceedings of the IEEE, 100(11):3056-3076 2012.
[3] Andrew R Barron and Jason M Klusowski. Approximation and estimation for nigh-dimensional deep learning networks. arXiv preprint arXiv: 180,:0099,,00 2018.
[4] Peter I.. Bartlett, Dylan J. Foster, and Matus Telgarsky. mnorrrally-oormalized margin bounds for neural networks. In Advances in Neural Information Processing Systems, pages 6240-6249, 2017.
[5] Gintare Karolina Dziugaite and Daniel M. Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Proceedings of the Thirty-Thire Conference Uncertainty in Artificial Intelligence, UAI 2017, Sydney, Australia, August 11-15, 2017, 2017.
[6] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware min- imization for efficiently improving generalization. In International Conference on Learning Representations, 2021.
[7] Noah Golowich, Alexander Rakhlin, and Ohad Shamir. Size-independent sample complexity of neural networks, 2019.
[8] Sven Gowal, Krishnamurthy Dj Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. Scalable verified training for provably robust image classification. In Proceedings of the IEEE International Conference on Computer Vision, pages 4842-48511 2019.
[9] Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran EI- Yaniv, and Yoshua Bengio. Quan- tized neural networks: Training neural networks with low precision weights and activations. The Journal of Machine Learning Research, 18(1):6869-6888, 2017.
[10] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy. and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. International Conference on Learning Representations, 2017.
[11] Yannan Liu, Lingxiao Wei, Bo Luo, and Qiang Xu. Fault injection attack on deep neural network. In IEEE/ACM International Conference Computer-Aided Design (ICCAD), pages 131-138, 2017.
[12] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep lcarning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
[13] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018
[14] Vaishnavh Nagarajan and Zico Kolter. Uniform convergence may be unable explain generalization in deep learning. In Advances in Neural Information Processing Systems, pages 11615-11626, 2019.
[15] Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning. In Advances in neural information processing systems, pages 5947-5956, 2017.
[16] Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. A PAC-bayesian approach to spectrally-normalizedd margin bounds for neural networks. In International Conference on Learning Representations, 2018

---

[17] Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. Norm-based capacity control in neural networks. Conference on Learning Theory. pages 1376-1401, 2015.
[18] Matthew D Norton and Johannes O Royset. Diametrical risk minimization: theory and compu- tations. Machine Learning, pages 1-19, 2021.
[19] Henning Petzka, Linara Adilova, Michael Kamp, and Cristian Sminchisescu. Feature-robustness. flatness and generalization error for deep neural networks. arXiv preprint vrXiv:2001.00939 2020.
[20] David Stutz, Nandhini Chandramoorthy, Matthias Hein, and Bernt Schiele. Bit error robust- ness for energy-efficient dnn accelerators. In A. Smola, A. Dimakis, and I. Stoica, editors, Proceedings of Machine Learning and Systems, volume 3, pages 569-598, 2021.
[21] Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin- Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy? ? comprehensive study on the robustness of 18 deep image classification models. In Proceedings of the European Conference on Computer Vision (ECCV), pages 631-648, 2018.
[22] Ryan Theisen, Jason M Klusowski, Huan Wang. Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. Global capacity measures for deep relu networks via path sampling. arXiv preprint arXiv: 1e.11990..2240 2019.
[23] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference on Learning Representations, 2019.
[24] Victor Van Der Veen, Yanick Fratantonio, Martina Lindorfer, Daniel Gruss, Clémentine Maurice, Giovanni Vigna, Herbert Bos, Kaveh Razavi, and Cristiano Giuffrida. Drammer: Deterministic rowhammer attacks on mobile platforms. In ACM SIGSAC conference on computer and communications security, pages 1675-1689, 2016.
[25] Yisen Wang, Xingjun Ma. James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In International Conference on Machine Learning, volume 1, page 2, 2019.
[26] Tsui-Wei Weng, Pu Zhao, Sijia Liu, Pin-Yu Chen, Xue Lin, and Luca Daniel. Towards certifi- cated model robustness against weight perturbations. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6356-6363, 2020.
[27] Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 33, 2020.
[28) Huan Xu and Shie Mannor. Robustness and generalization. Machine learning, 86(3):391-423, 2012.
[29] Dong Yin, Ramchandran Kannan, and Peter Bartlett. Rademacher complexity for adversarially robust generalization. In International Conference on Machine Learning, pages 7085-7094, 2019.
[30] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference Muchine Learning, 2019.
[31] Pu Zhao, Pin- Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. Bridging mode connectivity in loss landscapes and adversarial robustness. In International Conference on Learning Representations, 2020.
[32] Pu Zhao, Siyue Wang, Cheng Gongye, Yanzhi Wang, Yunsi Fei, and Xue Lin. Fault sneaking attack: A stealthy framework for misleading deep neural networks. In ACM/IEEE Design Automation Conference (DAC), pages 1-6, 2019.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [No] In this paper, we design a theory-driven loss which theoretically can be used more applications without any limitations.
(c) Did you discuss any potential negative societal impacts of your work? [No] The major contribution our work to propose theory-driven loss for training generalizable and robust networks against weight perturbations to improve the model robustness and performances. Namely, it can reduce potential negative impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Sectior
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [No] The code will
be shared upon request.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 4
(c) Did you report error bars (c.g., with respect to the random seed after running exper- iments multiple times)? [No] In our experiments, it does not include the effect random seeds.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, cloud provider)? [Yes] See Section 4
4. If you using existing assets (e.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're re using/curating? [Yes] We used publicly available datasets which can be used for educational research purposes such as this work. We have suitably cited all datasets.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [No] The datasets we used do not include personally identifiable information or offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "uF6SB3RNF_W", "reviewer": "Reviewer_Ljmn", "summary": "In this paper, the authors developed a formal analysis of the robustness associated with the pairwise class margin for neural networks against weight perturbations. The authors also characterized its generalization gap through Rademacher complexity and proposed a theory-driven loss function for robust generalization. The empirical results showed significantly improved performance in generalization and robustness.  ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. The authors aim to study a new problem in neural networks’ robustness to weight perturbations. While the concept is kind of similar to adversarial examples, I would love to see some explanation on why the weight perturbation adopts the Linf norm constraint by default.   \n2. One of the major issues in the paper is that it lacks necessary comments and intuitions for each theoretical result. For example, what does theorem 1 implies? Or how does this result help us? It is not clearly written. Readers need to go over the entire paper to understand the author's purpose to derive such an upper bound.\n3. Theorem 3, what is the Error of Weight Perturbation term defined? Should make it clear in the main paper\n4. Until Section 3.5, I start to understand that the authors are trying to propose a new surrogate loss for weight robust training. However, the question is: is this surrogate loss necessary? In the experiments, the authors only compare their own method in different parameter settings. However, a simple baseline to consider here is simple weight adversarial training (just like PGD attack to weight PGD attack, it is easy to build adversarial training to weight adversarial training). In this way, it does not require any surrogate loss derived in this paper. In other words, it makes much more sense in the case that weight adversarial training is performing poorly and the authors turn to study other surrogate losses for this problem. Yet if weight adversarial training is already doing ok, I don’t see the reason for proposing such a surrogate loss.  \n\n\nMinor comments:\n\n1 .   Line 308, typo (??)\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "see above", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "nMfRkS6buLL", "reviewer": "Reviewer_LdZY", "summary": "In this work the authors study the propagating effect of perturbing the weights of a neural network on the models final output. They use the resulting theorems to construct a bound on the Rademacher complexity of the model class under perturbed weights and provide regularizations terms to improve both the robustness against these weight perturbations and to reduce the generalization gap. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Dear authors and AC, \n\nBelow you will find my review for the paper in question. \n\nAs very clearly laid out in the introduction, the study of a neural networks sensitivity to perturbations of its weights is an research area of practical importance. The first theoretical characterization provided in this present work is therefore of considerable importance and might have significant impact on future work. I must however admit that I am not entirely up to date with research regarding weight quantization and compression. For this reason I would like to leave judging the works novelty to the other reviewers. \n\nMy main concerns however lie with the numerical validation performed and the unanswered questions regarding the practical usefulness of the bounds and regularization terms that arise from this section. \n- First and foremost, the experimental setup suggest that the additional of the surrogate loss might bring some serious computational restrictions. As far as I understand the experiments have been performed with only 1000 MNIST data samples for training. In the manuscript there is no explanation offered for this decision besides simply following related work. I would appreciate it if the authors could comment on this and/or provide some additional data showcasing the computational cost of training the model using the surrogate loss. The discussion offered in appendix section D.7 is in my opinion not clear enough. (eg. equation 2 referred to in this section seems to only take first layer perturbations into account)\n- Second, in the experimental setup the authors mention that no bias terms were used. It would be good to clarify why these were excluded. And, if this is because of them not fitting in the framework, this limitation should be more clearly communicated.\n- Third, I have my reservations regarding the effectiveness of the evaluation metric used. Specifically, I do not understand the use of the AUC metric. Even the best performing model in figure 1b quickly degrades to roughly 40% accuracy. At this level of accuracy the usability of the models is mostly completely gone. The further slow degradation after is therefore not really informative any more. In similar fashion, the supplementary experiment on weight quantization shows that the decrease in accuracy of a robust model is 3x less then a standard model for 2 bit quantization. However, this still degrades the model accuracy far enough to question its use. For 16-4 bit quantization the standard and robust model perform roughly the same. \n- Lastly, I believe that a comparison against standard L1 and L2 weight regularization is somewhat necessary. I suspect that keeping the norm of the weights small should suffice to see a lot of the same benefits as the proposed regularization terms. \n\nDespite the shortcomings of the numerical evaluation section mentioned above, I still believe that the theoretical study presented in the paper can be a valuable starting point for further studies. For this reason I am still inclined to vote for acceptance. If the concerns with the numerical evaluation above could be expanded upon by the authors I would be willing to improve my score. \n\nSome other minor remarks:\n- I appreciate the use of colour to clarify the mathematical notation, but I don't think it is used very effectively in some cases. For example, I do not see the use of colouring the equation in sec. 3.5.\n- There are numerous references to equation 2. These are broken and very  confusing. \n- Line 308 has a dead reference.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As I stated in my main review, the numerical evaluation (and more specifically the experimental setup) leave me to question a few practical implications of the theory presented. The authors should clarify these in their reply.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "NulDVrm5T20", "reviewer": "Reviewer_sPbL", "summary": "The paper studies the worst-case perturbations in the weight-space for multi-layer networks. The authors first derive a robust surrogate loss which is an upper bound on the loss under norm-bounded weight perturbations based on a product of the 1-inf matrix norm of layer weight matrices. Then the authors derive a generalization bound based on the Rademacher complexity which motivates their new loss function which is evaluated empirically on MNIST.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Overall assessment**:\n- Originality: the derived bounds (both for the worst-case loss and for generalization) are to the best of my knowledge novel in the context of weight perturbations, although they seem to be directly inspired by similar bounds in the context of input perturbations. \n- Quality: the submission seems to be technically sound but the empirical evaluation can be significantly improved (see **Cons**).\n- Clarity: the paper is clearly written and well-organized.\n- Significance: on the one hand, theoretically characterizing generalization is an important contribution but on the other hand I have concerns about the tightness of the proposed bound and usefulness of the proposed loss function (see **Cons**).\n\n**Pros**:\n- Formal analysis of generalization and the behaviour of the worst-case loss with respect to weight-space perturbations is a valuable contribution.\n- Novel theory-driven loss function that aims at improving generalization and robustness against weight perturbations, although I have some concerns about its effectiveness. \n\n\n\n**Cons**:\n- My main concern is the tightness of the derived bounds on the worst-case loss. Analogous layerwise bounds are simple and known for a long time in the context of **input-space** perturbations (as opposed to the **weight-space** perturbations studied in this paper), e.g. see\n[Certified Defenses against Adversarial Examples](https://arxiv.org/abs/1801.09344), Section 5 which can be trivially extended to multiple layers and to norms on weight matrices W^i other than the spectral one (e.g., including the 1-inf norm that this paper uses a lot). Although the loss function from Sec. 3.5 is not exactly the same as the objective that [Certified Defenses against Adversarial Examples](https://arxiv.org/abs/1801.09344) use as a baseline, they seem to be very connected since the proposed loss relies mainly on the 1-inf norm of the weights. Thus, it would be very useful if the authors could (1) discuss this connection and (2) empirically/theoretically justify the tightness of the proposed bound. The concern (2) comes from the fact that this layerwise bound is usually loose for the input-space perturbations, and thus has been used in the literature just as a baseline (and more involved approaches have been developed, e.g. the SDP relaxation of [Certified Defenses against Adversarial Examples](https://arxiv.org/abs/1801.09344)). There is an analysis of the tightness of the generalization bound in Appendix E but the tightness of the bounds on the worst-case loss doesn’t seem to be sufficiently discussed.\n- The empirical evaluation part requires some natural baselines such as, e.g., standard L1/L2 weight regularization. It seems that both additional terms from the theory-driven loss (from Sec. 3.5) are based primarily on the 1-inf norm of the weights, so a natural question is how different it is from the standard weight regularization methods. I don’t think that the current experimental part really justifies the proposed method. Related to this, I don’t think that Fig. 3 in the appendix is informative enough since it shows just the generalization gap but what is more interesting is the absolute value of test accuracy.\n- Another aspect of the empirical evaluation which is not very strong is that the evaluation is done only on one small-scale dataset (MNIST). As the authors note in the “Run-time analysis” paragraph, the method should be quite scalable, so it’s not clear to me why its applicability hasn’t been shown on more challenging datasets and models. I think this is an important point since the weight perturbations approaches from [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://arxiv.org/abs/2010.01412) and [Adversarial weight perturbation helps robust generalization](https://arxiv.org/pdf/2004.05884.pdf) are very scalable (lead to only 2x overhead during training) and easily applicable to arbitrary deep networks.\n- A discussion on some of the relevant prior works is missing, namely to [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://arxiv.org/abs/2010.01412) (published at ICLR 2021) which is not cited and to [Adversarial weight perturbation helps robust generalization](https://arxiv.org/pdf/2004.05884.pdf) (published at NeurIPS 2020) which is cited but not discussed in detail. It is important to discuss them as the authors claim in the abstract: *“the first formal analysis”* and later in the text that: *“We note that previous methods use random weight perturbations to evaluate the generalization ability (e.g. sharpness), which we call a standard generalization setting. In contrast, we study a different problem setup: generalization of models trained under worst-case (adversarial) weight perturbation”*\nThus, I’d recommend the authors to discuss these references in detail and contrast their generalization bounds to the ones derived in this paper, potentially also including a numerical comparison between both. \n\n\n**Other points**:\n- Line 20: *“weight quantification”* -> *“weight quantization”*\n- Line 22-23: *“weight perturbation sensitivity is also used as a metric to **evaluate** the generalization gap”* -- I think it’s worth being more precise here. What is shown in those papers is merely a **correlation**, and its exact relation to the generalization gap is not completely clear. So sharpness can’t be used to **evaluate** generalization gap.\n- Line 285-287: “**will** suffer from a notable generalization gap” -> “**can** suffer from a notable generalization gap” -- as the derived upper bound is only an upper bound and on the exact quantity of interest.\n- Line 308: citation is missing.\n- Citation style: consider using \\citet instead of \\cite when citation is used as a part of a sentence (e.g., “[1] showed ...”)\n- I think Theorem 3 can be put to the appendix as it doesn’t seem to give any new insights compared to Theorem 2.\n- It’s worth numbering equations.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think the authors could have discussed better the tightness of their bound on the worst-case loss. The authors don’t discuss any negative societal impact of their work but I think it’s not necessary for this paper. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "FNoxbxzzzzD", "reviewer": "Reviewer_Yipw", "summary": "The paper studies the effect of weight perturbations in a neural network on its performance in terms of generalization and adversarial robustness. Robustness to weight perturbations is important for improving a model’s fault tolerance and reducing its storage requirements without significantly compromising its performance. It uses Rademacher complexity to quantify the impact on generalization and pairwise class margins for adversarial robustness. It considers networks with non-negative monotone activation functions that are 1-Lipschitz. Using the theory developed in the paper, the authors propose a new loss function for training more generalizable and robust models.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: To the best of my knowledge, this work is the first study in understanding the effects of weight perturbations on a neural network’s ability to generalize on unseen data and its adversarial robustness. It combines well-known mathematical techniques to quantify and characterize a model’s robustness and generalization capability.\n\nQuality: The theoretical results appear to be technically sound though not particularly surprising. The experiments study how the generalization gap and robust accuracy vary with the parameters of the proposed loss terms. However, it is not clear how they affect the test accuracy. Fig 1(a) shows that the gap between train and test accuracies decreases as mu increases for different values of epsilon used during training. Does this drop in the gap result in any gain in test accuracy? Is there a setting of the parameters that helps achieve a better test accuracy than when the proposed loss function is not used? I believe showing an improvement in the test accuracy will better demonstrate the gain in generalization from the proposed loss function.\n\nClarity: The paper is well written.\n\nSignificance: The authors demonstrate through experiments that the proposed loss function helps improve robustness and generalization which are important qualities for any good machine learning model. However, it is difficult to determine the significance of this work without knowing the effect of the loss function parameters on test accuracy. One can always construct models that have high robustness and low generalization gap but have poor accuracy. I believe a study of how these parameters affect the accuracy of the model would help increase the impact of this paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This paper seeks to improve the robustness and generalizability of neural network models. The only limitation that comes to my mind of this type of work is its negative impact on the accuracy of the model. The authors have studied the trade-off between robustness and accuracy. However, as I mentioned in the main review a study of the test accuracy of the trained model with respect to the loss parameters is important to properly understand the gain in generalization capabilities.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Even though the paper received mixed scores, all the reviewers agreed on the originality and the significance of the contributions of the paper. The authors’ extensive rebuttal letters later on clarified several concerns/confusions, especially in terms of lack of empirical evaluation. While the paper is borderline, I believe that most raised issues can be properly addressed **provided the authors implement all the required changes, which they addressed in their rebuttal**. Hence I am recommending a weak accept.
Decision: Accept (Poster)