Efficient Training of Low-Curvature Neural Networks
Authors: Suraj Srinivas
Kyle Matoba
Himabindu Lakkaraju
François Fleuret
Venue: NeurIPS 2022 Accept (2022)
Keywords: Deep neural networks
curvature
spectral norm
Lipschitz constant
robustness

Abstract: Standard deep neural networks often have excess non-linearity, making them susceptible to issues 
such as low adversarial robustness and gradient instability. Common methods to address these 
downstream issues, such as adversarial training, are expensive and often sacrifice predictive accuracy. 

In this work, we address the core issue of excess non-linearity via curvature, and 
demonstrate low-curvature neural networks (LCNNs) that obtain drastically lower curvature 
than standard models while exhibiting similar predictive performance. This leads to improved 
robustness and stable gradients, at a fraction of the cost of standard adversarial training. 
To achieve this, we decompose overall model curvature in terms of curvatures and slopes of 
its constituent layers. To enable efficient curvature minimization of constituent layers, 
we introduce two novel architectural components: first, a non-linearity called centered-softplus 
that is a stable variant of the softplus non-linearity, and second, a Lipschitz-constrained 
batch normalization layer.

Our experiments show that LCNNs have lower curvature, more stable gradients and increased 
off-the-shelf adversarial robustness when compared to standard neural networks, all without 
affecting predictive performance. Our approach is easy to use and can be readily incorporated 
into existing neural network architectures.

Efficient Training of Low-Curvature Neural Networks
Suraj Srinivas"! Harvard University
Kyle Matoba" Idiap Research Institute & EPFL kyle www.mmaapooa...cmmmmmmm
serinivas@seas har.arrd
Himabindu Lakkaraju
François Fleuret
Harvard University
University of Geneva francois. fleuretuunige..h
hlakkaraju@hba edu
Abstract
Standard deep neural networks often have excess non-linearity, making them sus- ceptible to issues such as low adversarial robustness and gradient instability, Com- mon methods to address these downstream issues, such as adversarial training, are expensive and often sacrifice predictive accuracy.
In this work, we address the core issue of excess non-linearity via curvature, and demonstrate low-curvature neural networks (LCNNs) that obtain drastically lower curvature than standard models while exhibiting similar predictive performance. This leads to improved robustness and stable gradients, at fraction of the cost of standard adversarial training. To achieve this, we decompose overall model curva- ture in terms of curvatures and slopes of its constituent layers. To enable efficient curvature minimization of constituent layers, we introduce two novel architectural components: first, non non-lineatee lincarity called centered centered-softplus softplus that is stable variant the softplus non-linearity, and second, a Lipschitz Lipschitz-constrrimn constrained batch normal- ization layer.
Our experiments show that LCNNs have lower curvature, more stable gradients and increased off-the-she she-t adversarial robustness when compared to standard neu- ral networks, without affecting predictive performance. Our approach is easy to use and can be readily incorporated into existing neural network architectures.
Code implement method and replicate experiments is available
Introduction
The high degree of flexibility present in deep neural networks is critical achieving good perfor- mance in complex tasks such as image classification, language modelling and generative modelling of images 3 However, excessive flexibility undesirable as this can lead to model under- specification which results in unpredictable behaviour on out-of-domain inputs, such as vulner- ability to adversarial examples. Such under-specifiiattio can be avoided principle via Oceam's razor, which requires training models that are as simple possible for the task at hand, and not any more. Motivated by this principle, this work we focus on training neural network models without excess non-linearity in their input-output map (e.g.: see Fig lb), such that predictive performance remains unaffected.
Equual Contribution
Work done partially Idiap Research Institute
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

5
saftplus(B =100)
softplus(p =022
c-softplusij 1.0)
c-oftplustf 0.2)
2-
-1
(a) Decision boundary with standard NN (3 1000)
(b) Decision boundary with
-2
LCNN (8 =
(c) Softplus Centered Centered-Softplus Softplus
Decision boundaries of (a) standard NN and (b) LCNN trained on the two moons dataset. LCNN recovers highly regular decision boundaries in contrast to the standard NN. (c) Comparison of softplus and centered-softplus non-linearities (defined in $3.3). These behave similarly for large values, and converge to linear maps for low 3 values. However, softplus diverges while centered- softplus stays close to the origin.
Figure I:
Central to this work is precise notion of curvature, which is mathematical quantity that encodes the flexibility or the degree of non-linearity of a function at point. deep learning, the curvature of function at point often quantified the norm of the Hessian at that point 7 Hessian norms are zero everywhere if and only if the function linear, making them suitable to measure the degree of non-lineari.y. However, they suffer from a dependence makes them unsuitable to study its interplay with model robustness. In particular, robust models have small gradient norms [8], which naturally imply smaller Hessian norms. But they truly more linear as a result? To be able to study robustness independent of non-linearity, we propose normalized curvature, which normalizes the Hessian norm by its corresponding gradient norm, thus disentangling the two measures. Surprisingly, we find that normalized curvature is a stable measure across train and test samples (see Table B}. whereas the usual curvature not.
the scaling of model gradients, which
A conceptually straightforward approach to train models with low-curvature input- maps 6 involves directly penalizing curvature locally at training samples. However, these methods involve expensive Hessian computations, and only minimize local point-wise curvature and not curvature everywhere. A complementary approach is that of Dombrowski et al. [10]. who propose architec- tures that have small global curvature, but do not explicitly penalize this curvature during training. In contrast, we propose efficient mechanisms explicitly penalize the normalized curvature globally. In addition, while previous methods 7 penalize the Frobenius norm of the Hessian, we penalize its spectral norm, thus providing tighter and more interpretable robustness bounds
Our overall contributions in this paper are:
1. In $3.1 we propose to measure curvature of deep models via normalized curvature, which is invariant scaling of the model gradients.
2. In $3.2 we show that normalized curvature of neural networks can be upper bounded in terms of normalized curvatures and slopes of individual layers.
3. We introduce an architecture for training LCNNs combining a novel activation function called the centered-softplus ($3.3) and recent innovations to constrain the Lipschitz con- stant of convolutional ($3.4.1) and batch normalization (33.4.2) layers.
4. In $ we prove bounds the relative gradient robustness and adversarial robustness of models terms of normalized curvature, showing that controlling normalized curvature directly controls these properties.
5. In $ we show experiments demonstrating that our proposed innovations are successful in training low-curvature models without sacrificing training accuracy, and that such models have robust gradients, and are more robust adversarial examples out-of-the-box.
Note that the Frobenius norm of matrix strictly upper bounds its spectral norm

---

2 Related Work
Adversarial Robustness of Neural Networks: The well-known phenemenon of adversarial vul- nerabilities of neural networks (T 12 shows that adding small amounts of imperceptible noise can cause deep neural networks misclassify points with high confidence. The canonical method to de- fend against this vulnerability is adversarial training [T3] which trains models to accurately classify adversarial examples generated via an attack such as projected gradient descent (PGD). However, this approach is computationally expensive and provides no formal guarantees on robustness. Co- hen et al. [4] proposed randomized smoothing, which provides a formal guarantee on robustness by generating smooth classifer from any black-box classifier. Hein and Andriushchenko [8] identified the local Lipschitz constant as critical quantity to prove formal robustness guarantees. Moosavi Dez- fooli et al. [6] penalize the Frobenius norm of the Hessian, and show that they performs similarly to models trained via adversarial training. Qin et al. [9] introduce local linearity regularizer, which also implicitly penalizes the Hessian. Similar these works, we enforce low curvature induce robustness, but we focus on out of the box robustness of LCNNs.
Unreliable Gradient Interpretations in Neural Networks: Gradient explanations in neural net- works can be unreliable. Ghorbani al. [.S., Zhang et al. [T6] showed that for any input, it is possible to find adversarial inputs such that the gradient explanations for these points that are highly dissimilar to each other. Srinivas and Fleuret [17 showed that pre-softmax logit gradients however are independent of model behaviour, and as result we focus on post-softmax loss gradients in this work. Ros and Poshi-Veles [8] showed empirically that robustness can be improved by gradient regularization, however Dombrowski et al. [10] showed that gradient instability is primarily due to large Hessian norms. This suggests that the gradient penalization in Ros and Doshi- Velez [18] per- formed unintentional Hessian regularization, which is consistent with our experimental results. To alleviate this, Dombrowski et al. 7 proposed to train low curvature models via softplus activations and weight decay, broadly similar to our strategy. However, while Dombrowski et al. 7 focused on the Frobenius norm of the Hessian, we penalize the normalized curvature a scaled version of the Hessian spectral norm which is strictly smaller than the Frobenius norm, and this results in more sophisticated penalization strategy.
Lipschitz Layers in Neural Networks There has been extensive work on methods to bound the Lip- schitz constant of neural networks. Cisse et al. [19] introduced Parseval networks, which penalizes the deviation of linear layers from orthonormality since orthonormal linear operator evidently has a Lipschitz constant of one, this shrinks the Lipschitz constant of layer towards one. Trockman and Kolter (20) use reparameterization of the weight matrix, called the Cayley Transform, that is orthogonal by construction. Miyato et al. (20. Ryu et al. [22] proposed spectral normalization, where linear layers re- parametterizee by dividing by their spectral norm, ensuring that the overall spectral norm of the parameterized layer is one.
3 Training Low-Curvature Neural Networks
In this section, we introduce our approach for training low-curvature neural nets (LCNNss. Unless otherwise specified, we shall consider a neural network classifier that maps inputs x RR to logits which characterize the prediction, and can be further combined with the true label distribution and a loss function give scalar loss value f(x) E Rt.
3.1 Measuring Relative Model Non-Linearity via Normalized Curvature
We begin our analysis by discussing a desirable definition of curvature Cx()) E R+. While curva- ture well-studied topic in differential geometry 23 where Hessian normalization theme, our discussion will be largely independent of this literature, in a manner more suited to re- quirements in deep learning. Regardless of the definition, a typical property of curvature measure is that (x) = 0 V x Rdd + f is linear, and the higher the curvature, the further from linear the function is. Hence maxxeR" (x) can be thought of a measure of model's non-linearity. A common way to define curvature in machine learning (5 has been via Hessian norms. However, these measures are sensitive to gradient scaling, which is undesirable. After all, the degree of model lin-linyyrity must intuitively be independent of how large its gradients are.
common
3

---

For example, if two functions f.g are scaled (f = k x g), or rotated versions of each other (Vf = kx Vg). then we would like them to have similar curvatures in the interest of disentangling curvature [x) (i.e., degree of non-linearity) from scaling. It is easy to see that Hessian norms /x) do not have this property, as scaling the function also scales the corresponding Hessian. We would like to be able to recover low-curvature models with steep gradients (that are non-robust), precisely to be able to disentangle their properties from low-curvature models with small gradients (that are robust), which Hessian norms do not allow us to do.
To avoid this problem, we propose a definition of curvature that approximately normalized: Cx(x) = (x)12// f(x)] + E). Here [ and [ are the &g norm of the gradient and the spectral norm of the Hessian, respectively, where Vf(x) E R f²f(x) E Rdddd and 0 small constant ensure well-behavedness of the measure. This definition measures Hessian norm relative to the gradient norm, and captures notion of relative local linearity, which can be seen via an application of Taylor's theorem:
[[((+ f(x) - Vf(x)
1 2
max Cx(x) XERd
||e||2-
(1)
[Vf(x)]
relative local lincarity
normalized
Here the numerator on the left captures the local linearity error, scaled by the gradient norm in the denominator, which can be shown using Taylor's theorem to be upper bounded by the normalized curvature. We shall consider penalizing this notion of normalized curvature, and we shall simply refer this quantity curvature in the rest of the paper.
3.2 A Data-Free Upper Bound on Curvature
Directly penalizing the curvature is computationally expensive as it requires backpropagating an estimate of the Hessian norm, which itself requires backpropagating gradient-vector products. This requires chaining the entire computational graph the model at least three times. Moosavi Dezfooli et al. [6] reduce the complexity of this operation by computing a finite-differene approximation to the Hessian from gradients, but even this double-back baabllbbcckroppgattoon efficient penalization procedure takes a single backpropagation step.
expensive. We require an
To this end, we propose to minimize frte-free upper bound on the curvature. To illustrate the idea, we first show this upper bound for the simplified case of the composition of one-dimensional functions (f R RR)
Lemma 1. Given I-dimensional compositional function fL fL-1 fo
with + R for = 1,2,.... L, the normalized curvature Cj = 18 is bounded by
R
1 1f'1
11
We first have I' = S. Differentiating this expression we have I" EL f II' 8I 11 fk. Dividing by f', taking absolute value of both sides, and using the triangle inequality, we have the intended result.
Proof.
Extending the above expression functions F" + R not straightforward, as the intermediate layer Hessians are order three tensors. We derive this using a result from Wang et al. (24 that connects the spectral norms of order-n tensors to the spectral norm of their matrix "unfoldings". The full derivation is presented in the appendix, and the (simplified) result is stated below.
Theorem 1. Given a function = f o ft-10... o fi with f; : .Ra--1 + TR",, the curvature C₁ can be bounded by the sum of curvatures of individual layers Cj (x), i.e.
(x(x) S Σ nu C₁. (x) II f
ng max Cax (x') x²
II max x"
IV [1)=ff
The rightmost term is independent of x and thus holds uniformly across all data points. This bound shows that controlling the curvature and Lipschitz constant of each layer of neural network enables us to control the overall curvature of the model.

---

Practical neural networks typically consist of linear maps such as convolutions, fully connected layers, and batch normalization layers, and non-linear activation functions. Linear maps have zero curvature by definition, and non-linear layers often have bounded gradients (= 1), which simplifies computations. In the sections that follow, we shall see how to penalize the remaining terms, i.e., the curvature of the non-linear activations, and the Lipschitz constant of the linear layers.
3.3 Centered-Softplus Activation Function with Trainable Curvature
Theorem T shows that the curvature of neural network depends the curvature of its constituent activation functions. We thus propose to use activation functions with minimal curvature.
The kink at the origin of the ReLU function implies an undefined second derivative. On the other hand, a smooth activation such as the softplus function, s(x; 8) = log(1 + exp(3x))/5 3 is better suited analyzing questions of curvature. Despite not being common baseline choice, softplus does see regular use, especially where its smoothness facilitates analysis (25). The curvature of the softplus function is
(1 ds(z; 3) dx
(x) 3x
S 3
Thus using softplus with small 8 values ensures low curvature. However, we observe two crit- ical drawbacks of softplus preventing its usage with small 3: (1) divergence for small B, where s(2;8 - 0) do which ensures that well-behaved low curvature maps cannot be recovered and, (2) instability upon composition i.e., s" (I 0:5) 8oso...
0)
log(n-1), Thus 0
times
we have that s" = 0; - 8 as + oc, which shows that composing softplus functions ex- acerbates instability around the origin. This is critical for deep models with large number of layers n. which is precisely the scenario of interest to us. To remedy this problem, we propose centered- softplus so(r;; B), simple modification softplus by introducing a normalizing term as follows.
log 2 B
1 cxp(5rr
so = s(x;
log
(4)
3
2
This ensures that s(x 8) s"(x 0; B) 0 for any positive integer n, and hence also ensures stability upon composition. More importantly, we have so (2; + 0) = x/2 which a scaled linear map, while still retaining so(x;; well-behaved linear maps, as well as highly non-linear ReLU-like maps if required.
oo) ReLU(xr). This ensures that we are able to learn both
We further propose to cast 3 is learnable parameter and penalize its value, hence directly penalizing the curvature of that layer. Having accounted for the curvature of the non-lincarities, the next section discusses controlling the gradients of the linear layers.
3.4 Lipschitz Linear Layers
Theorem shows that penalizing the Lipschitz constants of the constituent linear layers in a model is necessary to penalize overall model curvature. There are broadly three classes of linear layers we consider: convolutions, fully connected layers, and batch normalization.
3.4.1 Spectrally Normalized Convolutions and Fully Connected Layers
We propose to penalize the Lipschitz constant of convolutions and fully connected layer via ex- isting spectral normalization-lik techniques. For fully connected layers, we use vanilla spectral normalization (Miyato et al. (2I)) which controls the spectral norm of fully connected layer by reparameterization replacing a weight matrix W, by W/| which has a unit spectral norm. Ryu et al. [22) generalize this to convolutions by presenting power iteration method that works directly on the linear mapping implicit a convolution: maintaining 3D left and right singular "vectors" of the 4D tensor of convolutional filters and developing the corresponding update rules.
We ignore self-attention layers in this work.
5

---

They call this "real" spectral normalization to distinguish it from the approximation that [21] pro- posed for convolutions. Using spectral normalization on fully connected layers and "real" spectral normalization on convolutional layers ensures that the spectral norm of these layers is exactly equal to one, further simplifying the bound in Theorem
3.4.2 7-Lipschitz Batch Normalization
In principle, at inference time batch normalization (BN) is multiplication by the diagonal matrix of inverse running standard deviation estimates Thus we can spectrally normalize the BN layer by computing the reciprocal of the smallest rumning standard deviation value across dimensions i, i.e., ||BN]|2 max, [[B[(>)]2 = 1/ min, running-std(i)) In practice, we found that models with such spectrally normalized BN layers tended to either diverge or fail to train in the first place, indicating that the scale introduced by BN is necessary for training. To remedy this, introduce the y-Lipschitz batch normalization, defined by
1 I-Lipschitz-BN/sa + / BN 12 7 >-Lipschitz-BN(xx) + min(7, BN x XI--ipschiiz-BN(x)
scaling factor
By clipping the scaling above at y (equivalently, the running standard deviation below, at 1/7), we can ensure that the Lipschitz constant of a batch normalization layer is at most equal to E R+ We provide a PyTocc-styyl code snippet in the appendix. As with 3, described in $3.3 we cast as a learnable parameter in order to penalize it during training. Gouk et al. [26] proposed a similar (simpler) solution, whereas they they fit common to all BN layers, we letry vary freely by layer.
3.5 Penalizing Curvature
We have discussed three architectural innovations centered-softplus activations with trainable 9, spectrally normalized linear and convolution layers and a 7-Lipschitz batch normalization layer. We now discuss methods to penalize the overall curvature of model built with these layers.
Since convolutional and fully connected layers have spectral norms equal to one by construction, they contribute nothing to curvature. Thus, restrict attention to batch normalization and activation layers. The set which will be subsequently referred to respectively 8SP and ,BN. For models with batch normalization, naively using the upper bound in Theorem is problematic due to the exponential growth in the product of Lipschitz constants of batch normalization layers. To alleviate this, we propose use a penalization R, where 7/ terms are aggregated additively across batch normalization layers, and independent of the 8, terms the following manner:
R2 = A3 X Σ B X
Σ log T3
(5)
&ESSP
JEyBN
An additive aggregation ensures that the penalization well-behaved during training and does not grow exponentially. Note that the underlying model is necessarily linear if the penalization term is zero, thus making it an appropriate measure of model non-linearity. Also note that 3; > 0, 7/ 2 by construction. We shall henceforth use the term "LCNN" refer model trained with the proposed architectural components (centered-softplus, spectral normalization, and 7-Lipschitz batch normalization) and associated regularization terms on B, 7. We next discuss the robustness and interpretability benefits that we obtain with LCNNs.
Why Train Low-Curvature Models?
In this section we discuss the advantages that low-curvature models offer, particularly it pertains to robustness and gradient stability. These statements apply not just to LCNNs, but low-curvature models in general obtained via any other mechanism.
We ignore the learnable parameters of batch normalization for simplicity. The architecture we propose subsequently also does not have trainable affine parameters.

---

4.1 Low Curvature Models have Stable Gradients
Recent work [5 16 has shown that gradient explanations are manipulable, and that we can easily find inputs whose explanations differ maximally from those at the original inputs, making them unreliable in practice for identifying important features. As we shall show, this amounts to models having a large curvature. In particular, we show that the relative gradient difference across points ||]]/2 away in the l sense is upper bounded by the normalized curvature Cy. as given below.
Proposition 1. Consider a model f with maxx Cj (x) S 8c, and two inputs x and x + E EE Rd) The relative distance between gradients these points is bounded by
[Vf(x+)) V2((x)l2 < ||ell20c exp(Ile|200)) ~ (1ell2C(xx) (Quadratic Approximation) [V
The proof expands fxx in a Taylor expansion around x x+C c and bounds the magnitude of the second and higher order terms over the neighborhood of x, and the full argument is given in the appendix. The upper bound is considerably simpler when we assume that the function locally quadratic, which corresponds to the rightmost term |||]|2c (x). Thus the smaller the model curvature, the more locally stable are the gradients.
4.2 Low Curvature is Necessary for lg Robustness
Having small gradient norm is known to be important aspect of adversarial robustness [8] However, small gradients alone are not sufficient, and low curvature is necessary, to achieve robust- ness. This is easy see intuitively for small noise values, but if the curvature is large, then gradient norms at neighboring points can quickly increase, leading to misclassification for even slightly larger noise levels. This effect is an instance of gradient-masker masking [27 which provides an illusion of robustness by making models only locally robust.
model may have low gradients
point leading to robustness
In the result below, we formalize this intuition and establish an upper bound on the distance between two nearby points, which show depends on both the gradient norm (as was known previously) and well as the max curvature of the underlying model.
Proposition 2. Consider a model I with maxx Cx(x) S 8c, then for two inputs x and x + (E Rd), we have the following expression for robustness
1If(x -xxx))
< || f l2 (1 + ||cll20e exp(Ilcl128e))
~ || f 2 (1 (e]l2C,(xx)) (Quadratic Approximation)
The proof uses similar techniques those of proposition I. and is also given in the appendix. This result shows that given two models with equally small gradients at data points, the greater robustness will be achieved by the model with the smaller curvature.
Experiments
In this section we perform experiments to (1) evaluate the effectiveness of our proposed method in training models with low curvature as originally intended, (2) evaluate whether low curvature models have robust gradients in practice, and (3) evaluate the effectiveness of low-curvature models for adversarial robustness. Our experiments are primarily conducted on a base ResNet-18 architecture ((8)) using the CIFAR 10 and CIFAR 100 datasets (29)), and using the Pytorch 30 framework. Our methods entailed fairly modest computation our most involved computations can be completed in under three GPU days, and all experimental results could be computed in less than 60 GPU- We used a mixture of GPUs primarily NVIDIA GeForce GTX 1080 Tis on an internal compute
cluster.
Baselines and Our Methods Our baseline model for comparison involves using ResNet- 18 model with softplus activation with high 3 = 10 to mimic ReI.U, and yet have well-defined curvatures.
7

---

Another baseline is gradient norm regularization (which we henceforth call 'GradReg') [31, where the same baseline model trained with an additional penalty on the gradient norm. We train two variants of our approach- a base LCNN,, which involves penalizing the curvature, and another variant combining LCNN penalty and gradient norm regularization (L.NNN + GradReg), which controls both the curvature and gradient norm. Our theory indicates that the LCNN GradReg variant is likely to produce more robust models, which we verify experimentally. We also compare with CURE [6), softplus with weight decay 7 and adversarial training with E PGD [33 with noise magnitude of 0.1 and iterations of PGD. We provide experimental details in the appendix.
Parameter Settings All our models are trained for 200 epochs with an SGD momentum optimizer, with momentum of 0.9 and an initial learning rate of 0.1 which decays by a factor of 10 150 and 175 epochs, and a weight decay of 5 x 10
5.1 Evaluating the Efficacy of Curvature Penalization
In this section, we evaluate whether LCNNs indeed reduce model curvature in practice. Table contains our results, from which we make the following observations: (1) most of our baselines except CURE and adversarial training do meaningfully lose predictive performance (2) GradReg and adversarially trained models are best at reducing gradient norm while LCNN-based models are best at penalizing curvature. Overall, these experimental results show that LCNN-based models indeed minimize curvature as intended.
We also observe in Table T that GradReg 31 has an unexpected regularizing effect on the Hessian and curvature. We conjecture that this is partly due to the following decomposition of the loss Hessian, which where LSE(x) is the LogSumExp function, and fı(x) the pre-softmax logit output. We observe that the first term strongly depends on the gradients, which may explain the Hessian regularization effects of GradReg, while the second term depends on the Hessian of the bulk of the neural network, which is penalized by the LCNN penalties. This also explains why combining both penalizations (LCNN + GradReg) further reduces curvature.
be written V f(x) fi(xx) VJ, LiE( T + V LSE(()) f((x),
We also measure the average per-epoch training times on a GTX 1080Ti, which are: standard models I softplus + weight decay (~ 100 sec), LCNN (~~ 160 sec), GradReg (~ 270 sec), LCNN+GradReg (~> 350 sec), CURE / Adversarial Training (~~ 500 sec). Note that the increase in computation for LCNN is primarily due the use of spectral normalization layers. The results show that LCNNs are indeed able to penalize curvature by only marginally (1.6x) increasing training time, and using LCNN+GradReg only increases time 1.3x over GradReg while providing curvature benefits.
Table 1: Model geometry various ResNet-18 models trained with various regularizers on the CIFARION 100 test dataset. Gradient norm regularized models (31) ('GradReg') are best at reducing gradient norms, while LCNN-based models are best at reducing curvature, leaving gradients unpe- nalized. We obtain the benefits of both by combining these penalties. Results are averaged across
two runs.
Model
Ex [V((())))
| 2<l12f(()) [xx]112
ExC,(x)
Accuracy (%)
Standard LCNNs
19.66 0:3
6061.96 = 96805
270.89 + 75.04
77.42 aal1
22.04 1.41
1143.62 99.38
69.50 2,41
77.30 +a11
GradReg [31
8,86 = 0.12
776.56 + 63.02
89.47
+ 5.86
77.20 a26
LCNNs GradReg
9.87
154.36 0.22
25.30 0.09
77.29 0:007
CURE 6
8,86
+ 979.45
116.31
+
76.48 000
Softplus + Wt. Decay 7
18.08 + 0.05
1052,84 # 7,27
70.39 + 0.88
77.44
+ a22
Adversarial Training 32
7.99
501.43 +
63.79
76.96 026
5.2
Impact of Curvature on Gradient Robustness
In $4.1 we showed that low-curvature models tend to have more robust gradients. Here we evaluate whether this prediction empirically by measuring the relative gradient robustness for the models with various ranges of curvature values and noise levels. In particular, we measure robustness random noise at fixed magnitudes ranging logarithmically from 1 10 3 to 1 x 10-1, We plot our results
8

---

Standard (Theoretical)
(Thereticca)
Relative Gradient Robustness
10'
Gradkeg (Theoratical)
Relative Gradient Robustness
10t
GradRog (Theoretica))
LCNN [Therretical)
LONN (Thesretica))
GradReg The
LENN GradRag (Taaeetiaa)
10a
10:1
10-1
20"1
Standard GradReg
Standard
GradReg
10-1
LONN
10-
LCMN
LCMN GradReq
LCMV GradReg
10-3
10t2
10
10-2
10-2
10:2
Noise Magnitude
Noise Magnitude
Gradient Robustness CIFARIO
Gradient Robustness on CIFAR100
Figure 2: Plot showing relative gradient robustness V [(x++)) ff(xlla as function of added noise
[VV(2))
||d]2 on (a) CIFARIO and (b) CIFARI 100 with ResNet-18 model. We observe that low-curvature models lead to an order of magnitude improvement in gradient robustness, and this improvement
closely follows the trend predicted by the theoretical upper bound in $4
in Figure 2 where find that results match theory (i.e, the simplified quadratic approximation in 14 quite closely in terms of the overall trends, and that low curvature models have an order of magnitude improvement in robustness over standard models.
5.3 Impact of Curvature on Adversarial Robustness
Our theory in $4.2 shows that having low curvature is necessary for robustness, along with having small gradient norms. In this section we evaluate this claim empirically, by evaluating adversarial examples via &g PGD [13 adversaries with various noise magnitudes. We the Cleverhans library [3 to implement PGD. Our results are present in Table 2 where we find that LCNN+GradReg models perform par with adversarial training, without resulting accuracy loss.
Table 2: Results indicating off-the-shelf model accuracies (%) upon using & PGD adversarial ex- amples across various noise magnitudes. Adversarial training performs the best overall, however sacrifices clean accuracy. LCNN+GradReg models perform similarly but without significant loss of clean accuracy, Results are averaged across runs.
Model
Acc. (%)
||<||2 0.05
||<||2
0.1
||e||a
0.15
||| 0.2
Standard LCNN
77.42
2.10
59.97 1:11
37.55 1.13
23.41
+:
16.11 4: 2:
77.16
61.17
39.72
25.60
17.66 +
GradReg LCNNs + GradReg
77.20 #
71.90
+ 31
61.06 =
49.19 + .11
38.09 + 47
77.29
72.68
63.36
52.96
42.70 +
CURE 6
76.48
71.39
61.28
49.60
39.04
Softplus + Wt. Decay 7
77.44 + 288
60.86
+ 36
38.04 # .43
23.85
+ .33
16.20 + 0:
Adversarial Training 32
76.96
72.76
64.70
54.80
44.98
5.4 Train-Tess Discrepancy in Model Geometry
During our experiments, we observed consistent phenomenon where the gradient norms and Hes- sian norms for test data points were much larger than those for the train data points, which hints form of overfitting with regards to these quantities. We term this phenomenon the train-test discrepancy in model geometry. Interestingly, we did not observe any such discrepancy using our proposed curvature measure, indicating that our proposed measure may be a more reliable measure of model geometry. We report our results in Table 3 where we measure the relative discrepancy, finding that the discrepancy for our proposed measure of curvature is multiple orders of magnitude smaller than the corresponding quantity for gradient and Hessian norms. We leave further investi-

---

gation of this phenomenon regarding why curvature is stable across train and test as topic for future work.
Table 3: Train-test descrepancy in model geometry, where the relative descrepancy Aug(X) = g(Xee) is shown for three different geometric measures. We observe that (1) there exists
g(Xua)
large train-test descrepancy, with the test gradient / hessian norms being > 10x the corresponding
values for the train set. (2) the descrepancy is 2-3 orders of magnitude smaller for our proposed
curvature measure, indicating that it may be stable model property.
Model
Xxx [f())l2
Xxxx V V f(x)]l2
(xxx
Standard
11.75
12.28
0.025
GradReg
11.33
11.22
0.017
LCNN
19.99
1 11.33
0.129
LCNNs + GradReg
21.82
10.43
0.146
Summary of Experimental Results Overall, our experiments show that:
(1) LCNNs have lower curvature than standard models advertised, and combining them with gradient norm regularization further decreases curvature (see Table [T]. The latter phenomenon is unexpected, as curvature measure ignores gradient scaling.
(2) LCNNs combined with gradient norm regularization achieve an order of magnitude improved gradient robustness over standard models (see Figure 2
(3) LCNNs combined with gradient norm regularization outperform adversarial training in terms of achieving a better predictive accuracy at a lower curvature (see Table and are competitive in terms of adversarial robustness (see Table 2). while being 1.4x faster.
(4) We observe that there exists a train-test discrepancy for standard geometric quantities like the gradient and Hessian norm, and this discrepancy disappears for our proposed curvature measure (see Table
We also present ablation experiments, additional adversarial attacks, and evaluations more datasets and architectures the Appendix.
6 Discussion
In this paper, we presented modular approach remove excess curvature in neural network mod- els. Importantly, we found that combining vanilla LCNNs with gradient norm regularization resulted in models with the smallest curvature, the most stable gradients as well those that are the most adversarially robust. Notably, this procedure achieves adversarial robustness without explicitly gen- erating adversarial examples during training.
The current limitations of our approach are that we only consider convolutional and fully connected layers, and not self-attention recurrent layers. We also do not investigate the learning- -thoorttc benefits (or harms) of low-curvature models, or study their generalization for small number of train- ing samples, their robustness label noise (which we already observe in Fig. Tb Investigating these are important topics for future work.
Acknowledgments and Disclosure of Funding
The authors would like to thank the anonymous reviewers for their helpful feedback and all the funding agencies listed below for supporting this work. SS and HL are supported in part by the NSF awards # #1SS-2008461 and -IS-204998,, and research awards from Google, JP Morgan, Amazon, Harvard Data Science Initiative, and D Institute at Harvard. KM and SS (partly) are supported by the Swiss National Science Foundation under grant number 188-188758 "CORTT". HI. would like to thank Sujatha and Mohan I Lakkaraju for their continued support and encouragement.
10

---

References
[1] Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recogni- tion, pages 770-778, 2016.
[2] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmano, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022.
[3] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint urXiv:2204.06/25. 2022.
[4] Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew Hoffman, et al. Un- derspecificattion presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395, 1(3), 2020.
[5] Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can gen- eralize for deep nets. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learn- ing Research, pages 1019-1028. PMLR, 06-11 Aug 2017. URL https: /t https://proceeediii prt.cedioccee. mlr. wwt.ppree////ootee /inhh/////dnnn/// hhmmlll
[6] Seyed Mohsen Moosavi Dezfooli, Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Ro- bustness via curvature regularization, and vice versa. In 2019 IEEE/CVF Conference on Com- puter Vision and Patter Recognition (CVPR), pages 9070-9078, Los Alamitos, CA. USA, jun 2019, IEEE Computer Society. URL https://doi: ieseccootterrroccisttyyyco org/10. 1109/CVPR. 2019. 2019.00929
[7] Ann-Kathrin Dombrowski, Christopher J. Anders, Klaus-Robert Müller, and Pan Kessel. To- wards robust explanations for deep neural networks. Pattern Recognition, 121:108199 2022. ISSN 0031-3203. URL httpt:tttts:/ioo.o hrg.://o..orrrrg/ 1016/j01...... - patcog 2021. 108194
[8] Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classi- fier against adversarial manipulation. Advances in newral information processing systems, 30, 2017.
[9] Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhus- sein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. Advances in Neural Information Processing Systems, 32, 2019.
[10] Ann-Kathrin Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus-Robert Müller, and Pan Kessel. Explanations can be manipulated and geometry is to blame. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. wttpatttss///roreeeeiiiiiii www.innnrsip....e httppp/ppccppppp/////e///// bb33600cdd222009994455bbbb.bb Paper. .dff
[11] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014. URL http: arxiv org/ /tlos///1.... 1312.6999219999
[12] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and Harnessing Ad- versarial Examples. arXiv e-prints, Dec 2014. URL http::tttttiii...tt/aaa org/ abs 1412/1112.6000 6572
[13] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Con- ference on Learning Representations, 2018. URL https:///peereriiew..ne wttp://wwweeeeeo/eiiiiiieee iod
TJzIBfZAb
[14] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via random- ized smoothing. In International Conference on Machine Learning, pages 1310-1320. PMLR, 2019.
11

---

[15] Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. Proceedings of the AAAI conference on artificial intelligence, 33(01):33811333 1-3688 2019. URL
https://doo org/10. hete//...100///00 (t1/609/aaa..vv.. v33101 33013681
[16] Xinyang Zhang, Ningfei Wang, Hua Shen, Shouling Ji, Xiapu Luo, and Ting Wang. Inter- pretable deep learning under fire. In 29th {USENIX) Security Symposium (!USENIX) Security 20), 2020.
[17] Suraj Srinivas and Francois Fleuret. Rethinking the role of gradient- attribution methods for model interpretability. In International Conference on Learning Representations, 2020.
[18] Andrew Slavin Ros and Finale Deshi-Velez. Improving the adversarial robustness and in- terpretability of deep neural networks by regularizing their input gradients. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Ap- plications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, 18/IATI I 18/EAAT 978-1-5 118-1-57735-800-
AAAI Press, 2018. ISBN
[19] Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval Networks: Improving Robustness Adversarial Examples. arXiv e-prints, art. arXiv:1704.0888 April 2017.
[20] Asher Trockman and Zico Kolter. Orthogonalizing convolutional layers with the cayley transform. In International Conference on Learning Representations, 2021. URL https:
[21] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. malization for Generative Adversarial Networks. arXiv e-prints, Feb 2018. URL http:
Spectral Nor-
// wwwxiv..rg/abs////o
[22] Ernest Ryu, Jialin Liu, Sicheng Wang, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Plug- and-play methods provably converge with properly trained denoisers. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference Ma- chine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5546-5557.
PMLR, 09-15 Jun 2019. URL tttttps://ppoceedinge mlr
html
[23] John M Lee. Riemannian Manifolds: An Introduction to Curvature, volume 176. Springer Science & Business Media, 2006.
[24] Miaoyan Wang, Khanh Dao Duc, Jonathan Fischer, and Yun S. Song. Operator norm inequal- ities between tensor unfoldings the partition lattice. Linear Algebra and its Applications, 520:44-66, 2017. ISSN 0024-3795. URL patttps:/doi...
2017.01.
017
[25] Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, and David Duvenaud. Scalable reversible generative models with free-form continuous dynamics. In International Conference Learn- ing Representations, 2019. URL. htt.htttt:://oeeeer// wwt.:/nee//eeeeee tetet/ffoumm hirruummmddddJxxncCCCCCC
[26] Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning. 100())3933414 2021.
[27] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pages 274-283. PMLR, 2018.
[28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Patlern Recognition (CVPR), pages 770-778, 2016. URL httt::ttt///oo ooooor//111 1109/CVPR. 2016. 69
[29] Alex Krizhevsky. Learning multiple layers of features from tiny images. Techni- es. toronto. edw.eduudaaa adu//rriz/
cal report, University of Toronto, 2009. URL.
Learning-featio www.trestte2eee 2009-TR.
12

---

[30] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS 2017 Workshop Autodiff Program, 2017.
[31] H. Drucker and Y. Le Cun. Improving generalization performance using double backpropaga- tion. IEEE Transactions on Neural Networks, 3(6):911-977, 1992. doi: 241109977.1656000
[32] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards Deep Learning Models Resistant to Adversarial Attacks. arXiv e-prints, June 2017. URL http:
06083
[33] Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Kurakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan,, Zhishuai Zhang, Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhav Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hen- dricks, Jonas Rauber, and Rujun Long. Technical report on the CleverHans v2.1.0 adversarial examples library. arXiv preprint 06100.07688)
2018. URL htttthh/t///rtt org/ org/abs/
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A] We present foundational work on the geometry neural network models and as such we do not foresee any negative societal impacts beyond those that already exist with the usage of neural network models.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] proofs are provided in the appendix
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] some training details are mentioned in the appendix
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'te re using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifi- able information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
We did not use crowdsourcing conducted research with human subjects
13

---

(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "xmf6QGAsbpN", "reviewer": "Reviewer_o9LW", "summary": "This paper proposes a set of modifications to neural networks to control their curvature. Three modifications are proposed: 1) Centered-softplus activation functions 2) Spectrally normalization of fully-connected, convolutional, batch normalization layers 3) regularization of activation and batch normalization learnable curvatures parameters. In particular, the paper proposes an alternative measure of curvature that is inversely proportional to the norm of the gradient. The paper evaluates curvature controlled models for robustness.", "questions": "- Have you evaluated the method when combined with adversarial training?\n- ​​Eq 4: Can you explain why softplus converges to x/2 when beta goes to zero?", "limitations": "Yes", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- A curvature measure coupling both the Hessian and gradient is interesting. Because of division by the gradient norm, the method might learn to automatically stay away from gradient masking and zero gradients.  Table 1 supports this by showing that the norm of both the gradient and the Hessian are smaller than other robustness methods considered.\n- Centered softplus is also an interesting modification to softplus as it ensures increasing the depth does not result in activation blowup near zero.\n\nWeaknesses:\n- There have been challenges with second-order approaches to robustness such as gradient stability and gradient masking [2]. Although the ideas in this work could resolve issues in prior work, we do not see evaluations of gradient stability and gradient masking. For example, plots of the gradient norm during the training of various models on various datasets would show us if the curvature regularization is actually stable and allows us to train any model architecture. More importantly, the paper should test the model for gradient masking using for example black-box attacks. Adaptive attack should also be considered. For example, make sure there is no spectral normalization at test time by substituting the weights with their normalized copy and replace softplus with a piecewise linear approximation such as relu at test time.\n- Robust evaluation does not include strong attacks such as the ones in the RobustBench [2]. The strongest attack is PGD and the number of steps is not mentioned. Also, the evaluation is limited to L2 robustness without a discussion on why the method may or may not work for Linf robustness.\n- Ablation studies are required to disentangle the impact of three proposed modifications.\n- Related works such as [1,3,4] are missing from comparison and discussion.\n\n[1] Ma, A., Faghri, F., Papernot, N., & Farahmand, A. M. (2020). Soar: Second-order adversarial regularization. arXiv preprint arXiv:2004.01832.\n[2] Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., ... & Hein, M. (2020). Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670.\n[3] Anil, C., Lucas, J., & Grosse, R. (2019, May). Sorting out Lipschitz function approximation. In International Conference on Machine Learning (pp. 291-301). PMLR.\n[4] Qin, C., Martens, J., Gowal, S., Krishnan, D., Dvijotham, K., Fawzi, A., ... & Kohli, P. (2019). Adversarial robustness through local linearization. Advances in Neural Information Processing Systems, 32.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DIAe_58tGii", "reviewer": "Reviewer_J6Z1", "summary": "The authors propose to build a kind of low-curvature neural networks (LCNNs), which can lower network curvature for achieving robust models. The key to reducing the curvature in LCNNs is centered-softplus activation function and $\\gamma$-Lipschitz BN layer, which are specifically designed by the authors with better curvature property compared to those standard layers. Meanwhile, the authors give a theoretical upper bound on the curvature of networks, and demonstrate that these layers can minimize such bound. Finally, the authors empirically show that LCNNs can give lower curvature via training Resnet18 on Cifar100, and also show that LCNNs can improve the performance further in combination with the GradReg in adversarial experiments.\n", "questions": "I am willing to raise my score if the authors can address my concerns.", "limitations": "In my opinion, this paper do not involve related issues for its current version.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "&nbsp;\n## **Strengths**:\n1. The paper is clear and well written. \n2. The authors show some interesting insight about the curvature property of the stacked layer networks.\n3. The proposed centered-softplus has some nice properties, and can be promising to be implemented in future model.\n4. The LCNN models show a universal improvement in adversarial experiments.\n\n\n&nbsp;\n## **Weaknesses**:\n\n*Q1*:\n\nThe authors claims that conventional measures such as Hessian norm is sensitive to gradient scaling (section 3.1). And the authors propose a kind of definition of curvature that is approximately normalized, $C_f(x) = ||\\nabla_x^2 f(x)||_2 / (||\\nabla_x f(x)||_2 + \\epsilon)$. As far as I know, Dinh (citation 1 in the authors' paper) also gives a definition that is very similar to the authors', which is $|\\nabla^2 L(\\theta)||_2\\epsilon^2 / 2(1 + L(\\theta))$ (Left column at page 3 in their paper). Note that Dinh's form is to measure the curvature in the weight space while the authors measure in the input space, but this does not affect the measure metric. So I am wondering what is the main difference between the two formula in essence.\n\n*Q2*:\n\nThe authors try to change the function property to reduce the network curvature, where the authors propose the centered-softplus activation function. My second concern is that to my understanding, the low curvature will normally denote a low gradient norm. The Lipchitz constant of a function equals to the maximum gradient norm for a small region. A low curvature generally means that the change in the neighborhood is relatively small, which implies a low Lipchitz constant and then a small gradient norm. So in our training, we will use the typical backpropagation to sequentially update the weight at each layer according to the gradient of pervious layers. My concern is that if a function has a low curvature, we may more likely to suffer from the gradient vanishment for these shallow layers far from the output layer. This is the reason why the relu function is mostly used for building models. Considering that the c-softplus is a low curvature function, especially around zero, I am somehow worried about the side-effect it brings in training, i.e. the gradient vanishment. Also, in todays modeling concept, the community are building very deep models. I think that the techniques proposed by the authors may not be well suited for these scalable deep models. Low curvature may decrease their expressivity, since the weights in the shallow layers are limited to some extent. The authors only use ResNet18 for demonstration, so it will be more interesting to see such layer to be applied in deeper models.\n\n*Q3*:\n\nMy third concern is about some details in experiments. Firstly, since training LCNN will further involve two extra hyper-parameters, so I am wondering how the authors tune these parameters in practice. Is there any advice about the two parameters? Based on the experiment results, introducing two extra parameters can only improve little performance, but will severely burden the tuning. Secondly, to my understand, $\\nabla^2 f$ will not be computed directly, so I am also wondering how this is computed. Thirdly, the computation cost of training with the authors' scheme is 1.6 times higher than that using the standard scheme. So in my opinion, such an additional cost is not quite proper to be called as \"marginally increasing the training time\". \n\n#### **Minor**:\n\nIt will be very interesting to see that the authors train the LCNN with SAM (Sharpness-Aware Minimization for Efficiently Improving Generalization, Pierre Foret et, ICLR2021), which is a techniques used for smoothing the loss curvature.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Csq4uO2Pbir", "reviewer": "Reviewer_b1TK", "summary": "The authors provide a new method for minimizing decision boundary curvature in an effort to improve adversarial robustness without compromising unperturbed generalization (test set) accuracy. They propose combining an activation function with a trainable curvature parameter as well as a Lipschitz constrained batch-normalization layer. Their empirical results demonstrate success in their goal of maintaining accuracy while improving robustness.", "questions": "1) Can you compare your definition of curvature to that found in differential geometry? The concept of curvature in n-dimensional spaces has been long established in the differential geometry community, and is associated with the process of Principal Curvature Decomposition. The most accessible work on the subject, in my opinion, is from John M Lee [1, 2]. Importantly, following Lee, the curvature of a Riemannian submanifold (which a NN decision boundary reasonably approximates) must be calculated by projecting the Hessian onto the function’s level-set (i.e. decision boundary) and then normalizing by the first fundamental form, or metric (see [2], chapter 8). This is very close to what you propose, but it is different in that your proposal is missing the projection operation. Moosavi-Dezfooli also mentions this in [3], section 5, although I believe they are also missing details found in the more formal treatment provided by Lee. For additional context, another similar treatment can be found in [4].  I think it would improve the quality of the submission if the authors provided some context for how their work fits into this broader mathematical description of curvature.\n\n2) What is the relationship between decision boundary curvature and neural activation curvature? From my understanding, the prior work from Moosavi-Dezfooli attempts to minimize a network’s decision boundary curvature directly. On the other hand, your regularization on the beta term in the centered softplus activation function is intended to encourage more linear (i.e. flatter) activation functions for each neuron in all layers and the batch-norm regularization is layer specific. Following Moosavi-Dezfooli and colleagues, for some logit (i.e. penultimate layer) neurons f_i(x), f_j(x), the binary decision boundary can be written down as a b_{ij}(x) = f_i(x) - f_j(x). So one can pretty easily deduce how the curvature of f_i and f_j impacts b_{ij}. However, the relationship between neuron activation curvature and decision boundary curvature for earlier layers is not clear to me. It seems reasonable to think that individual neurons could exhibit curvature in their activation maps while still contributing to (relatively) flat decision boundaries downstream. Conversely, ReLU networks are piecewise linear, and therefore technically contribute zero curvature in the decision boundary space. Even if we squint and allow for an approximate definition of curvature (in the sense that piecewise linear functions approximate a curve in the limit), we have an example of a linear-almost-everywhere activation function producing non-flat decision boundaries. Figure 1 and table 1 both seem to indicate that the proposed regularization is indeed minimizing boundary curvature, but I am interested to hear from the authors why that is happening in their case.\n\n3) Is it possible to do more ablation studies? My apologies if I missed this, but is there an analysis of the individual effects of the new activation function vs the batch norm modification vs the combination?\n\n4) You may consider a more thorough robustness evaluation. As I’m sure the authors are aware, it is difficult to validate adversarial robustness claims, and many studies that initially claim robustness ultimately prove to fail under more careful scrutiny [5, 6]. While I think it is unreasonable to ask the authors to satisfy every consideration made in [5], I do think additional care should be taken (e.g. providing results for l1 bounded attacks, for gradient-free attacks, and/or auto-PGD attack). I would also make sure to include a section (appendix would be fine) that explains the thread model and what care was taken to explore the parameter space of the attacks.\n\n5) typo on line 231: “that the function [is] locally quadratic”\n\n[1] John M. Lee. Introduction to Smooth Manifolds. Number 218 in Graduate Texts in\nMathematics. Springer, 2 edition, 2013. ISBN 978-1-4419-9981-8 978-1-4419-9982-5.\n\n[2] John M. Lee. Introduction to Riemannian Manifolds. Springer International Publishing, 2nd edition, 2018. ISBN 978-3-319-91754-2.\n\n[3] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. Robustness of classifiers to universal perturbations: A geometric perspective. In International Conference on Learning Representations, ICLR, 2018.\n\n[4] Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. Exponential expressivity in deep neural networks through transient chaos. Advances in neural information processing systems, 29:3360–3368, 2016.\n\n[5] Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. ArXiv preprint, abs/1902.06705, 2019. URL https://arxiv.org/abs/1902.06705.\n\n[6] Croce, Francesco, and Matthias Hein. \"Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.\" International conference on machine learning. PMLR, 2020.\n", "limitations": "The authors provide a list of limitations in the discussion.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality:** As the authors state, the concept of adapting boundary curvature to improve robustness is not new. Additionally, as I mention in point 1 below, the idea of normalizing the network hessian by the gradient is also not new. However, as far as I know their method of using an activation function with a learnable curvature parameter coupled with the bounded batch normalization layer is indeed novel. I believe this study provides an interesting (and original) piece of evidence supporting a budding theory about the relationship between decision boundary curvature and adversarial robustness.\n\n**Quality:** The figures illustrate their intended points well. I did not attempt to replicate or validate any of the mathematics, but I did read it for correctness and found no mistakes. However, I have requests below for more rigor in the experimentation and mathematical exposition.\n\n**Clarity:** Overall the paper is clearly communicated.\n\n**Significance:** While the ideas presented are not too far from existing work, I still think the paper is significant. Adversarial training continues to be the only long-standing applicable defense against adversarial attacks, and demonstrates a widely-believed tradeoff between accuracy and robustness. In my opinion, the significance of this submission is that their results suggest an alternative method which defies the accuracy-robustness tradeoff. However, I am assuming that their measurements of curvature and robustness are correct, and I make a couple of notes below asking for more support to ensure that this is true.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "3PJUxD-sawK", "reviewer": "Reviewer_1uxf", "summary": "This paper proposes to measure the curvature of neural networks by dividing the spectral norm of the input-output Hessian by the L2 norm of the gradient. It argues that this normalized curvature measure has the advantage of being invariant to gradient rescaling. The paper then introduces an approach for training low-curvature networks by using a modified softplus nonlinearity and Lipschitz-constrained linear and batchnorm layers and demonstrates that networks trained in this manner are more robust to adversarial attacks. It also observes that the proposed curvature measure is significantly more consistent between the train and test set than the gradient and Hessian norm.", "questions": "* Please provide some discussion around flat regions (see limitations). It may not be an issue, but the addition of the small constant suggests to me that you are aware that there might be a problem .\n* Could GradReg be combined with CURE? If so, this should be included as a baseline to have a fair comparison with your LCNNs + GradReg.\n* My main suggestion would be to either focus on establishing the proposed curvature measure as a generally useful tool or broaden/deepen the empirical comparison. At the moment the paper sits in between two chairs in my eyes and does not make a convincing case on either of them.\n\nMinor:\n* I find the title to be of somewhat questionable taste given that many readers (if the paper is accepted) will have been affected by the pandemic and some may have strong opinions on covid-related policies one way or another (I will caveat this with saying that I dislike click-bait titles and prefer them to be informative). I would encourage the authors to drop the \"flatten the curve\" bit.\n* I would suggest clarifying in the abstract that you are considering input-output Hessians/gradients (when I read gradient/Hessian I think of the loss w.r.t. the parameters, which surely is just my personal bias, but that may be shared by other readers).\n\nNote regarding the score: I would have put this as a borderline reject in past years, but feel comfortable erring on the lower side since that score seems to be discouraged this year.", "limitations": "The paper somewhat sweeps the (potential) issue of the curvature measure not being defined in zero-gradient regions of the input space under the rug by adding \"a small constant to ensure numerical stability\" to the denominator. This is a fairly nonsensical justification since for a symbolic definition numerical stability is not a concern until the measure is implemented. I would strongly suggest adding an explicit discussion rather than side-stepping the issue in this manner. Clearly, for networks with a relu nonlinearity after the input layer, such flat regions in the input space exist, although this may not be the case for the suggested centered softplus nonlinearities.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper is clearly structured and easy to follow.\n* The (overall) approach is novel as far as I am aware (although see weaknesses on the Lipschitz batchnorm layers).\n* General purpose parameterization/regularization approaches as presented here are potentially useful beyond adversarial examples.\n\nWeaknesses:\n* The proposed curvature measure is not (really) defined in zero-gradient regions and there is no discussion around this (see limitations).\n* The motivation of the curvature measure is fairly hand-wavy -- do we want to learn (approximately) locally linear functions when training neural networks? This might be the case, but would need to be supported with references. Moving up section 4 (4.2 in particular) could help, but would make this more of an adversarial robustness paper (which might help clarifying the storyline, see next point).\n* I find the overall storyline of the paper a bit incoherent, it spends a fair amount of space justifying the curvature measure, but is ultimately not really theoretically focused. On the other hand, the empirical evaluation is fairly limited (just one basic setting that isn't just for illustration) and does not make a particularly strong case for the proposed method. I'd suggest either expanding the theoretical discussion around the curvature measure or expanding the experimental evaluation and demonstrating the benefit of low-curvature networks in other use cases than adversarial examples or compare to SOTA attacks/defences (I don't follow the literature on adversarial robustness, but my impression is that this has been a fast-moving field over recent years, so comparing on a single attack type to effectively two baselines seems quite limited).\n* If I'm not mistaken, the Lipschitz parameterization of batchnorm layer has previously been proposed in (Gouk et al., Regularisation of Neural Networks by Enforcing Lipschitz Continuity, arXiv:1804.04368, 2018).\n* The proposed network parametrization appears to not work particularly well on its own and relies on the simultaneous use of gradient norm regularization (at least for the adversarial robustness use case).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept