Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors provide a new method for minimizing decision boundary curvature in an effort to improve adversarial robustness without compromising unperturbed generalization (test set) accuracy. They propose combining an activation function with a trainable curvature parameter as well as a Lipschitz constrained batch-normalization layer. Their empirical results demonstrate success in their goal of maintaining accuracy while improving robustness.
strengths_and_weaknesses: **Originality:** As the authors state, the concept of adapting boundary curvature to improve robustness is not new. Additionally, as I mention in point 1 below, the idea of normalizing the network hessian by the gradient is also not new. However, as far as I know their method of using an activation function with a learnable curvature parameter coupled with the bounded batch normalization layer is indeed novel. I believe this study provides an interesting (and original) piece of evidence supporting a budding theory about the relationship between decision boundary curvature and adversarial robustness.

**Quality:** The figures illustrate their intended points well. I did not attempt to replicate or validate any of the mathematics, but I did read it for correctness and found no mistakes. However, I have requests below for more rigor in the experimentation and mathematical exposition.

**Clarity:** Overall the paper is clearly communicated.

**Significance:** While the ideas presented are not too far from existing work, I still think the paper is significant. Adversarial training continues to be the only long-standing applicable defense against adversarial attacks, and demonstrates a widely-believed tradeoff between accuracy and robustness. In my opinion, the significance of this submission is that their results suggest an alternative method which defies the accuracy-robustness tradeoff. However, I am assuming that their measurements of curvature and robustness are correct, and I make a couple of notes below asking for more support to ensure that this is true.

questions: 1) Can you compare your definition of curvature to that found in differential geometry? The concept of curvature in n-dimensional spaces has been long established in the differential geometry community, and is associated with the process of Principal Curvature Decomposition. The most accessible work on the subject, in my opinion, is from John M Lee [1, 2]. Importantly, following Lee, the curvature of a Riemannian submanifold (which a NN decision boundary reasonably approximates) must be calculated by projecting the Hessian onto the function’s level-set (i.e. decision boundary) and then normalizing by the first fundamental form, or metric (see [2], chapter 8). This is very close to what you propose, but it is different in that your proposal is missing the projection operation. Moosavi-Dezfooli also mentions this in [3], section 5, although I believe they are also missing details found in the more formal treatment provided by Lee. For additional context, another similar treatment can be found in [4].  I think it would improve the quality of the submission if the authors provided some context for how their work fits into this broader mathematical description of curvature.

2) What is the relationship between decision boundary curvature and neural activation curvature? From my understanding, the prior work from Moosavi-Dezfooli attempts to minimize a network’s decision boundary curvature directly. On the other hand, your regularization on the beta term in the centered softplus activation function is intended to encourage more linear (i.e. flatter) activation functions for each neuron in all layers and the batch-norm regularization is layer specific. Following Moosavi-Dezfooli and colleagues, for some logit (i.e. penultimate layer) neurons f_i(x), f_j(x), the binary decision boundary can be written down as a b_{ij}(x) = f_i(x) - f_j(x). So one can pretty easily deduce how the curvature of f_i and f_j impacts b_{ij}. However, the relationship between neuron activation curvature and decision boundary curvature for earlier layers is not clear to me. It seems reasonable to think that individual neurons could exhibit curvature in their activation maps while still contributing to (relatively) flat decision boundaries downstream. Conversely, ReLU networks are piecewise linear, and therefore technically contribute zero curvature in the decision boundary space. Even if we squint and allow for an approximate definition of curvature (in the sense that piecewise linear functions approximate a curve in the limit), we have an example of a linear-almost-everywhere activation function producing non-flat decision boundaries. Figure 1 and table 1 both seem to indicate that the proposed regularization is indeed minimizing boundary curvature, but I am interested to hear from the authors why that is happening in their case.

3) Is it possible to do more ablation studies? My apologies if I missed this, but is there an analysis of the individual effects of the new activation function vs the batch norm modification vs the combination?

4) You may consider a more thorough robustness evaluation. As I’m sure the authors are aware, it is difficult to validate adversarial robustness claims, and many studies that initially claim robustness ultimately prove to fail under more careful scrutiny [5, 6]. While I think it is unreasonable to ask the authors to satisfy every consideration made in [5], I do think additional care should be taken (e.g. providing results for l1 bounded attacks, for gradient-free attacks, and/or auto-PGD attack). I would also make sure to include a section (appendix would be fine) that explains the thread model and what care was taken to explore the parameter space of the attacks.

5) typo on line 231: “that the function [is] locally quadratic”

[1] John M. Lee. Introduction to Smooth Manifolds. Number 218 in Graduate Texts in
Mathematics. Springer, 2 edition, 2013. ISBN 978-1-4419-9981-8 978-1-4419-9982-5.

[2] John M. Lee. Introduction to Riemannian Manifolds. Springer International Publishing, 2nd edition, 2018. ISBN 978-3-319-91754-2.

[3] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. Robustness of classifiers to universal perturbations: A geometric perspective. In International Conference on Learning Representations, ICLR, 2018.

[4] Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. Exponential expressivity in deep neural networks through transient chaos. Advances in neural information processing systems, 29:3360–3368, 2016.

[5] Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. ArXiv preprint, abs/1902.06705, 2019. URL https://arxiv.org/abs/1902.06705.

[6] Croce, Francesco, and Matthias Hein. "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks." International conference on machine learning. PMLR, 2020.

limitations: The authors provide a list of limitations in the discussion.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a set of modifications to neural networks to control their curvature. Three modifications are proposed: 1) Centered-softplus activation functions 2) Spectrally normalization of fully-connected, convolutional, batch normalization layers 3) regularization of activation and batch normalization learnable curvatures parameters. In particular, the paper proposes an alternative measure of curvature that is inversely proportional to the norm of the gradient. The paper evaluates curvature controlled models for robustness.
strengths_and_weaknesses: Strengths:
- A curvature measure coupling both the Hessian and gradient is interesting. Because of division by the gradient norm, the method might learn to automatically stay away from gradient masking and zero gradients.  Table 1 supports this by showing that the norm of both the gradient and the Hessian are smaller than other robustness methods considered.
- Centered softplus is also an interesting modification to softplus as it ensures increasing the depth does not result in activation blowup near zero.

Weaknesses:
- There have been challenges with second-order approaches to robustness such as gradient stability and gradient masking [2]. Although the ideas in this work could resolve issues in prior work, we do not see evaluations of gradient stability and gradient masking. For example, plots of the gradient norm during the training of various models on various datasets would show us if the curvature regularization is actually stable and allows us to train any model architecture. More importantly, the paper should test the model for gradient masking using for example black-box attacks. Adaptive attack should also be considered. For example, make sure there is no spectral normalization at test time by substituting the weights with their normalized copy and replace softplus with a piecewise linear approximation such as relu at test time.
- Robust evaluation does not include strong attacks such as the ones in the RobustBench [2]. The strongest attack is PGD and the number of steps is not mentioned. Also, the evaluation is limited to L2 robustness without a discussion on why the method may or may not work for Linf robustness.
- Ablation studies are required to disentangle the impact of three proposed modifications.
- Related works such as [1,3,4] are missing from comparison and discussion.

[1] Ma, A., Faghri, F., Papernot, N., & Farahmand, A. M. (2020). Soar: Second-order adversarial regularization. arXiv preprint arXiv:2004.01832.
[2] Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., ... & Hein, M. (2020). Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670.
[3] Anil, C., Lucas, J., & Grosse, R. (2019, May). Sorting out Lipschitz function approximation. In International Conference on Machine Learning (pp. 291-301). PMLR.
[4] Qin, C., Martens, J., Gowal, S., Krishnan, D., Dvijotham, K., Fawzi, A., ... & Kohli, P. (2019). Adversarial robustness through local linearization. Advances in Neural Information Processing Systems, 32.

questions: - Have you evaluated the method when combined with adversarial training?
- ​​Eq 4: Can you explain why softplus converges to x/2 when beta goes to zero?
limitations: Yes
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors propose to build a kind of low-curvature neural networks (LCNNs), which can lower network curvature for achieving robust models. The key to reducing the curvature in LCNNs is centered-softplus activation function and $\gamma$-Lipschitz BN layer, which are specifically designed by the authors with better curvature property compared to those standard layers. Meanwhile, the authors give a theoretical upper bound on the curvature of networks, and demonstrate that these layers can minimize such bound. Finally, the authors empirically show that LCNNs can give lower curvature via training Resnet18 on Cifar100, and also show that LCNNs can improve the performance further in combination with the GradReg in adversarial experiments.

strengths_and_weaknesses: &nbsp;
## **Strengths**:
1. The paper is clear and well written. 
2. The authors show some interesting insight about the curvature property of the stacked layer networks.
3. The proposed centered-softplus has some nice properties, and can be promising to be implemented in future model.
4. The LCNN models show a universal improvement in adversarial experiments.


&nbsp;
## **Weaknesses**:

*Q1*:

The authors claims that conventional measures such as Hessian norm is sensitive to gradient scaling (section 3.1). And the authors propose a kind of definition of curvature that is approximately normalized, $C_f(x) = ||\nabla_x^2 f(x)||_2 / (||\nabla_x f(x)||_2 + \epsilon)$. As far as I know, Dinh (citation 1 in the authors' paper) also gives a definition that is very similar to the authors', which is $|\nabla^2 L(\theta)||_2\epsilon^2 / 2(1 + L(\theta))$ (Left column at page 3 in their paper). Note that Dinh's form is to measure the curvature in the weight space while the authors measure in the input space, but this does not affect the measure metric. So I am wondering what is the main difference between the two formula in essence.

*Q2*:

The authors try to change the function property to reduce the network curvature, where the authors propose the centered-softplus activation function. My second concern is that to my understanding, the low curvature will normally denote a low gradient norm. The Lipchitz constant of a function equals to the maximum gradient norm for a small region. A low curvature generally means that the change in the neighborhood is relatively small, which implies a low Lipchitz constant and then a small gradient norm. So in our training, we will use the typical backpropagation to sequentially update the weight at each layer according to the gradient of pervious layers. My concern is that if a function has a low curvature, we may more likely to suffer from the gradient vanishment for these shallow layers far from the output layer. This is the reason why the relu function is mostly used for building models. Considering that the c-softplus is a low curvature function, especially around zero, I am somehow worried about the side-effect it brings in training, i.e. the gradient vanishment. Also, in todays modeling concept, the community are building very deep models. I think that the techniques proposed by the authors may not be well suited for these scalable deep models. Low curvature may decrease their expressivity, since the weights in the shallow layers are limited to some extent. The authors only use ResNet18 for demonstration, so it will be more interesting to see such layer to be applied in deeper models.

*Q3*:

My third concern is about some details in experiments. Firstly, since training LCNN will further involve two extra hyper-parameters, so I am wondering how the authors tune these parameters in practice. Is there any advice about the two parameters? Based on the experiment results, introducing two extra parameters can only improve little performance, but will severely burden the tuning. Secondly, to my understand, $\nabla^2 f$ will not be computed directly, so I am also wondering how this is computed. Thirdly, the computation cost of training with the authors' scheme is 1.6 times higher than that using the standard scheme. So in my opinion, such an additional cost is not quite proper to be called as "marginally increasing the training time". 

#### **Minor**:

It will be very interesting to see that the authors train the LCNN with SAM (Sharpness-Aware Minimization for Efficiently Improving Generalization, Pierre Foret et, ICLR2021), which is a techniques used for smoothing the loss curvature.


questions: I am willing to raise my score if the authors can address my concerns.
limitations: In my opinion, this paper do not involve related issues for its current version.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes to measure the curvature of neural networks by dividing the spectral norm of the input-output Hessian by the L2 norm of the gradient. It argues that this normalized curvature measure has the advantage of being invariant to gradient rescaling. The paper then introduces an approach for training low-curvature networks by using a modified softplus nonlinearity and Lipschitz-constrained linear and batchnorm layers and demonstrates that networks trained in this manner are more robust to adversarial attacks. It also observes that the proposed curvature measure is significantly more consistent between the train and test set than the gradient and Hessian norm.
strengths_and_weaknesses: Strengths:
* The paper is clearly structured and easy to follow.
* The (overall) approach is novel as far as I am aware (although see weaknesses on the Lipschitz batchnorm layers).
* General purpose parameterization/regularization approaches as presented here are potentially useful beyond adversarial examples.

Weaknesses:
* The proposed curvature measure is not (really) defined in zero-gradient regions and there is no discussion around this (see limitations).
* The motivation of the curvature measure is fairly hand-wavy -- do we want to learn (approximately) locally linear functions when training neural networks? This might be the case, but would need to be supported with references. Moving up section 4 (4.2 in particular) could help, but would make this more of an adversarial robustness paper (which might help clarifying the storyline, see next point).
* I find the overall storyline of the paper a bit incoherent, it spends a fair amount of space justifying the curvature measure, but is ultimately not really theoretically focused. On the other hand, the empirical evaluation is fairly limited (just one basic setting that isn't just for illustration) and does not make a particularly strong case for the proposed method. I'd suggest either expanding the theoretical discussion around the curvature measure or expanding the experimental evaluation and demonstrating the benefit of low-curvature networks in other use cases than adversarial examples or compare to SOTA attacks/defences (I don't follow the literature on adversarial robustness, but my impression is that this has been a fast-moving field over recent years, so comparing on a single attack type to effectively two baselines seems quite limited).
* If I'm not mistaken, the Lipschitz parameterization of batchnorm layer has previously been proposed in (Gouk et al., Regularisation of Neural Networks by Enforcing Lipschitz Continuity, arXiv:1804.04368, 2018).
* The proposed network parametrization appears to not work particularly well on its own and relies on the simultaneous use of gradient norm regularization (at least for the adversarial robustness use case).
questions: * Please provide some discussion around flat regions (see limitations). It may not be an issue, but the addition of the small constant suggests to me that you are aware that there might be a problem .
* Could GradReg be combined with CURE? If so, this should be included as a baseline to have a fair comparison with your LCNNs + GradReg.
* My main suggestion would be to either focus on establishing the proposed curvature measure as a generally useful tool or broaden/deepen the empirical comparison. At the moment the paper sits in between two chairs in my eyes and does not make a convincing case on either of them.

Minor:
* I find the title to be of somewhat questionable taste given that many readers (if the paper is accepted) will have been affected by the pandemic and some may have strong opinions on covid-related policies one way or another (I will caveat this with saying that I dislike click-bait titles and prefer them to be informative). I would encourage the authors to drop the "flatten the curve" bit.
* I would suggest clarifying in the abstract that you are considering input-output Hessians/gradients (when I read gradient/Hessian I think of the loss w.r.t. the parameters, which surely is just my personal bias, but that may be shared by other readers).

Note regarding the score: I would have put this as a borderline reject in past years, but feel comfortable erring on the lower side since that score seems to be discouraged this year.
limitations: The paper somewhat sweeps the (potential) issue of the curvature measure not being defined in zero-gradient regions of the input space under the rug by adding "a small constant to ensure numerical stability" to the denominator. This is a fairly nonsensical justification since for a symbolic definition numerical stability is not a concern until the measure is implemented. I would strongly suggest adding an explicit discussion rather than side-stepping the issue in this manner. Clearly, for networks with a relu nonlinearity after the input layer, such flat regions in the input space exist, although this may not be the case for the suggested centered softplus nonlinearities.
review_title: null

Meta-review: metareview: This paper studies low-curvature training of neural networks. Authors first propose a normalized curvature metric (that is invariant to the scaling of the gradient) and provide an upper bound for it in terms of the curvatures and slopes of different layers. Then they move on to the main contribution which is introducing an alternative activation and batch-normalization components which effectively limits the curvature. Finally, they show that when trained with these alternative components, one can achieve the same accuracy while having more stable gradients and more robust networks without significant training time increase. In my opinion, the fact that the robustness to input perturbations and the stable gradients come with simply substituting the previous components with the new one is the biggest advantage. On top of that, in terms of the training time, the proposed method is superior to other robust baseline approaches.

While I'm recommending acceptance, I strongly suggest authors to make the following improvements for the camera-ready to increase the impact of their work significantly:

1) More data & more models: Current experiments are very limited in terms of architectures and datasets. I suggest adding ImageNet results and two other small datasets (say SVHN and Fashion MNIST). Also, I suggest repeating the experiments on some none-ResNet architectures as well.

2) Authors have argued that the stable gradients are useful for interpretably. While this might have been established separately, it is still interesting to have at least one experiment to demonstrate this.

3) Even though this method is faster than other robust baselines, it is still 1.6x slower than standard training which is a significant limitation for adaptation of these components. I don't see an inherent reason for this slowness. Would be great if the implementation can be further optimized in terms of the running time.

4) One thing that I think is missing at the moment is a clear bottom line written somewhere regarding what we can conclude from the experiments (all this information is in the paper, but I think it would be better if it was highlighted somehow as “main conclusions” or sth of that form).

confidence: Less certain
Decision: Accept