Do Wider Neural Networks Really Help Adversarial Robustness?
Authors: Boxi Wu
Jinghui Chen
Deng Cai
Xiaofei He
Quanquan Gu
Venue: NeurIPS 2021 Poster (2021)
Keywords: Adversarial Examples
Network Width
Neural Tangent Kernel

Abstract: Adversarial training is a powerful type of defense against adversarial examples. Previous empirical results suggest that adversarial training requires wider networks for better performances. However, it remains elusive how does neural network width affect model robustness. In this paper, we carefully examine the relationship between network width and model robustness. Specifically, we show that the model robustness is closely related to the tradeoff between natural accuracy and perturbation stability, which is controlled by the robust regularization parameter λ. With the same λ, wider networks can achieve better natural accuracy but worse perturbation stability, leading to a potentially worse overall model robustness. To understand the origin of this phenomenon, we further relate the perturbation stability with the network's local Lipschitzness. By leveraging recent results on neural tangent kernels, we theoretically show that wider networks tend to have worse perturbation stability. Our analyses suggest that: 1) the common strategy of first fine-tuning λ on small networks and then directly use it for wide model training could lead to deteriorated model robustness; 2) one needs to properly enlarge λ to unleash the robustness potential of wider models fully. Finally, we propose a new Width Adjusted Regularization (WAR) method that adaptively enlarges λ on wide models and significantly saves the tuning time.

Do Wider Neural Networks Really Help Adversarial Robustness?
Boxi Wu
Jinghui Chen
State Key Lab of CAD&CG
Pennsylvania State University State College, PA 16801
Zhejiang University boxiwudzju. edu. cn
jzc5917dpsu. edu
Deng Cai
Xiaofei He
Quanquan Gu Dept. of Computer Science UCLA qguêcs. ucla edu
State Key Lab of CAD&CG
State Key Lab of CAD&CG Zhejiang University
Zhejiang University
dengcai8cad.2)) edu. on
edu. edu. on xiaofeihe@cad zju.
Abstract
Adversarial training is a powerful type of defense against adversarial examples. Previous empirical results suggest that adversarial training requires wider networks for better performances. However, it remains elusive how does neural network width affect model robustness. In this paper, we carefully examine the relationship between network width and model robustness. Specifically, we show that the model robustness is closely related to the tradeoff between natural accuracy and perturbation stability, which is controlled by the robust regularization parameter X. With the same X, wider networks can achieve better natural accuracy but worse perturbation stability, leading to a potentially worse overall model robustness. To understand the origin of this phenomenon, we further relate the perturbation stability with the network's local Lipschitzness. By leveraging recent results on neural tangent kernels, we theoretically show that wider networks tend have worse perturbation stability. Our analyses suggest that: 1) the common strategy of first fine-tuning À on small networks and then directly use for wide model training could lead to deteriorated model robustness; 2) one needs to properly enlarge À to unleash the robustness potential of wider models fully. Finally, we propose a new Width Adjusted Regularization (WAR) method that adaptively enlarges À on wide models and significantly saves the tuning time.
1 Introduction
Researchers have found that Deep Neural Networks (DNNs) suffer badly from adversarial examples [59]. By perturbing the original inputs with an intentionally computed, undetectable noise, one can deceive DNNs and even arbitrarily modify their predictions purpose. To defend against adversarial examples and further improve model robustness, various defense approaches have been proposed [48, 42, 18, 40, 67, 26, 58, 55). Among them, adversarial training [23, 41] has been shown to be the most effective type of defenses [5]. Adversarial training can seen as a form of data augmentation by first finding the adversarial examples and then training DNN models those examples. Specifically, given a DNN classifier ff parameterized by 0, a general form of adversarial training with loss function
'Equal contribution.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

L can be defined as:
N Σ N
argmin
[(0:x,,1)
max E.B....))
[2(0; X y1) [ y:]]
(1.1)
natural risk
robust regularization
where {(x,, M0)==1> are training data, B(x, c) with radius centered at x, and p 2 1, and standard empirical risk minimization, the extra robust regularization term encourages the data points within [x(x, <) to be classified as the same class, i.e., encourages the predictions to be stable. The regularization parameter À adjusts the strength of robust regularization. When À = 1, it recovers the formulation in [41], and when À = 0.5, it recovers the formulation in [23]. Furthermore, replacing the loss difference in robust regularization term with the KL-ddivergence based regularization recovers the formulation in [70].
xIIp e} denotes the 4p norm ball is the regularization parameter. Compared with
{
||
One common belief in the practice of adversar- ial training is that, compared with the standard empirical risk minimization, adversarial training requires much wider neural networks to achieve better robustness. [41] provided an intuitive planation: robust classification requires a much more complicated decision boundary, needs to handle the presence of possible adversarial examples. However, remains elusive how the network width affects model robustness. To an- swer this question, we first examine whether the larger network width contributes to both the nat- ural risk and the robust regularization term in (1.1). Interestingly, when tracing the value changes in (1.1) during adversarial training. we observe that the value of the robust regulariza-
(a) Natural Risk
(b) Robust Regularization
Figure 1: Plots of both natural risk and robust regularization in (1.1). Two 34-layer WideResNet [69] are trained by TRADES [70] on CIFARIO [37] with widen factor being 1 and 10.
tion part actually gets worse on wider models, suggesting that larger network width does not lead to better stability in predictions. In Figure I, we show the loss value comparison of two different wide models trained by TRADES [70] with achieves better natural risk but incurs a larger value on robust
as suggested in the original paper. We can see that the
wider model (i.c., WideResNet-34- regularization. This motivates us to find out the cause this phenomenon.
In this paper, we study the relationship between neural network width and model robustness for adversarially trained neural networks. Our contributions can be summarized follows:
1. We show that the model robustness is closely related to both natural accuracy and perturbation stability, a new metric we proposed to characterize the strength of robust regularization. The balance between the two is controlled by the robust regularization parameter X. With the same value of A. the natural accuracy is improved on wider models while the perturbation stability often worsens, leading to possible decrease in the overall model robustness. This suggests that proper tuning of X on wide models is necessary despite being extremely time-consuning, while directly using the fine-tuned À small networks train wider ones, as many people did in practice [41, 70], may lead to deteriorated model robustness.
2. Unlike previous understandings that there exists a trade-off between natural accuracy and robust accuracy, we show that the real trade-off should between natural accuracy and perturbation stability. And the robust accuracy is actually the consequence of this trade-off.
3. To understand the origin of the lower perturbation stability of wider networks, we further relate perturbation stability with the network's 's local Lipschitznesss. By leveraging recent results on neural tangent kernels [36, 3, 73, 8, 21], we show that with the same value of X, larger network width naturally leads worse perturbation stability, which explains our empirical findings.
Our analyses suggest that to unleash the potential of wider model architectures fully, one should mitigate the perturbation stability deterioration and enlarge robust regularization parameter À for training wider models. Empirical results verified the effectiveness of this strategy on benchmark datasets. In order to alleviate the heavy burden for tuning on wide models, we develop the

---

Width Adjusted Regularization (WAR) method to transfer the knowledge we gain from fine-tuning smaller networks into the training of wider networks and significantl save the tuning time.
Notation. For a d-dimensional vector x = [21,..., val", we use ||x||p = (Ei |my/") with p 2 1 to denote its Ep norm. 1(-) represents the indicator function and A represents the universal quantifier.
2 Related Work
Adversarial examples were first found in [59]. Since then, tremendous work have been done exploring the origins of this intriguing property of deep learning [25, 38, 20, 60, 22, 72] as well as designing more powerful attacks [23, 47, 43, 41, 9, 11] under various attack settings. [5] identified the gradient masking problem and showed that many defense methods could be broken with a few changes on the attacker. [13] proposed gradient-free black-box attacks and [32, 33, 12] further improved its efficiency. Recently, [34, 35] pointed out that adversarial examples are generated from the non-robust or invariant features hidden in the training data.
Adversarial attacks:
Defensive adversarial learning: Many defense approaches have been proposed to directly learn a robust model that can defend against adversarial attacks. [41] proposed general framework of robust training by solving a min-max optimization problem. [62] proposed a new criterion to evaluate the convergence quality quantitatively. [70] theoretically studied the trade-off between natural accuracy and robust accuracy for adversarially trained models. [63] followed this framework and further improved its robustness by differentiating correctly classified and misclassified examples. [14] solve the problem by restricting the variation of outputs with respect to the inputs. [15, 54, 39] developed provably robust adversarial learning methods that have the theoretical guarantees on robustness. Recent works in [65, 50] focus on creating adversarial robust networks with faster training protocol. Another line of works focuses on increasing the effective size of the training data, either by pre-trained learning [10, 1, 44J. Very recently, [66] proposed to conduct adversarial weight perturbation aside from input perturbation to obtain more robust models. [24] achieves further robust models by practical techniques like weight averaging.
models [30] or by semi-supervin
Robustness and generalization: Earlier works like [23] found that adversarial learning can reduce overfitting and help generalization. However, as the arms race between attackers and defenses keeps going, is observed that strong adversarial attacks can cause severe damage the model's s natural accuracy [41, 70J. Many works [70, 61, 52, 19] attempt to explain this trade-off between robustness and natural generalization, while some other works proposed different perspectives. [56] confirmed that more training data has the potential to close this gap. [6] suggested that a robust model is computationally difficult to learn and optimize. [72] showed that there still a large gap between the currently achieved model robustness and the theoretically achievable robustness limit on natural image distributions. [2] showed that the adversarial examples stem from the accumulation of small dense mixtures in the hidden weights during training and adversarial training works by removing such mixtures. Very recently, [51] showed that this trade-off stems from overparameterizatton and insufficient data in the linear regression setting. [68] proved that both accuracy and robustness are achievable through locally Lipschitz functions with separated data, and the gap between theory and practice is due to either failure to impose local Lipschitzness or insufficient generalization. [7] also studied the relationship between robustness and network size. In particular, [7] shows that overparametrization is necessary for robustness two-layer neural networks, while we show that when networks get wider, they will have worse perturbation stability, and therefore larger regularization is needed to achieve better robustness.
3 Empirical Study on Network Width and Adversarial Robustness
In this section, we empirically study the relation between network width and robustness by first taking a closer look at the robust accuracy and the associated robust examples.
3.1 Characterization of Robust Examples
Robust accuracy the standard evaluation metric of robustness, which measures the ratio of robust examples, i.c., examples that can still be correctly classified after adversarial attacks.
3

---

Previous empirical results suggest that wide models enjoy both better generalization ability and model robustness. Specifically, [41] proposed extend ResNet [28] architecture to WideResNet [69] with a widen factor 10 for adversarial training on the CIFAR 10 dataset and found that the increased model capacity significantly improves both robust accuracy and natural accuracy. Later works [70, 63]
follow this finding and report their best result using the wide networks.
However, as shown by our findings in Figure wider models actually lead worse robust regular- ization effects, suggesting that wider models are not better in all aspects, and the relation between model robustness and network width may be more intricate than what people understood previ- ously. To understand the intrinsic relationship between model robustness and network width, let us first take closer look the robust examples. Mathematically. robust examples can be defined as Swa == (x: : VX E IB [x(, e), f(0; x) = 3). Note that by definition of robust examples, we have the following equation holds:
{x VX E (xx e), ((0 = v)
f(;x)(x) = v)
A Ax V& B(x, f(0; 8(0:2))
examples.S.pe
conseity classiffed examplesssooneet
stable examples:
(3.1)
where A is the logical conjunction operator. (3.1) suggests that the robust examples are the intersection of two other sets: the correctly clas- sified examples (examples whose predictions are the correct labels) and the stable examples (ex- amples whose predictions are the same within the t, norm ball). A more direct illustration of this relationship can be found in Figure 2. While the natural accuracy measures the ratio of cor- rectly classified examples [Sooreel against the whole sample set, to our knowledge, there does not exist a metric measuring the ratio of stable examples |Stable| against whole the sample set. Here we formally define this ratio as the pertur- bation stability, which measures the fraction of examples whose predictions cannot be perturbed as reflected in the robust regularization term in (1.1).
Adversartal aa Attack
Standard Tr Tra fning
4 (085)
2
Bird SOM0OO
Adversar fal Training
Figure 2: An illustration of robust, correctly classi- fied, and stable examples in (3.1).
3.2 Evaluation of Perturbation Stability
We apply the TRADES [70] method, which one of the strongest baselines in robust training, on CIFARIO 10 dataset and plot the robust accuracy, natural accuracy, and perturbation stability against the training epochs in Figure 3. Experiments are conducted on WideResNet-344 34 [69] with various widen factors. For each network, when robust accuracy reaches the highest point, we record all three metrics and show their changing trend against network width in Figure 3(d). From Figure 3(d), we can observe that the perturbation stability decreases monotonically as the network width increases. This suggests that wider models are actually more vulnerable to adversarial perturbation. In this sense, the increased network width could hurt the overall model robustness to a certain extent. This can be seen from Figure 3(d), where the robust accuracy of widen-factor 5 is actually slightly better than that of widen-factor 10.
Aside from the relation with model width, we can also gain other insights from perturbation stability:
1. Unlike robust accuracy and natural accuracy, perturbation stability gradually gets worse during the training process. This makes sense since untrained model that always outputs the same label will have perfect stability, and the training process tends to break this perfect stability. From another perspective, the role of robust regularization in (1.1) is to encourage perturbation stability, such that the model predictions remain the same under small perturbations, which in turn improves model robustness.
2. Previous works [70, 61, 52] have argued that there exists a trade-off between natural accuracy and robust accuracy. However, from (3.1), we can see that robust accuracy and natural accuracy coupled with each other, robust example must first be correctly classified. When the

---

natural accuracy goes to zero, the robust accuracy will become zero. On the other hand, higher natural accuracy also implies that more examples will likely become robust examples. Works including [51] and [45] also challenged this robust-natural trade-off [61] does not hold for some cases. Therefore, we argue that the real trade-off' here should be between natural accuracy and perturbation stability and the robust accuracy is actually the consequence of this trade-off.
[53] has recently shown that adversarial training suffers from over-fitting as the robust accuracy might get worse as training proceeds, which can be seen in Figure 3(a). We found that the origin of this over-fitting mainly attributed to the degenerate perturbation stability (Figure 3(c)) rather than the natural risk (Figure 3(b)). Future works of adversarial training may consider evaluating our perturbation stability to understand how their method takes effects. Do they only help natural risk, or robust regularization, or maybe both of them.
VISW Pasting
fustord
Mymopply hoppp
(a) Robust Accuracy
(b) Natural Accuracy
(c) Porturbation Stability (d) Metries against Width
Figure 3: Plots of (a) robust accuracy, (b) natural accuracy, and (c) perturbation stability against training epochs for networks of different width. Results acquired CIFAR with the adversarial training method TRADES and architectures of WideResNet-334 Training schedule is the same as the original work [70]. We record all three metries when robust accuracy reaches the highest point and plot them against network width in (d).
Why Larger Network Width Leads to Worse Perturbation Stability?
Our empirical findings in Section 3 explains why the larger network width may not help model robustness as leads to worse perturbation stability. However, it still remains unclear what is the underlying reasons for the negative correlation between the perturbation stability and the model width. In this section, we show that larger network width naturally leads to worse perturbation stability from theoretical perspective. Specifically, Lipschitzness and then study the relationship between local Lipschitzness and the model width by leveraging recent studies neural tangent kernels [36, 3, 8, 73, 21].
first relate perturbation stability with the network's local
4.1 Perturbation Stability and Local Lipschitzness
Previous works [29, 64] usually relate local Lipschitzness with network robustness, suggesting that smaller local Lipschitzness leads to robust models. Here we show that local Lipshetzness more directly linked to perturbation stability, through which it further influences model robustness.
As start, let first recall the definition of Lipschitz continuity and its relation with gradient norms.
Lemma 4.1 (Lipschitz continuity and gradient norm [49]). Let E IRd denotes a convex compact set, f is a Lipschitz function if for all x D, it satisfies
lf(x') f(x) & L||x' x
where L supxe and 1/p 1/9 1.
Intuitively speaking, Lipsehitz continuity guarantees that small perturbation the input will not lead to large changes in the function output. In the adversarial training setting where the perturbation x! can only chosen within the neighborhood of x, we focus on the local Lipsehitz constant where we restrict x' E [B(x e) and L supx [I| f(x')()))) (xx'))))+
5

---

Now suppose our neural network loss function is local Lipschitz, let x' be our computed adversarial example x and x be the original example, the robust regularization term satisfies
max ScB(x.c)
((00 - L(O; x v)] - T. max *E3(x.()
[[| x||p] S cl,
(4.1)
where the first inequality is due local Lipschitz continuity and L = supx'ER(x.
vv)llo)
(4.1) shows that the local Lipschitz constant is directly related to the robust regularization term, which can be used as a surrogate loss for the perturbation stability.
4.2 Local Lipschitzness and Network Width
Now we study how the network width affects the perturbation stability via studying the local Lipschitz constant.
Recently, a line of research emerges, which tries to theoretically understand the optimization and generalization behaviors of over- over-parameterized deep neural networks through the lens of the neural tangent kernel (NTK) [36, 3, 8, 73]. By showing the equivalence between over-parameterized neural networks and NTK in the finite width setting, this type of analysis characterizes the optimization and generalization performance of deep learning by the network architecture (c.g.. network width, which we are particularly interested in). Recently, [21] also analyzed the convergence of adversarial training for over-parametente constant increases with the model width.
neural networks using NTK. Here, we will show that the local Lipschitz
In specific, let m be the network width and H be the network depth. Define an H-layer fully connected neural network as follows
f(x) a (WWH-) (.(-))))....
where ( Ra (hh) E mm h 2.... H are the weight matrices, E R" is the output layer weight vector, and (--) is the entry-wise ReLU activation function. For notational simplicity, we denote by W = (W((",,,
w the collection of weight matrices and by Wo- = (W(") ..... w the collection of initial weight matrices. Following [21], we assume the first layer and the last layer's 's weights are fixed, and W is updated via projected gradient descent with projection set (RR) following lemma upper bounding the input gradient norm.
{w- |/W(() w || S R/vm,h 1,2, H.. We have the
Lemma 4.2. For any given input x E Rd and l norm perturbation limit e, if > max(d, (2 log( H))), R/ R//m + 6 E c/ (H m)²) for some sufficient small c > 0, then with probability at least - (HH) C, the input gradient norm satisfies
/R/////////// H), we have for any x' [(xx e) and Lipschitz loss
[VC(((X) M) = O OVmH).
The proof of Lemma 4.2 can be found in the sup- plemental materials. Note that Lemma 4.2 holds for any x' E B(x, E). therefore, the maximum input gradient norm in the e-ball is also in the
Iocat Covrrse
order of O O() Lemma 4.2 suggests that
the local Lipschitz constant is closely related to the neural network width In particular, the local Lipschitz constant scales as the square
des--aactrr den-facttr 3
root of the network width. This in theory ex-
plains why wider networks are more vulnerable to adversarial perturbation.
Eseere
Figure 4: Plot of approximated local Lipschitz constant along the adversarial training trajectory. Models are trained by TRADES [70] on CIFARIO dataset using WideResNet model. Wider networks in general have larger local Lipschitz constants.
In order to further verify the above theoreti- cal result, we empirically calculate the local Lipschitz constant. In detail, for commonly used lox norm threat model, we evaluate the quantity supx {I ((0: y)llı} along adversarial training trajectory for networks with different widths. Note that solving this maxi- mization problem along the entire training trajectory is computationally expensive even intractable.

---

Table 1: The three metrics under PGD attack with different À CIFARIO dataset using WideResNet- 34 model. We test TRADES as well as our (generalized) adversarial training. Each experiment is repeated three times. The highest robustness value for each column is annotated with bold number. From the table, we can tell that: 1) The best choice of À increases as the network width increases; 2) For models with the same width, the larger À always leads higher perturbation stability; With the same X, the larger width always hurts perturbation stability, which backs up our claim in Section 4.2.
Robust Accuracy (%)
Natural Accuracy (9)
Perturbation Stability (%)
width-1
width-5
width-10
width-1
width-S
width-10
width-1
width-5
width-10 10
TRADES [70]
47.81=009
54.45+.16
54.18:139
76.26:t.10
84.442.06
84.90:.80
69.334.05
68.27:..22
67.25+..39
48.01+.06
55.34+-17
45
73.78++30
82.77+007
84.13+.25 28
71.924.33
70.66++266
69.08+-88
12 47.87=.06
55.611.04
55.98:.13
72.294.25
81.59
83.59..62
73.33:.16
72.00:1.00
20.18:.67
15 47.15=.13
55.49:.15
55.96:1.9
70.98.1.24
80.69:.08
12
73.794.27
72.87:003
70.874.23
18 47.02=113
55.43++.2
56.43--.17
70.134.06
79.97+.12
82.21+.21
74.63+t.1
73.77++13
72.04++.00
21
46,26:2.19 19
55.31+.20
56.07:.21
68.954.38
79.25
81.742.12
75.17:1.28
74.15:1.38
72.11
Adversarial Training [41]
1.00
47.99
50.87:1.2
50. 13
77.30-1.01
85.82
85.62::.1
66.482.24
62.23::.42
61.6244.6
1.25
49.24
53.10-+09
46
74.04::.47
84.73+2.2
86.25:.12
70.34
65.24::.08
62.9441.35
1,50
49.11 19.11+003
54.15+-03
53.25++55
12.1652
84.35+.19
85.50+.57
72.10+-11
66.65-+.06
64.51+-72
1.75
48.324.63 63
54.36:t.14
53.654.80
70. 70.66-4.46
83.95..30
85.52..24
72.43+.40
67.31:t03
65.6744.10
2,00
47.44 06
54.10+.15
55.78+.22
69.67++.99
83,49+.06
85.41
72.73+004
67.53++001
65.71
Therefore, we approximate this quantity by choosing the maximum input gradient e -oorm among the 10 attack steps for each iteration. Figure 4 shows that larger network width indeed leads to larger local Lipschitz constant values. This backup the theoretical results in Lemma 4.2.
5 Experiments
From Section 4, we know that wider networks have worse perturbation stability. This suggests that to fully unleash the potential of wide model architectures, we need carefully control the decreasing of the perturbation stability on wide models. One natural strategy to do this is by adopting a larger robust regularization parameter À in (I.1). In this section, we conduct thorough experiments to verify whether this strategy can mitigate the negative effects on perturbation stability and achieve better performances for wider networks.
is worth noting that due to the high computational overhead of adversarial training on wide networks, previous works [70] tuned À on smaller networks (ResNet18 [27]) and directly apply it on wider ones, neglecting the influence of model capacity. Our analysis suggests that using the same for models with different widths is suboptimal, and one should use larger À for wider models in order to get better model robustness.
5.1 Experimental Settings
We conduct our experiments on CIFARIO [37] dataset, which is the most popular dataset in the adversarial training literature. It contains images from 10 different categories, with 50k images for training and 10k for testing. Here we first conduct our experiments using the TRADES [70] method. Networks are chosen from WedeResNet [69] with different widen factor from 1, 5, 10. The batch size is set to 128, and we train each model for 100 epochs. The initial learning rate is set to be 0.1. We adopt slightly different learning rate decay schedule: instead of dividing the learning rate by 10 after 75-th epoch and 90-th epoch as in [41, 70, 63], we halve the learning rate for every epoch after the 75-th epoch, for the purpose of preventing over-fitting For evaluating the model robustness, we perform the standard PGD attack [41] using 20 steps with step size 0.007, and c = 8/255. Note that previous works [70, 63] report their results using step size 0.003, which we found is actually less effective than ours. All experiments are conducted on a single NVIDIA V100 GPU.
5.2 Model Robustness with Larger Robust Regularization Parameter
We first compare the robustness performance of models with different network width using robust regularization parameters chosen from (6, 9, 12, 15, 18, 21) for TRADES [70]. Results of different evaluation metries are presented in Table 1.
7

---

Table 2: Robust accuracy (%) for different datasets, architectures and regularization parameters under various attacks. The highest results are evaluated for three times of randomly started attack. Our approach of boosting regularization for wider models apply to all cases. The value of w and represents the network width.
widen-factor/ growth-rate
regulari- zation
Dataset
Architecture
PGD
C&W
FAB
Square
6
47.92.1.01
44.95.1.03
44.31
49.25
w
12
47.91=.04
44.244.02
43.71=.05
47.75+.02
46.92+-05
43.48+-03
43.00+-01
46.01+.05
À 6
54.50:t.33
53.14:1.3
02.13=005
56.791102
WideResNet-34
= 5
À 12
55.56=t.04
53,28++04
52,55=002
56.88+.055
À 18
55.21+.02
52.64+-02
52.18=00
56.31+.01
CIFARIO
À 6
54.234.04
54.024.03
52.68=007
57.64+.03
10
À 12
55.80=.06
54.41#.01
53.57=004
57.72+.10
À 18
56.2911.10
54,5711.22
54.06 02
58.041.05
À 6
44.79:1.02
40.83:1.03
40.07=03 03
45.661.05
k 12
À 12
44.66.1.03
40.9111.03
39.88 00
44.23:1.04
A 18
44.38.1.05
40.63.1.03
39.42 01
43.3111.44
DenseNet-BCC BC-40
6
55.511101
52.761.04
51.74=002
57.24:1.01
64
55.85.1.03
52.98:1.02
57.34:1.04 03
55.7111.03
52.83.1.06
51.66=.04
55.21
À = 6
24.28:1.02
20.24.1.01
19.97-02
22.91:102
10
À = 12
24.18:t.04
20.15:+.02
19.83+.01
22.78+.01
À 18
23.99:1.03 20.0111.2
19.01
22.04:1.01
À 6
30.73:1.03
27.25
26.01
30.111.03
CIFARTO0
WideResNet-34
== 5
À = 12
31.57+-02
27.83+-00 .02
27.08-W0
30.45+.01
À 18
31.38:t.01
27.66:t.04
26.94=.03
30.02:1.01
A 6
30.48+002
27.98+.01
27.00=.11
30.45+006
10
A = 12
31.75+-09
29.25+-0M
28.14+.03
31.23+.04
A = 18
32.98:1003
29.83+1.01
28.78-002
32.02
From Table 1, we can observe that the best robust accuracy for width-1 network is achieved when A 9, yet for width-5 network, the best robust accuracy is achieved when X = 12, and for width-10 network, the best is 18. This suggests that wider networks indeed need a larger robust regularization parameter unleash the power of wide model architecture fully. Our exploration also suggests that the optimal choice of À for width-10 network is 18 under the same setting as [70], which is three times larger than the one used in the original paper, leading to average improvement of 2.25% on robust accuracy. It is also worth noting that enlarging À indeed leads to improved perturbation stability. Under the same X, wider networks have worse perturbation stability. This observation is rather consistent with our empirical and theoretical findings in Sections 3 and 4. As stated in Section 3.2, the real trade-off is between natural accuracy and perturbation stability rather than robust accuracy. Also, the stability provides clear hint for finding the best choice of X.
We further show that strategy also applies to the original adversarial training [41], shown by the bottom part of Table 1. Proper adaptations should be made to boost the robust regularization for original (generalized) adversarial training. We show the detail of the adaptations in the Appendix. As shown by the table, the large improvements on both TRADES and adversarial training using our boosting strategy suggest that adopting larger À is crucial in unleashing the full potential of wide models, which is usually neglected in practice.
5.3 Experiments on Different Datasets and Architectures
To show that our theory is universal and is applicable to various datasets and architectures, we conduct extra experiments on the CIFAR 100 dataset and DenseNet model [31]. For the DenseNet models, the growth rate & denotes how fast the number of channels grows and thus becomes suitable measure of network width. Following the original paper [31], we choose DenseNet-BC-40 and use models with different growth rates verify our theory.
Experimental results are shown Table 2. For completeness, we also report the results under four different attack methods and settings, including PGD [41], C&W [9], FAB [16], and Square [4]. We

---

Table 3: Comparison of TRADES with different
Algorithm 1 Width Adjusted Regularization
tuning strategies. N/A denotes no fine-tuning of the current model (tuning on small networks only).
I: Input: initial weights 9. WAR parameter Ç. learn-
ing rate 1), adversarial attack A
Manual represents exhaustive fine-tuning,
2: Ào 0, a = 0.1
Model
Tuning
A
PGD
GPU hours
3: for 1,...,, T do
4: Get mini-hatch {(x1, y1 un)...., (xm; yom)
WRN-34-1
N/A
6.00
47.81
12+18=30
5: for = L...., m (in parallel) do
Manual
9.00
48.01
12x6=72
WAR
9.12
48.06
12+ 18=30
6:
&; <- A(x:)
7:
lux L(0;;xi, 81)
N/A
6.00
54,45
20+18=38
8:
Lod + L(O,; x,. V.) L(0:;x1.31
WRN-34-5
Manual
12.00
55.61
60x6=120
9:
d + max(>:-1 + a: (6 Unna/mb)) ) 0)
WAR
14.37
55.62
20+18=38
10:
0. + 01-1-(1/m) Di=1 Vollme + llot))
N/A 6.00
54.18
32+18=50
11:
end for
WRN-34-10
Manual
18.00
56.43
32x6=192
12:
end for
WAR
16.43
56.46
32+18-50
adopt the best X from Table and show the corresponding performance on models with different widths. It can be seen that our strategy of using a larger robust regularization parameter works very well across different datasets and networks. On the WideResNet model, we observe clear patterns as in Section 5.2. On the DenseNet model, although the best regularization À is different from that of WideResNet, wider models, in general, still require larger À for better robustness. On CIFAR100, our strategy raises the standard PGD score of the widest model from 30.48% to 32.98%.
5.4 Width Adjusted Regularization
Our previous analysis has shown that larger model width may hurt adversarial robustness without properly choosing the regularization parameter X. However, exhaustively cross-vacs coossvalldatingg À on wider networks can be extremely time-consuming in practice. To address this issue, we investigate the possibility of automatically adjusting À according to the model width, based on our existing knowledge obtained in fine-tuning smaller networks, which is much cheaper. Note that the key to achieving the best robustness is to well balance between the natural risk term and the robust regularization term in I). Although the regularization parameter À cannot be directly applied from thinner networks wider networks (as suggested by our analyses), the best ratio between the natural risk and the robust regularization across different width models can be kept roughly the same. Following this idea, we design the Width Adjusted Regularization (WAR) method, which is summarized in Algorithm 1. Specifically, we first manually tune the best À for thin network and record the ratio Ç between the natural risk and the robust regularization when the training converges. Then, on training wider networks, we adaptively? adjust À to encourage the ratio between the natural risk and the robust regularization to stay close to Ç. Let's take an example here. We first cross-validar A on a thin network with widen factor 0.5 and identify the best À = 6 and Ç = 30 with 18 GPU hours in total. Now we compare three different strategies for training wider models and summarize the results in Table 3: 1) directly apply À 6 with no fine-tuning on the current model; 2) exhaustive manual one-tuning from = 6.0 to 21.0 (6 trials) as in Table 1; 3) our WAR strategy. Table 3 shows that the final À generated by WAR on wider models are consistent with the exhaustively tuned best A. Compared to the exhaustive manual tuning strategy, WAR achieves even slightly better model robustness with much less overall training time (~44 times speedup for WRN-34-10 10 model). On the other hand, directly using À = with no tuning on the wide models leads to much worse model robustness while having the same overall training time. This verifies the effectiveness of our proposed WAR method.
5.5 Comparison of Robustness on Wide Models
Previous experiments in Section 5.2 and Section 5.3 have shown the effectiveness of our proposed strategy on using larger robust regularization parameter for wider models. In order to ensure that this strategy does not lead to any obfuscated gradient problem [5] and gives false sense of robustness, we further conduct experiments using stronger attacks. In particular, we choose to evaluate our best models on the Autok algorithm [17], which is an ensemble attack method that contains four different white-box and black-box attacks for the best attack performances.
the learning rate for A in Algorithm I is not sensitive and needs no extra tuning.

---

We evaluate models trained with WAR, with or without extra unlabeled data [10], and report the robust accuracy in Table Note that the results of other baselines are directly obtained from the AutoAttack leaderboard". From Table 4, we can see that our WAR significantly improves the baseline TRADES models on WeteResNet. This experiment further verifies the effectiveness of our proposed strategy.
6 Conclusions
In this paper, we studied the relation between net- work width and adversarial robustness in adversarial training. a principled approach to train robust neural networks. We showed that the model robustness is closely related to both natural accuracy and pertur- bation stability, while the balance between the two controlled by the robust regularization parameter A. With the same value of X. the natural accuracy is better on wider models while the perturbation sta- bility actually becomes worse, leading possible decrease in the overall model robustness. We showed the origin of this problem by relating perturbation stability with local Lipschitzness and leveraging re- cent studies on the neural tangent kernel to prove that larger network width leads to worse perturbation stability. Our analyses suggest that: 1) proper tuning of À on wider models is necessary despite being extremely time-consumings 2) practitioners should adopt larger À for training wider networks. Finally, we propose the Width Adjusted Regularization, which significantly saves the tuning time for robust training on wide models.
Table 4: Robust accuracy (%) comparison on CIFARIO under AutoAttack. i indicates training with extra unlabeled data.
Methods
Model
AutoAttack
TRADES [70]
WRN-34-10
53.08
Early-Stop [53]
WRN-344-0
53.42
FAT [71]
WRN-34-10
53.51
HE [46]
WRN-34-20
53.74
WAR
WRN-34-110
54.73
MART [63]]
WRN-28-10
$6.29
HYDRA [57]]
WRN-28-10
$7.14
RST [10]f
WRN-28-10
59.53
WAR
WRN-28-10
60.02
WART
WRN-28-20
61.84
7 Acknowledgments
We thank the anonymous reviewers for their helpful comments. Boxi Wu, Deng Cai and Xi- aofei He are supported in part by The National Key Research and Development Program of China (Grant Nos: 2018AAA0101 400), The National Nature Science Foundation of China (Grant Nos: 62036009, U1909203, 61936006), and Innovation Capability Support Program of Shaanxi (Program No. 2021TD-05).
References
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In NeurlPS, 2019.
[2] Zeyuan Allen-Zhu and Yuanzhi Li. Feature purification: How adversarial training performs robust deep learning. arXiv preprint wrXiv:2005.10190 2020.
[3] Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over- parameterization. In International Conference Machine Learning, pages 242-252. PMI.R, 2019.
[4] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient box adversarial attack via random search. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-M Minchae Frahm, editors, ECCV, volume 12368 of Lecture Notes in Computer Science, pages 484 501. Springer, 2020.
[5] Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give false sense of security: Cireumventing defenses to adversarial examples. In ICML, volume 80 of Proceedings of Machine Learning Research, pages 274-28- 283,, PMLR, 2018.
[6] Sébastien Bubeck, Yin Tat Lee, Eric Price. and Ilya P. Razenshteyn. Adversarial examples from computa- tional constraints. In ICML, volume 97 of Proceedings Machine Learning Rexearch, pages 831-840. PMLR, 2019.
hhttps://githan http:///gtttuu...m/aa.//..// attack
10

---

[7] Sébastien Bubeck, Yuanzhi Li, and Dheeraj Nagaraj. A law of robustness for two-layers neural networks. arXiv preprint urXiv:2009.14444 2020.
[8] Yuan Cao and Quanquan Gu. Generalization bounds of stochastic gradient descent for wide and deep neural networks. In Advances in Neural Information Processing Systems, pages 10836-10844 2019.
[9] Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In SP, pages 39-57. IEEE Computer Society, 2017.
[10] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C. Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In NeurIPS, 2019.
[1I] Jinghui Chen and Quanquan Gu. Rays: A ray searching method for hard-label adversarial attack. In Proceedings of the 26rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2020.
[12] Jinghui Chen, Dongruo Zhou, Jinfeng Yi, and Quanquan Gu. A frank-wolfe framework for efficient and effective adversarial attacks. In AAAI, 2020.
[13] Pin- Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. 700: zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In AlSec @ @CS, pages 15-26. ACM, 2017.
[14] Moustapha Cissé, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In ICML, volume 70 of Proceedings of Machine Learning Research, pages 854-863. PMLR, 2017.
[15] Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. In JCML, volume 97 of Proceedings of Machine Learning Research, pages 1310-1320. PMLR, 2019.
[16] Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 2199-2205 PMLR, 2020.
[17] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. /CML, 2020.
[18] Guneet S Dhillon, Kamyar Azizzadcnesheli, Zachary C. Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial defense. ICLR, 2018.
[19] Edgar Dohrihan, Hamed Hassani, David Hong, and Alexander Robey. Provable tradeoffs in adversarially robust classification. arXiv preprint W0X1:2000005166, 2020.
[20] Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Analysis classifiers' robustness adversarial perturbations. Mach. Learn., 107(3):48-550,, 2018.
[21] Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsich, Liwei Wang, and Jason D Lec. Convergence of adversarial training in overparametrized neural networks. In Advances in Neural Information Processing Systems, pages 13029-13040, 2019.
[22] Justin Gilmer, Luke Metz, Fartash Faghri, Sam Schoenholz, Maithra Raghu, Martin Wattenberg, and Tan Goodfellow. Adversarial spheres, 2018
[23] lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun, editors, ICLR, 2015.
[24] Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.0359. 2020.
[25] Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. In Yoshua Bengio and Yann LeCun, editors, JCLR, 2015.
[26] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations. ICLR, 2018
[27] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.

---

[28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016.
[29] Matthias Hein and Maksym Andriushchenko. Formal guarantees the robustness of a classifier against adversarial manipulation. In Advances in Neural Information Processing Systems, pages 2266-2276, 2017.
[30] Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In ICMI, volume 97 of Proceedings of Machine Learning Research, pages 2712-2721. PMLR, 2019.
[31] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[32] Andrew Tlyas, Logan Engstrom, Anish Athalye, Jessy Lin, Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Black-box adversarial attacks with limited queries and information. In Proceedings of the 35th International Conference on Machine Learning. 2018.
[33] Andrew Iyyas, Logan Engstrom, and Aleksander Madry, Prior convictions: Black-box adversarial attacks with bandits and priors. ICLR, 2019.
[34] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry, Adversarial examples are not bugs, they are features. In NeurlPS, 2019.
[35] Jörn-Henrik Jacobsen, Jens Behrmann, Richard S. Zemel, and Matthias Bethge. Excessive invariance causes adversarial vulnerability. In ICLR. OpenReview.net, 2019.
[36] Arthur Jacot, Clément Hongler, and Franck Gabriel. Neural tangent kemel: Convergence and generalization in neural networks. NeurlPS, 2018
[37] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. 2009.
[38] Alexey Kurakin, lan Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In /CLR. OpenReview.net, 2017.
[39] Mathias Lécuyer. Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In SP, pages 656-672. IEEE, 2019.
[40] Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In CVPR, pages 1778-1787, 2018
[41] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR. OpenReview.net, 2018.
[42] Dongyu Meng and Hao Chen. Magnet: a two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 135-147, 2017.
[43] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method fool deep neural networks. CVPR, pages 2574-2582. IEEE Computer Society, 2016.
[44] Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyato. Robustness to adversarial perturba- tions in learning from incomplete data. In NeurlPS, 2019.
[45] Preetum Nakkiran. Adversarial robustness may be at odds with simplicity. CoRR, abs//901 000532,,, 2019.
[46] Tianyu Pang, Xiao Yang. Yinpeng Dong, Kun Xu, Hang Su, and Jun Zhu. Boosting adversarial training with hypersphere embedding. CoRR, abs/2002.08619, 2020.
[47] Nicolas Paperot, Patrick D. McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In EuroS&P, pages 372-387, IEEE, 2016.
[48] Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In SP, pages 582-597. IEEE Computer Society, 2016.
[49] Remigijus Paulavičius and Julius Žilinskas. Analysis of different norms and corresponding lipschitz constants for global optimization. Technological and Economic Development of Economy, 12(4):301-33) 3066 2006.
12

---

[50] Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. In NeurIPS, 2019.
[51] Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. arXiv preprint arXiv:2002.10711 2020.
[52] Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C. Duchi, and Percy Liang. Adversarial training can hurt generalization. CoRR, 0tss/1906.00033 2019.
[53] Leslie Rice, Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. CoRR, 1556200 2020.
[54] Hadi Salman. Jerry Li, Ilya P. Razenshteyn, Pengchuan Zhang. Huan Zhang, Sébastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. In NeurlPS, 2019.
[55] Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan: Protecting classifiers against adversarial attacks using generative models. ICLR, 2018.
[56] Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras. Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data, In NewrIPS, 2018.
[57] Vikash Schwag. Shiqi Wang, Prateek Mittal, and Suman Jana. On pruning adversarially robust neural networks. CoRR, abs/2002. 10509, 2020.
[58] Yang Song, Tacsup Kim, Schastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. ICLR, 2018.
[59] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun, editors, ICLR, 2014.
[60] Florian Tramèr, Nicolas Papernot, Ian J. Goodfellow, Dan Boneh, and Patrick D. McDaniel. The space of transferable adversarial examples. CoRR,
2017.
[61] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry, Robust- ness may be at odds with accuracy. In ICLR. OpenReview.net, 2019. 2019.
[62] Yisen Wang. Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. On the convergence and robustness of adversarial training. In ICML, pages 1586-6595, 2019.
[63] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR. OpenReview.net, 2020.
[64] Tsui- Wei Weng, Huan Zhang, Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. International Conference on Learning Representations, 2018.
[65] Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. CoRR, abs/2001.0300 2020.
[66] Dongxian Wa, Shu-Tao Xia, and Yisen Wang, Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 33, 2020.
[67] Cihang Xie, Jianyu Wang. Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. ICIR, 2018.
[68] Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ Salakhutdinov, and Kamalika Chaudhuri. closer look accuracy vs. robustness. Advances in Neural Information Processing Systems, 33, 2020.
[69] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC. BMVA Press, 2016.
[70] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing. Laurent Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In JCML, volume 97 of Proceedings of Machine Learning Research, pages 7472-7482. PMLR, 2019.
[71] Jingteng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan S. Kankanhalli. Attacks which do not kill training make adversarial learning stronger. CoRR, 112/2 2020.
13

---

[72] Xiao Zhang, Jinghui Chen, Quanquan Gu, and David Evans. Understanding the intrinsic robustness of image distributions using conditional generative models. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statisticx, 2020.
[73] Difan Zou, Yuan Cao, Dongruo Zhou, and Quanquan Gu. Gradient descent optimizes wverparamettrizee deep relu networks. Machine Learning. 109(3):467 2020.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes] Section
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A] This work aims
making machine learning models more robust and trustworthy.
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes]
2. If you are including thcoretical results...
(a) Did you state the full of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect the random seed after running experiments multiple times)? [Yes]
(d) Did you include total amount of compute the type resources used (e.g.. type GPUs, internal cluster, or cloud provider)? [Yes]
4. If you using existing assets (c.g., code, data, models) curating/relcasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material URL? [Yes]
(d) Did you discuss whether and how consent obtained from people whose data you're us-
ing/curating? [Yes]
(e) Did you discuss whether the data you using/curating contains personally identifiable informa- tion or offensive content? [Yes]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "vilRlhisyc1", "reviewer": "Reviewer_ej3Q", "summary": "In this paper, the authors evaluate the impact of network width on the robustness of models trained through adversarial training. The authors (1) formulates robust accuracy as the intersection of natural accuracy and perturbation stability and show that there is tradeoff between them (2) theoretically show that increasing network width leads to higher local Lipschitz constant which leads to lower perturbation stability, (3) empirically show that networks with larger width need a larger robust regularization hyperparameter to perform well and propose a method (WAR) to tune this value.\n\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: To the best of my knowledge, the contributions of this submission are original. Prior works have indicated that adversarial training performs better with models of larger capacity, but the authors explore the impact of increasing the model width and empirically and theoretically show that this is not necessarily the case.  The authors do a good job of citing related works.\n\nQuality: The quality of the submission is good.  The authors provide theory as well as rigorous experiments to show that higher model width leads to lower perturbation stability and the need for a higher robust regularization value.  The scope of these experiments is good, the authors experiment with multiple architectures (WideResNet, DenseNet-BC) as well as different adversarial training techniques (TRADES, Adversarial Training), datasets (CIFAR-10, CIFAR-100, Imagenet), and different attack methods.  For the section where the authors introduce WAR, I can only see experiments on WideResNet and what I think is the CIFAR-10 dataset.  I think the authors can benefit by also extending these experiments to different architectures and datasets as well.\n\nClarity: Overall the paper is clear, but the paper can be proofread for grammar.  Some comments:\n- try to use in text citations (\\citet{}) when using the citation as a subject or object in a sentence\n- I think the first paragraph under section 3.1 can be placed with related works\n- line 163: \"unlearned model\" --> \"untrained model\"\n- last sentence of 1: \"... such that the learned models cannot be easily perturbed for the sake of model robustness\" --> this wording is awkward.  I think what is meant is that the models aren't fooled by small perturbations which in turn improves model robustness\n- 171-173: \"Works including [49] and [43] also challenged this robust-natural trade-off [59] does not hold for some cases\" --> awkward wording and \"does not hold for some cases\" is very vague and can be elaborated on\n- 173-174: \"Therefore, we argue ... of this trade-off\"-->  these 2 sentences should be 1 sentence joined by a comma\n- 179-180: \"Do they only help natural risk, to robust regularization, or maybe both of them.\" --> should be combined with the previous sentence and should be a question\n- line 182: \"explains why the\" --> \"illustrates how\"\n- line 183: \"it still remains unclear what is the underlying reasons ...\" --> \"it still remains unclear what the underlying reasons are ...\"\n\nSignificance: The results lead to a better understanding of how width impacts the robustness of neural networks.  The authors also propose WAR to finetune the robust regularization hyperparameter for models of larger width which maintains the same tuning time that is used in practice while improving robustness for these larger models.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I do not see any discussion of limitations.  A potential concern is how well WAR performs on other architectures/datasets.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "pGV-2YMNSMW", "reviewer": "Reviewer_TGwT", "summary": "The authors investigate the relation between network width and adversarial robustness. There are some attempts to connect to some theory, but the results are largely empirical. \n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The main problem I find with the current paper is that the main message is not clear. There are some empirical comparisons based on the width of neural networks in relation to adversarial robustness, and some theorems based on previous results on Lipschitz bounds on neural networks. But the main novelty or contribution of the paper is not clear. \n\n- The only main message I could find is that increasing the width does not necessarily add to the adversarial robustness, contrary to the claims of some of the previous works. But this alone is not a very significant contribution. \n\n- There is a problem with analyzing the relation between natural accuracy and robust accuracy using equation 3.1. Equation 3.1 is based on accuracy, whereas in practice we always optimize a surrogate loss function such as the cross-entropy loss. The tradeoff in the optimization is not the direct tradeoff of accuracies, but tradeoff of their respective loss functions. Given the cross-entropy loss can be unbounded, the actual tradeoff might not be similar to equation 3.1. \n\n- The analysis of perturbation stability using Lipschitz constant in section 4 is not particularly new. And apart from stating the bounds, there is little insight on how the bounds affect the tradeoffs and relate to the empirical results. \n \n- The rest of the paper is on empirical comparison of width vs adversarial robustness. But without better connections to other parts of the paper they seem like standalone empirical results. \n\n- I believe the authors need to improve on the clarity of presentation of their paper, and state more clearly what their actual contributions are. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "VYua3JWLo8", "reviewer": "Reviewer_7XmE", "summary": "The paper studies the connection between adversarial robustness and model capacity in the form of width. The key idea is to decompose robust accuracy into standard accuracy and stability---i.e., on what fraction of test inputs the prediction of the model cannot be changed by an adversary.\n\nThe authors find that while increased width tends to improve robust accuracy (as has also been previously observed) the gain are mostly coming from an improvement in the standard (test) accuracy of the model, while the stability of wider models is actually _decreased._ To counteract this phenomenon, they suggest increasing the regularization term corresponding to stability as the width of the model increases and indeed observe empirical performance improvements. The authors further support their hypothesis by theoretically analyzing the effect of width on robustness in an NTK setting and observing similar results. Finally, they propose a heuristic scheme for adjusting the stability regularization parameter during training and demonstrate promising results.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this paper makes a significant contribution on an important topic---we still do not fully understand the connection between capacity and robustness. The perspective proposed is simple and intuitive but at the same time appears to be quite useful for reasoning about the empirical behavior of wide models. I believe that these results will be of interest to anyone who cares about robustness.\n\nQuestions and comments to authors (not affecting score):\n- Figure 1 and 3: What is epsilon/norm used for training and eval?\n- Did you perform additional experiments with different norms and values of epsilon? Were the results consistent?\n- L167 (and similar discussion in other places): In all three works mentioned [68,59,50], the underlying trade-off actually manifests as a stability-accuracy trade-off. Specifically, in [68, 59] the trade-off manifests because robust models cannot rely on predictive but unstable features, while in [50] the trade-off is a precisely a result of the requirement that the model's prediction needs to be stable around training points. Thus, while this discussion appears to be contradicting these prior works, it actually doesn't.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, they do.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Bcx7B2zMANg", "reviewer": "Reviewer_vSzi", "summary": "The paper discusses how the width of ResNets related to their robustness. They do adversarial training with a hyper-parameter for the strength of the robust regularization and show that making the model wider while keeping the hyper-parameter constant will increase clean accuracy but reduce the robust generalization. \n\nThey define a new metric, perturbation stability, which is the rate of samples keeping their label prediction under adversarial attacks, even if the label prediction is wrong without pertubations. The authors connect this metric to the local Lipschitzness and use it to study how the width of a network affects it. They show that both through an analytical calculation and empirical results that the local Lipschitz constant increases with the width of a network.\n\nIn further experiments the authors show that wider networks need a higher weight on the robust regularization loss to get optimal robust accuracy, while previous work often optimized this weight for smaller networks and then used the same value after increasing the width. The results are verified for other datasets and architectures and finally the authors present a strategy to adaptively choose the hyper-parameter for wider models by first optimizing it on smaller models and then optimizing during training to achieve the same ratio between natural loss and robust regularization loss for the wider model that was optimal for the smaller model.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think the paper is well written and gives a new perspective on adversarial training of wide networks. The authors do a good job of exploring the topic from different directions:\n- Given a motivation why they suspect better tuned $\\lambda$ on wider models would help improve robustness\n- Giving analytical insight through the Lipschitz constants.\n- Showing the improvement in robust accuracy through empirical results.\n- Proposing a method to automatically tune $\\lambda$ for wider models while training.\n\nBecause of this I think this paper should be accepted to Neurips.\n\nSome things that could be improved:\n- In line 64 and at later points the authors say that increasing the width of models while keeping $\\lambda$ constant would decrease the overall robustness. I find the a bit inaccurate as robustness is usually looked at in terms of the robust accuracy and it (as the authors show in figure 3) usually increase with the width. I think the authors should clarify that they mean the perturbation stability. \n- The authors in their analysis in section 4 show that $\\lambda$ should grow with $\\sqrt{mH}$ but don't use that later on. Could this be an easier way to optimize $\\lambda$?\n\nMinor comments:\n- Line 3 \"it remains elusive how does neural network width affect model\" should be \"it remains elusive how neural network width affects model\"\n- Line 45 same as above\n- The notation in line 81-82 seems unnecessary. The defined notation is very standard and other similarly standard notation isn't explained.\n- Line 84 \"tremendous work\" instead of \"tremendous works\"\n- Line 216 \"increases with\" instead of \"increases as\"\n- Line 264 \"90-th epoch as in\" instead of \"90-th epoch in\"", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addressed limitations and societal impact adequately.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers had raised a few concerns which were mostly resolved after the authors' response. All the reviewer agreed in the discussions that the contributions of the paper are important and interesting. Also, the reviews contain all the points mentioned in the discussions, so there is nothing more to add here. 
Decision: Accept (Poster)