Robust Binary Models by Pruning Randomly-initialized Networks
Authors: Chen Liu
Ziqi Zhao
Sabine Süsstrunk
Mathieu Salzmann
Venue: NeurIPS 2022 Accept (2022)
Keywords: Adversarial Robustness
Model Compression

Abstract: Robustness to adversarial attacks was shown to require a larger model capacity, and thus a larger memory footprint. In this paper, we introduce an approach to obtain robust yet compact models by pruning randomly-initialized binary networks. Unlike adversarial training, which learns the model parameters, we initialize the model parameters as either +1 or −1, keep them fixed, and find a subnetwork structure that is robust to attacks. Our method confirms the Strong Lottery Ticket Hypothesis in the presence of adversarial attacks, and extends this to binary networks. Furthermore, it yields more compact networks with competitive performance than existing works by 1) adaptively pruning different network layers; 2) exploiting an effective binary initialization scheme; 3) incorporating a last batch normalization layer to improve training stability. Our experiments demonstrate that our approach not only always outperforms the state-of-the-art robust binary networks, but also can achieve accuracy better than full-precision ones on some datasets. Finally, we show the structured patterns of our pruned binary networks.


Robust Binary Models by Pruning Randomly-initializer Networks
Chen Liu' Ziqi Zhao" Sabine Süsstrunk Mathieu Salzmann École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, "chen.liu, ziqi zhao, sabine. susstrunk, mathieu. nathieu.salzzann) www.mannlleaaaeemmmaaaaa @epf1. ch
Abstract
Robustness to adversarial attacks was shown to require a larger model capacity, and thus larger memory footprint. In this paper, we introduce approach obtain ro- bust yet compact models by pruning randomly-initializz adversarial training. which learns the model parameters, we initialize the model parameters either +1 or -1, keep them fixed, and find a subnetwork structure that is robust to attacks. Our method confirms the Strong Lottery Ticket Hypoth- evis in the presence of adversarial attacks, and extends this to binary networks. Furthermore, yields more compact networks with competitive performance than existing works by 1) adaptively pruning different network layers; 2) exploiting an effective binary initialization scheme; 3) incorporating a last batch normalization layer improve training stability. Our experiments demonstrate that our approach not only always outperforms the state-of-the- robust binary networks, but also achieve accuracy better than full pelecrrcniionn ones some datasets. Finally, we show the structured patterns of our pruned binary networks.
binary networks. Unlike
Introduction
Deep neural networks have achieved unprecedented success in machine learning 19 ever, their waae-of-the-an performance comes with costs. First, modern deep neural networks usually have millions of parameters, making them difficult to deploy on devices with limited memory or computational power. Second, these models are vulnerable to adversarial attacks: imperceptible perturbations of the input can dramatically change their output and lead to incorrect predictions (55]
30 56
How-
Furthermore, jointly addressing both issues is complicated by the fact that, as shown in 4 achieving robustness against adversarial attacks typically requires higher network capacity.
In this paper, introduce an approach obtaining compact and robust binary neural networks. Our method follows a fundamentally different philosophy from typical adversarial training 44]; instead of using adversarial examples train the model parameters, we fix the model parameters and search for robust network structure. To this end, and to simultancously achieve compactness, we prune vundomly-inttialin binary networks. The resulting sparse and binary networks have much smaller memory footprint than the dense or full-precision ones. They are inherently lightweight and robust.
Our work is motivated by the Strong Lottery Ticket Hypothesis 51 which observed that within random overparameterized network, there exists subnetwork achieving performance similar to that of trained networks with the same number of parameters. The Robust Scratch Ticket [2]] extends this hypothesis to the context of adversarial robustness. Here, we introduce novel pruning strategy that yields higher compression rates, and investigate the case of binary model parameters. Specifically, we develop an adaptive pruming strategy to adaptively use different pruning rates for different layers. Furthermore, we introduce carmalizaiion-aasee technique increase the algorithm's 's stability for
indicates equal contributions
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

binary networks. The subnetworks obtained by our method are consequently more compact than full-precision one, while achieving a similar robustness attacks.
We conduct extensive experiments on standard henchmarks to confirm the effectiveness of our method.
We obtain both better performance and more stable training behavior than existing works [22) Furthermore, our approach outperforms the state-of-the- robust binary networks [25]] achieving performance on par with or even better than the state-of-the-ar robust full-precision ones while producing much more compact networks.
Finally, we conduct preliminary investigations the structure of the robust subnetworks obtained by our algorithm. We find our methods prefer to prune the whole channel or the kernel in the convolutional layers. In addition, for two consecutive convolutional layers, the kernels pruned in the first layer are well aligned with the channels pruned in the second layer. Altogether, our work sheds some light on understanding the structure of robust networks of high parameter sparsity, it also indicates the potential of regular pruning.
Our code is available at http:////tbbbooool////////oobaa/
Notation. We use light letters, lowercase bold letters, and uppercase bold letters to represent scalars, vectors, and higher dimensional tensors, respectively. @ is the elementwise multiplication operation. We use the term adversarial budget to represent the range of allowable perturbations. Specifically, the adversarial budget S. is based on the loo norm and defined as { oc e}, with the strength of the adversarial budget. We refer the proportion of pruned parameters over the total number of parameters in layer model as the pruning rate r.
2 Related Work
Adversarial Robustness. Deep neural networks have been shown to be vulnerable to adversarial attacks [55] 45
To generate adversarial examples, the Fast Gradient Sign Method (FGSM) (24 perturbs the input in the direction of the input gradient. The Iterative Fast Gradient Sign Method (IFGSM) |س improves FGSM by running it iteratively. Projected Gradient Descent (PGD) 44 uses random initialization and multiple restarts on top of IFGSM to further strengthen the attack. Recently AutoAttack (AA) I5 has led state-of-the-ard attacks by ensembling different types of attacks; is used to reliably benchmark the robustness of models 13 and we thus use it in our experiments.
Many works have proposed defense mechanisms against these adversarial attacks. Early ones 5 47 59 used obfuscated gradients [ [5] and thus were ineffective against adaptive attacks. As a
[3] 64] have in practice become the mainstream approach to obtain robust models. Specifically, given a dataset {(x, y:) M model parameterized by and loss function L, adversarial training solves the min-max optimization problem:
consequence, adversarial training [4]] and its variants [1
6 32
5
37
58
N min Σ max L(f(w, + As). U:) A,ES,
N
In practice, this is achieved by first generating adversarial examples x; + A, usually by PGD, and then using these examples to train the model parameters.
While effective, adversarial training was shown to require larger model capacity 44 60
Specifi- cally, as the model capacity decreases, adversarial training first fails to converge while the training on clean inputs still yields non-trivial performance. Conversely, as the model capacity increases, the performance of training clean inputs saturates before that of adversarial training. This highlights the challenge of finding robust yet compact models. Here, we introduce a solution to this problem.
Model Compression. There are many ways to compress deep neural networks to achieve lower memory consumption and faster inference, including pruning, quantization, and parameter encoding. The pioneering works used rnformation-hhhorree compress models by removing unimportant weights. The seminal work 28 proposed to prune the parameters with the smallest absolute values for deep networks. This motivated many follow-up works, performing either irregular pruning 26 5l (55), which removes individual parameters, or
methods (39 or second-order derivatives [ to
regular pruning [31 42 which aims to discard entire convolutional kernels. In contrast to pruning, seeks to reduce the memory consumption and inference time by using low low-precision parameters. An extreme case of quantization is binarization, which can take the
quantization 67

---

form of only binarizing the parameters [10 I1 or binarizing both parameters and intermediate
activations [33. The models can be further compressed by combining pruning with quantization and [7
Huffman coding
Recently, some efforts have been made to incorporate adversarial training into model compression.
40 50]
suggest quantization as a defense against adversarial attacks. [46] uses Bayesian connec-
tivity sampling prune the network while preserving its robustness. B dynamically generates
robust subnetwork during adversarial training. [62] uses the alternating direction method of multipli-
[ extends this framework to include other model compression techniques, such quantization. Furthermore, [54] introduces the HYDRA framework, which improves the performance of compressed robust models by a three-phase method: Pretraining, score-based pruning, and fine-tuning. Here, we follow a different strategy: Instead of performing adversarial training, we search for a robust binary subnetwork in a one. We show that our approach outperforms those based on adversarial training.
ers (ADMM) to alternatively conduct adversarial training and network pruning.
Lottery Ticket Hypothesis. This hypothesis, introduced in 221, states that overparameterized neural networks contain sparse subnetworks that can be trained in isolation to achieve competitive performance. These subnetworks are called the winning tickets. Based this interesting observation, further proposed the Strong Lottery Ticket Hypothesis. They showed that there exist winning tickets with competitive performance even without training. Furthermore, p proposed an iterative randomization scheme to reduce the size of the network in which one searches for the winning tickets. 1 introduced the Multi-Priie (Pizee Lottery Ticket Hypothesis to learn compact yet accurate binary networks by pruning and quantizing randomly weighted DNNS [17 showed that "lottery-ticker style" approaches can also improve robustness against corruption in the frequency domain.
68
The recent work of combines robustness with the Strong Lottery Ticket Hypothesis and demon- strates the existence of robust sub-networks within a random network. Here, focus on lighter- weight binary networks, and introduce an adaptive pruning strategy and last batch normalization layer achieve higher pruning rates than 2 while maintaining competitive accuracy.
3 Methodology
3.1 Preliminaries: Edge-Popup under Adversarial Attacks
Let us first formulate the problem in similar manner to (22. We consider neural network parameterized by w E R.. For an input sample (x, y). the neural network outputs f(w,x) x). [[f(w,)) x), y) then represents the training loss objective, where L is the softmax cross-entropy loss. Given a dataset (xx; 2/) ) an adversarial budget Se, and a predefined pruning rate r, we search for a binary pruning mask m that solves the following optimization problem:
1 N N
min
Σ max C(f(w m, m. + +i), s.t. m E 13", sum(mm) = (1 rr.. (2) A.ES.
Here, function sum calculates the summation of all the elements in vector. contrast adversarial training, we do not optimize the model parameters w in 2 instead w contains initialiii parameters that are kept fixed during optimization. As such, our algorithm aims find pruned network structure, encoded via the n--dimensional binary vector m, corresponding to robust subnetwork. Since the mask is discrete vector, it cannot be directly optimized by gradient- based methods. To overcome this, we replace it with continuous "score" variable, s E R", from which we calculate the mask as
m (s,
where M is a binarization function. It constructs binary mask from the continuous- scores based on a pruning strategy and a required pruning rate r. The pruning strategy can be global or laser-wise, and retains the parameters with the highest scores. In the layer-wise case, it automatically determines the number of parameters retained in each layer.
To update the scores we use the same edge-popup strategy as in [22 29 Specifically, we use straight through estimation 4 to calculate the gradient ac/Os. Note that the approximation made in straight through estimation does not affect the adversarial example generation in adversarial training. Our experiments will show that we can effectively generate adversarial examples by PGD. We provide the pseudo-code of our algorithm in Appendix C
3

---

3.2 Adaptive Pruning
As mentioned above, in addition to the given pruning rate r, the binarization function M in Equa- tion (3) also depends on the pruning strategy. [25] uses global pruning, retaining the (1 r)m parameters with the highest scores, regardless of which layer they belong to. However, global pruming does not consider the topology of the network, and the fact that the magnitude of the scores s can differ from layer to layer. Furthermore, when the pruning rate r close to 1, gloobal pruning may prune some layers entirely, thus causing a trivial performance. Therefore, other works [22 51 54 use layer-wise pruning strategies. For an L-layer network with (n,) parameters and predefined pruning rate r, such strategies first allocate the number of parameters {m, ) to retain in each layer, and then retain the parameters with the highest scores in each layer.
In Appendix A. we discuss special cases of layer-wise pruning: fixed pruning rate and fixed number of paramelers. With fixed pruning rate, we have 1 1- = m1 m2
mz Theorem A. indicates that this maximizes the size of the search space of the subnetwork. However, this strategy might retain too few parameters for the small layers when is big, which has two serious drawbacks: 1) It greatly limits the expression power of the network; 2) it makes the edge-popup algorithm less stable, because adding or removing a single parameter then has a large impact on the network's output. This instability becomes even more pronounced in the presence of adversarial samples, because the gradients of the model parameters are more scattered than when training on clean inputs (41.
With fixed number of parameters, we have mi = M2 = = m. When the allocated number of retained parameters exceeds the total number of the original parameters in one layer, we leave this layer totally unpruned. As shown in Theorem A.2 this strategy maximizes the number of paths from the input layer to the output layer. In contrast to the fixed pruning rate, a fixed number of parameters may retain too many parameters in small layers. In the extreme case, some layers may be entirely unpruned when the pruning rate r is small. This is problematic in our settings, since the model parameters are random and not updated.
In other words, the two strategies discussed above are two extremes: the fixed pruning rute one suffers when r big, whereas the fixed number of parameters one suffers when r small. To address this, propose strategy in-between these two extremes. Specifically, determine the number of parameters retained in each layer by solving the following system of equations:
DE= E
m
mi
m2
mL n
1-r=
(4)
n
n4
where p [0, 1] is hyper-parameter controlling the trade-off between the two extreme cases. When p = O, the strategy 4 is the fixed number of parameters one. When p 1, the strategy becomes the fixed pruning rate one. By setting 0 < p < we can retain a higher proportion of parameters in the smaller layers without sacrificing the big layers too much. We call this strategy adaptive pruning. As discussed above, the strategy obtained with p tends fail with a big r, while the strategy resulting from setting p = 0 tends to fail with small r. This indicates that we need to assign small values of p given big r and big values of p otherwise. We validate this and study the influence of p on the results of our approach in experiments.
3.3 Binary Initialization and Last Normalization Layer
Our work focuses on binary networks, which have much smaller memory footprint than full- precision networks but are more challenging to train. To address this, we therefore study the influence of the binary initialization scheme and introduce a last normalization layer approach to facilitate training and boost the performance.
Binary initialization. The empirical studies of [5]] demonstrate the importance of the initial- ization scheme on the performance of a pruned network. As a result, (51) proposes the Signed Kaiming Constant initialization: The parameters in layer i are uniformly sampled from the set
[_1((1-]] where 4-1 represents the fan-out of the previous layer. Correspond-
ingly, the scores s are initialized based on uniform distribution U
The magnitude of the Signed Kaiming Constant initialization carefully calculated to keep the variance of the intermediate activations stable from the input to the output. In modern deep neural

---

networks. the convolutional layers, potentially together with activation functions, are typically followed by a batch normalization layer. In [5I] and our settings. these batch normalization layers only estimate the running statistics of their inputs, they do not have trainable parameters representing affine transformations. Because of these batch normalization layers, the magnitudes of the convolutional layers do not affect the outputs of the Cconvolution-batee norm" blocks. Furthermore, the fully- connected layers on top of the convolutional ones are homogeneous because their bias terms are always initialized to zero and not updated during training. The activation functions we use, such as ReLU or leaky ReLU [ are also homogeneous. Therefore, the magnitudes of parameters in these fully-connected layers do not change the predicted labels of the model either.
Based on the analysis above, we conclude that the magnitudes of the model parameters at initialization do not change the predicted labels. Therefore, we propose to scale the model parameters w in all linear layers, i.e., convolutional and fully-connected ones, so that they are all sampled from {-1,+1) Correspondingly, the scores s are initialized based on uniform distribution [-a, a). where a is factor controlling the variance.
Our binary initialization scheme is beneficial to model compression and acceleration, since there are no longer multiplication operations in linear layers. We discuss the efficiency improvement of the binary networks in detail in Appendix A.2 Theoretically, for the RN34 models we use in this paper, binary initialization can save approximately 45% and 32% FLOP operations compared with their full precision counterparts in the training phase and evaluation phase, respectively. Since we use irregular pruning in our method, taking full advantage of this improvement requires lower-level and hardware customization.
Last normalization layer. Although scaling the model parameters does not affect the expression power of the network nor change the predicted label given the input, it does change the optimization landscape of the problem 2 because the softmax cross-entropy function used to calculate the loss objective is not homogeneous, Compared with the Signed Kaiming Constant method, our binary
initialization multiplies the parameters initialized in the last layer by V [ Therefore, the output logits fed to the softmax cross-entropy function are also multiplied by the same factor. In
practice, V / >> 1 greatly increases the output logits. Large logits will cause numerical instability and thus greatly worsen the optimization performance. In particular, our detailed analysis in Appendix A.. shows that such scaling causes gradient vanishing for correctly classified inputs and, even worse, gradient exploding for misclassified ones.
2
To address this issue, we add another I-dimensional batch normalization layer at the end of the model, just before the softmax layer. The analysis in Appendix A.3 shows this normalization layer cancels out the multiplication factor applied to the weights in the last layer and thus facilitates the optimization. In our experiments, we show that this normalization layer greatly improves the performance of both the Signed Kaiming Constant method and our Binary Initialization one. Furthermore, the last normalization layer also makes the performance more robust to different score s initializations.
4 Experiments
In this section, we present extensive experimental results to validate our approach. First, we describe an ablation study and sensitivity analysis. Then, we compare our performance with existing works, which achieve robustness and compression in either full full-precision or binary cases. We also include adversarial training 44 as a baseline. Finally, we analyze the structure of the pruned networks that we obtain. We show some interesting patterns of these post-pruning networks, suggesting the potential of our approach for more effective compression.
Unless explicitly stated otherwise, we use a 34-layer Residual Network (RN34) 30, the same as the one in ST 54
We use the CIFARIO dataset 36 in the ablation study; we also use the CIFAR10K
dataset 36 and the ImageNet100 dataset 16 20 in the comparisons with the baselines.
We train
We call a function f homogeneous if it satisties Vz Va E Rt. f(x) = af(x).
'Note that the RN34 used in these papers and ours differs from the WideRN34-10 used in 44
58
which is
larger and has almost twice the number of trainable parameters.
'lll these datasets are free for non-commerical use. CIFARIO and CIFARI00 are downloadable on PoTch..
ImageNet can be downloaded from Kaggle and the subset can be found in Contunuum's documentation
5

---

the models for 400 epochs on CIFARI0/100 and 100 epochs on ImageNet100. We use cosine annealing learning rate scheduler with an initial value of 0.1. Unless specified, we employ PGD attacks 44 to generate adversarial examples during training, but we use AutoAttack (AA) 1S for our robustness evaluation. While PGD much faster than AutoAttack and thus suitable for training. AutoAttack is the current state-of-the- metric of robustness. We use an loe norm-based adversarial budget, and the perturbation strength e is 8/255 for CIFARIO, 10, 4/255 for CIFAR 100 and 2/255 for ImageNet100. More details about the experimental settings and hyper-parameters are listed in Appendix D.I
attack method, and we thus consider it a more reliable
4.1 Ablation Study and Sensitivity Analysis
Pruning Strategy and Pruning Rates. We first focus binary initialization and on the models with the last batch normalization layer (LBN). We compare the performance our method under different pruning rates and adaptive pruming strategies with different values of p. The scores are initialized from a uniform distribution U| U000.1, 0.01).
Our results are summarized in Table in which we include 7 different values of pruning rate and 7 different values of p in the adaptive pruning strategy. First, we notice that the best performance 0.99 and p 0.1. For the fixed pruning rate strategy (p 1.0), the best performance is achieved when r = 0.8. Compared with the vanilla (i.e., non-adversarial) case in [5]], which uses the fixed praning rate strategy and shows that 0.5 achieves the best clean accuracy, the best performance for robust accuracy is achieved at much higher pruning rate. This interesting observation is also consistent with the existing work 12 which shows that adversarial training implicitly encourages sparse convolutional kernels.
is achieved when
Prune Strategy
r= 0.5
r= 0.8
r= 0.9
r
0.95
== 0.99
r
0.995
= 0.998
p= 0.0
2.16
6.86
23.01
41.61
44.60
40.70
34.97
p 0.1
4.35
15.03
28.12
42.65
44.88
40.97
33.09
p 0.2
8.01
19.21
27.99
43.72
42.92
40.52
32.99
p
9.21
32.70
42.84
43.62
42.45
40.55
30.08
p == 0.8
28.90
41.51
43.64
43.88
39.12
33.61
28.07
P 0.9
39.09
41.71
43.07
42.28
38.68
33.89
17.43
p 1.0
42.85
43.23
42.13
41.12
34.57
26.67
20.56
Table Robust accuracy (in %) on the CIFARI0 test set under different pruning rates and values of in adaptive pruning. The best result for each pruning rate is marked in bold.
Table I further demonstrates the benefits of our adaptive praning strategy. For larger pruning rates a smaller value prevails; for smaller pruning rates, a bigger value of prevails. This is consistent with our analysis in Section 3.2 In particular, compared with the best results for fixed pruning rate strategy (p- = 1.0, r = 0.8), which is the pruning strategy used [51]. our best adaptive pruning (p 0.1. 0.99) achieves not only better performance but also higher pruning rate. That is to say, using our adaptive pruning strategy improves both robustness and compression rates.
In Figure of Appendix D.2.6 we provide the learning curves when 0.99 and when = 0.5. Regardless the pruning rate r. these curves indicate the importance of the pruming strategy: well chosen p value not only improves the performance but also makes training more stable.
Last Normalization Layer. We then study how the last batch normalization layer (LBN) introduced in Section 5.5 affects the performance. We focus on the binary initialization first and report the performance of models with and without the last normalization layer under different values of the hyper-parameter controlling the variance of the initial score s. Based on the results in Table we use the adaptive pruning strategy with p 0.1 and pruning rate 0.99.
The results are provided in Table 2 and clearly show that the last batch normalization layer (LBN) greatly improves the performance. Furthermore, LBN makes the performance much less sensitive to the initialization of the scores, which in practice facilities the hyper-parameter selection.
Initialization Scheme. Finally, we compare the performance of the binary initialization with the Signed Kaiming Constant. We fix the pruning rate to r 0.99 and employ an adaptive pruning strategy with different values of Our results are summarized in Table For binary initialization,

---

Value of a in in Score Initialization
Prune Strategy
Signed
Binary
no LBN
LBN
no LBN
LBN
no LBN
LBN
p 0.0
39.38
42.83
40.94
44.65
0.001
33.08
45.06
p 0.1
39.62
45.01
41.01
45.06
0.01
39.96
44.88
p = 0.2
36.66
45.04
37.85
41.58
0.1
41.01
44.63
0.5
39.98
42.64
40.61
39.95
31.04
44.41
0.8
37.96
41.71
35.15
38.95
p 0.9
34.75
40.14
35.64
35.81
p =
36.88
39.32
30.02
30.62
Table 2: Robust accuracy (in %) on the CIFARIO test set for models with and without the last batch normalization layer (LBN) under different values of for score s initialization. The best results marked in bold.
Table Robust accuracy (in %) on the CIFAR10 test set with the Signed Kaiming Constant (Signed KC) and the binary initialization. We include models both with and without the last batch normalization layer (LBN). The best results are marked bold.
use the optimal initialization scheme of the from Table 2 for Signed Kaiming Constant initialization, we use the optimal setting from [51] to initialize s.
Based the results in Table 3 we can conclude that the binary initialization achieves a comparable performance with the Signed Kaiming Constant. Furthermore, the last batch normalization layer also improves the performance when using the Signed Kaiming Constant. We show in Table S of Appendix D.2.1 that these conclusions are also valid in a non-adversarial setting.
4.2 Comparison with Existing Methods
Baselines. In this section, we compare our approach with the state-of-the-art methods targeting model compression and robustness. Specifically, we include FlyingBird, FlyingBird+ R Bayesian Connectivity Sampling (BCS) 46 Robust Scratch Ticket (RST) (22). HYDRA [54] and ATMC 25).
as well adversarial training (AT) with early stopping (53]. Given our previous results, the pruning rate to = 0.99. For adversarial training, we use the full RN34 model and some smaller networks with approximately the same number of parameters as our pruned models. These smaller networks have the same architecture as the RN34 except that they have fewer channels. The details of these small networks are shown in Table 7 of Appendix D.I We follow the official implementations of all the baselines, and thus, unlike in our method, the normalization layers in all the baselines that update model parameters have an affine transformation with trainable parameters.
fix
ATMC supports quantization but its parameterization introduces learnable quantized values. That is, although the models obtained by ATMC's 1-bit quantization have only two parameter values in each layer, these values are different from layer to layer and are not necessarily 1 and +1. This means that, compared with the binary networks obtained with our method, those from ATMC have more trainable parameters and thus flexibility. Nevertheless, we still include ATMC for comparison in the case of binary networks. Similarly our method, RST does not update the model parameters. It initializes the model parameters with full-precision values, and we thus only provide full-precision results for RST. The other baselines and AT are designed for quantization and do not inherently support binary networks. To address this, we use BinaryConner layers so that their parameters are binary. BinaryConnect generates binarized model parameters by taking the sign of the weights during the forward pass, and uses straight-through estimation for gradient calculation.
[0] to replace the model's linear
Our method uses binary initialization and the last batch normalization layer, so the models we obtained are inherently binary. In addition to using PGD-based adversarial examples, we accelerate our method by using adversarial examples based on FGSM [24] with ATTA (6). FGSM with ATTA generates adversarial examples by one-step attacks with accumulated perturbations across epochs. This is much cheaper than the 10-step PGD attacks. For CIFAR10 and CIFAR 100, we provide the results of our method when using FGSM with ATTA as "Ours(fast)" For ImageNet100. since the dataset is bigger and the images are of much higher resolution, the computational cost for multi-step PGD huge. Therefore, we use FGSM with ATTA to generate adversarial examples for all methods on ImageNet100. To decrease the memory overhead introduced
Table 4 for comparison.
7

---

Pruning
CIFARIO
CIFARI00
ImageNet100
Method
Architecture
Strategy
FP
Binary
FP
Binary
FP
Binary
AT
RN34
Not Pruned
43.26
40.34
36.63
26.49
53.92
34.20
AT
RN34-LBN
Not Pruned
42.39
39.58
35.15
32.98
55.14
35.36
AT
Small RN34
Not Pruned
38.81
26.00
27.68
15.85
23,40
10.44
FlyingBird
RN34
Dynamic
45.86
34.37
35.91
23.32
37.70
9.54
FlyingBird+
RN34
Dynamic
44.57
33.33
34.30
22.64
37.70
9.52
BCS
RN34
Dynamic
43.51
31.85
RST
RN34
p 1.0
34.95
21.96
17.54
RST
RN34-LBN
p = 1.0
37.23
23.14
15.36
HYDRA
RN34
p= 0.1
42.73
29.28
33.00
23.60
43.18
18.22
ATMC
RN34
Global
34.14
25.62
25.10
11.09
22.18
5.78
ATMC
RN34
p 0.1
34.58
24.62
25.37
11.04
23.52
4.58
Ours
RN34-LBN
p= 0.1
45.06
34.83
33.04
Ours(fast)
RN34-LBN p 0.1
40.77
34.45
Table 4: Robust accuracy (in %) on the CIFARIO, CIFARI00 and ImageNet100 test sets for the baselines and our proposed method. "RN34-LBN" represents ResNet34 with the last batch normal- ization layer. "Small RN34" refers to Small RN34-p0.1 Table 7 of Appendix [.I] The pruning rate is set to 0.99 except for the not-pruned methods. Among the pruned models, the best results for the full-precision (FP) models are underlined; the best results for the binary models are marked in bold. The values of for CIFARIO, CIFAR 100 and ImageNet100 are 8/255, 4/255 and 2 2/255, respectively. means not applicable or trivial performance.
by ATTA, we only store the downsampled perturbations in the current epoch for the perturbation initialization of the next epoch. We provide the pseudo-code and more details in Appendix C
Results. Our main results on CIFARIO, CIFARI00 and ImageNet100 are summarized in Table 4 where report the robust accuracy under AutoAttack (AA), which considered evaluation metric for robustness [15] The results of all baselines are based on their default settings in architecture and pruning strategy based publicly available codes. F The exceptions are that also include adaptive pruning (p = 0.1) for HYDRA, ATMC, and the last batch normalization layer for RST, because we noticed such changes to improve their performance.
reliable
Our method using the adaptive pruming strategy (p = 0.1) achieves better performance than all baselines in case of binary models. On CIFARIO 10 and CIFAR100, also achieve comparable performance to methods using fuee-precision models. Furthermore, our method achieves results comparable with AT on the original unpruned models that has 100x more trainable parameters. In addition, our method based on FGSM with ATTA, which is much faster than multi-step PGD, also achieves better performance than all baselines in the case of binary networks. On ImageNet our method, which aims to train binary networks, also outperforms most full-precision networks trained by the baselines. BCS yields almost trivial performance on ImageNet (< 3%) and is thus not included. This suggests that BCS cannot converge using a high compression rate and facing complicated dataset. Compared with adversarial training on the full network, which has 100 times as many parameters as ours, we achieve comparable performance with the binary networks, but worse performance than the full- precision networks. Note that fitting the digh-dimensional ImageNet100 dataset under adversarial attacks using only 1 % of the binary parameters is extremely challenging. As demonstrated in Table many baselines only achieve low robust accuracy this setting.
For all baselines except RST, the last normalization layer does not improve the performance; it even hurts the performance in the full-precision cases. This is because these baselines (except RST) update the model parameters w. In the full precision cases, the magnitude of w, and thus of the output logits, automatically adjusted during training. The issue resulting from large output logits that we pointed out in Section 3 does thus not happen in these cases, so the last batch normalization layer is not necessary. practice, we observed this layer to slow down the training convergence of these models.
'Publicly available code on GitHub: FlyingBind/FlyingBiittt VITA-Group//paanee Group/Sparsity Win-Robustt Generalization BCS: inspire-groute groph/hggoup/hyyrr ATMC: VITA Grou//AAAA All the codes are free use for non-commerical purposes.
RST. RICE CICEE-EIC/Robust CeCC/oobut--Sracchh Ticket HYDRA:
8

---

For the pruning strategy, the proposed adaptive pruning strategy (p = 0.1) consistently achieves better performance than the fix pruning rate strategy (p = 1.0) and than global pruning. FlyingBird. FlyingBird+ and BCS dynamically assign retrained parameters during training, which has similar benefits to adaptive pruning but the cost of training efficiency [ Furthermore, although the value of p is selected based on the ablation study on 10, it also performs well on CIFAR 1 100 and ImageNet100, This observation indicates that for a fixed value of r, the selection of p generalizes well across different datasets.
We further compare the baseline methods with various settings such adding the last batch normal- ization layer, changing the pruning strategy, using different AT methods, and provide a complete set of comparison results Appendix D.2.2
The conclusions drawn from Table 4 remain valid.
Pruning
RNI8
RN50
Method
Architecture
Strategy
FP
Binary
FP
Binary
AT
RN
Not Pruned
41.50
39.13
43.24
31.18
AT
RN-LBN
Not Pruned
42.25
39.86
44.33
37.25
AT
Small RN
Not Pruned
28.13
30.35
26.03
32.25
FlyingBird
RN
Dynamic
42.15
27.08
35.91
26.33
FlyingBird+
RN
Dynamic
38.55
27.84
29.54
25.40
BCS
RN
Dynamic
39.60
21.46
41.85
17.54
RST
RN
p 1.0
31.98
35,40
RST
RN-LBN
p = 1.0
33.27
34.71
HYDRA
RN
p == 0.1
40.20
30.90
44.14
22.36
ATMC
RN
Global
32.21
17.73
25.23
6.82
ATMC
RN
p 0.1
32.31
19.67
33.61
16.12
Ours
RN-LBN
= 0.1
39.65
42.72
Ours (fast)
RN-LBN
p 0.1
30.86
37.93
Table 5: Robust accuracy (in %) on the CIFARIO test for AT, FlyingBird(4), BCS, RST, HYDRA,
ATMC and our proposed method on the RNI8 and RN50 models. "RN-LBN" represents networks with the last batch normalization layer. Among the compressed models, the best results for full precision (FP) models are underlined; the best results for binary models are marked in bold.
All the results in Table f are based on a RN34 architecture, Table provides the results on CIFARIO using smaller 18-layer network (RN18) and larger 50-layer network (RN50). These results confirm the effectiveness of our method on different network architectures.
In addition to robust accuracy, Table 10 in Appendix D.2.3 demonstrates the accuracy on the clean test set for the models in Table 4 Our method also yields competitive performance on clean inputs. Specifically, we achieve the best performance among all methods for binary networks. Combining the results in Table 8i and TO we conclude that our method yields better trade-off between accuracy on clean inputs and accuracy on adversarially perturbed inputs.
special case of adversarial training, where e 0. Therefore, our method, as well as all baselines, are applicable to vanilla training. The results when are provided Table IT of Appendix D.2.4] Our method achieves the best performance among the pruned binary networks. This indicates that our method is competitive under difference adversarial budgets.
Finally, vanilla training can be considered
4.3 Analysis of the Subnetwork Patterns
In this work, we use irregular pruning. Compared with regular pruning, irregular pruning is more flexible but less structured, which means that it requires lower-level customization to fully take advan- tage of parameter sparsity for acceleration. However, visualizing the masks m of the convolutional layers in our pruned binary network with a pruning rate r = 0.99 allowed us to find that the mask structured to some degree. For example, we visualize the mask of a convolutional layer with 256 input channels and 256 output channels in Figure 5 of Appendix D.2.5, We notice that the retained parameters are quite concentrated and structured: Most retained parameters concentrate on few input or output channels, while many other channels (40% of the total) are completely pruned.

---

Furthermore, we visualize two consecutive convolutional layers in the same residual block of the RN34 model. We call them layerl and layer2 following the forward pass. In Figure [ we plot the distribution of the retained parameters in each input channel and in each output channel, respectively. We find that many output channels layerl and input channels of layer2, 40% of all channels in this case, are totally pruned. As reference, we also plot the distribution of random pruning, based on the average of 500 simulations. As demonstrated in Figure |I the distribution of the retained parameters in each channel is much more uniform in this case. Our theoretical analysis in Appendix A.4 demonstrates that, in a randomly pruned network, it is almost impossible to have even one entirely pruned channel. The comparison indicates that the mask obtained by our method is structured.
#Remaining weights
60
random
#Remaining weights
100
random
#Remaining weights
random
#Remaining weights
random
40
pruned
75
pruned
100
75
pruned
2000
pruned
50
50
100
20
25
25
0
200 Channel index
100
0
200 Channel Index
100
100
200
200 Channel Index
100
Channel index
(a) LI Input Channel
(b) L1 Output Channel
(c) L.2 Input Channel
(d) L2 Output Channel
Figure I: Number of retained parameters in each input and output channel of layerl (LI) and layer2 (L2) in the same residual block. We sort the numbers and plot the curves from the largest on the left to the smallest on the right. The red curves represent the mask obtained by our method; the blue curves depict what happens when randomly pruning the corresponding layer.
The observations in Figure I also also hold for kernels: a few kernels, of size 3 and thus having 9 entries, are totally unpruned. We show the distribution of the number of retained parameters in each kernel in Figure 2 and provide the distribution by random uniform pruning as a reference. Random uniform pruning yields no kernels with more than 3 retained parameters, but many such kernels can be observed in the masks generated by method. Finally, in Figure S of Appendix D.2.5 we visualize the positions of the pruned output channels of layerl and the pruned input chan- nels of layer2. We observe those pruned channels to be aligned. That is, some neurons representing both the output channels of layerl and the input channels of layer2 are entirely removed. We defer additional discussions, figures and results to Appendix D.2.5]
Number kernels
10a
random pruned
104
10a
10a
10:
2
3
4
6 7
Number kept weights kerne
Figure 2: Distribution of the number of retained parameters in each kernel. The y-axis is in log-scale.
The pattern of the structures learned by our method indicates the potential of regular pruning for randomly -inttaiiii network in the presence of adversarial attacks. We leave this as future work.
5 Conclusion
We have proposed a method to obtain robust binary models by pruning randomly-initialized networks, thus extending the Strong Lottery Ticket Hypothesis to the case of robust binary networks. In contrast to the state-of-the-art methods, we learn the structure of robust subnetworks without updating the parameters. Furthermore, we have proposed an adaptive pruning strategy and last batch normalization layer to stabilize the training and improve performance. Finally, we relide on binary initialization to obtain more compact models.
Our extensive results on various benchmarks have demonstrated that our approach outperforms existing methods for training compressed robust models. Furthermore, we have observed interesting structured patterns occurring in the parameters retained in the subnetworks. This opens the door to further investigations on the structure of the robust subnetworks and on the design of regular pruning strategies in the adversarial scenario.
10

---

References
[1] Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, pages 12192-12202, 2019.
[2] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: a query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision, pages 484-501. Springer, 2020.
[3] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv: Drxiiv/802.... (802.00420) 2018.
[4] Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint
2013.
[5] Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. Thermometer encoding: One hot way to resist adversarial examples. In International Conference on Learning Representations, 2018.
[6] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In Advances Neural Information Processing Systems, pages 11 11190-111 10000012011,.1111110001001111111 1201, 2019.
[7] Jinghui Chen, Yu Cheng, Zhe Gan, Quanquan Gu, and Jingjing Liu. Efficient robust training via backward smoothing, 2021.
[8] Tianlong Chen, Zhenyu Zhang, pengjun wang. Santosh Balachandra, Haoyu Ma, Zehao Wang, and Zhangyang Wang. Sparsity winning twice: Better robust generalization from more efficient training. In International Conference Learning Representations, 2022.
[9] Daiki Chijiwa, Shin'y ya Yamaguchi, Yasutoshi Ida, Kenji Umakoshi, and Tomohiro INOUE. Pruning randomly initialized neural networks with iterative randomization. In A. Beygelzimer, Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[10] Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David, Binaryconnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems, pages 3123-3131, 2015.
[11] Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran EI- Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ or-1, arXiv preprint arXiv:1602.02830, 2016.
[12] Francesco Croce, Maksym Andriushchenko. and Matthias Hein. Provable robustness of relu networks via maximization linear regions. AISTATS 2019, 2019.
[13] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. arXiv preprint urXiv:2010.09670 2020.
[14] Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International Conference on Machine Learning, pages 2196-2205. PMLR, 2020.
[15] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, 2020.
[16] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEFE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.

---

[17] James Diffenderfer, Brian R Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura. winning hand: Compressing deep networks can improve out-of-distribution robustness. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Newral Information Processing Systems, 2021.
[18] James Diffenderfer and Bhavya Kailkhura. Multi-prize lottery ticket hypothesis: Finding accurate binary neural networks by pruning randomly weighted network. In International Conference on Learning Representations, 2021.
[19] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[20] Arthur Douillard and Timothée Lesort. Continuum: Simple management of complex continual learning scenarios, 2021.
[21] Jonathan Frankle and Michael Carbin, The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019.
[22] Yonggan Fu, Qixuan Yu, Yang Zhang. Shang Wu, Xu Ouyang, David Daniel Cox, and Yingyan Lin. Drawing robust scratch tickets: Subnetworks with inborn robustness are found within randomly initialized networks. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[23] Angus Galloway, Graham W. Taylor, and Medhat Moussa. Attacking binarized neural networks. In International Conference on Learning Representations, 2018.
[24] lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6572, 2014.
[25] Shupeng Gui, Haotao N Wang, Haichuan Yang, Chen Yu, Zhangyang Wang, and Ji Liu. Model compression with adversarial robustness: unified optimization framework. In Advances in Neural Information Processing Systems, pages 1283-1294, 2019.
[26] Yiwen Guo, Anbang Yao, and Yurong Chen. Dynamic network surgery for efficient dnns. Advances in Neural Information Processing Systems, 29:1379-1387, 2016.
[27] Song Han, Huizi Mao, and William Dally. Deep compression: Compressing deep neural net- works with pruning, trained quantization and huffman coding. arXiv preprint 110.v:1510.00149 2015.
[28] Song Han, Jeff Pool, John Tran, and William J Dally. Learning both weights and connections for efficient neural networks. Advances in neural information processing systems, 28:1 3143, 2015.
[29] Babak Hassibi, David G Stork, and Gregory J Wolff. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks, pages 293-299, IEEE, 1993.
[30] Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[31] Yihui Hc, Xiangyu Zhang, and Jian Sun. Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE international conference on computer vision, pages 1389-1397, 1000-139,,001,,, 2017.
[32] Dan Hendryeks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, pages 2712- 2721, 2019.
[33] Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks. Advances in neural information processing systems, 29, 2016.
12

---

[34] Qing Jin, Linjie Yang, and Zhenyu Liao. Adabits: Neural network quantization with adaptive bit-widths. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2146-2156, 2020.
[35] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
[36] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[37] Nupur Kumari, Mayank Singh, Abhishek Sinha, Harshitha Machiraju, Balaji Krishnamurthy, and Vineeth N Balasubramanian. Harnessing the vulnerability of latent layers in adversarially trained models. In Proceedings of the Twentt-Fighth International Joint Conference on Arti- ficial Intelligence, IJCAI-19, pages 2779-2785. International Joint Conferences on Artificial Intelligence Organization, 7 2019.
[38] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv: (XX.:0007..05533 2016.
[39] Yann LeCun, John S Denker, and Sara A Solla. Optimal brain damage. In Advances in neural information processing systems, pages 598-605, 1990.
[40] Lin, Chuang Gan, and Song Han. Defensive quantization: When efficiency meets robustness. In International Conference on Learning Representations, 2019.
[41] Chen Liu, Mathieu Salzmann, Tao Lin, Ryota Tomioka, and Sabine Süsstrank. On the loss landscape of adversarial training: Identifying challenges and how to overcome them. Advances in Neural Information Processing Systems, 33, 2020.
[42] Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang, Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE international conference on compuler vision, pages 2736-2744, 2017.
[43] Andrew I. Maas, Awni Y Hannun, Andrew Y Ng, et al. Rectifier nonlinearities improve neural network acoustic models. Citeseer.
[44] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
[45] Seyed-Mohsen Moosavi-Dezfoolia Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Univer- sal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1765-1773, 2017.
[46] Ozan Özdenizei and Robert Legenstein. Training adversarially robust sparse networks via bayesian connectivity sampling. In International Conference on Machine Learning, pages 8314-8324. PMLR, 2021.
[47] Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, and Jun Zhu. Rethinking softmax cross-entropy loss for adversarial robustness. In International Conference on Learning Representations, 2020.
[48] Antonio Polino, Razvan Pascanu, and Dan Alistarh. Model compression via distillation and quantization. In International Conference on Learning Representations, 2018
[49] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. arXiv preprint 1801v 09344, 2018.
[50] Adnan Siraj Rakin, Jinfeng Yi, Boqing Gong, and Deliang Fan. Defend deep neural networks against adversarial examples via fixed and dynamic quantized activation functions. arXiv preprint arXiv: 2018
[51] Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari. What's hidden in a randomly weighted neural network? In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 11893-11902, 2020.
13

---

[52] Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy Mann. Data augmentation can improve robustness. In A. Beygelzimer, Y. Dauphin. P. Liang, J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[53] Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning, pages 8093-8104, PMLR, 2020.
[54] Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. Hydra: Pruning adversarially robust neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 19655-19666. Curran Associates, Inc., 2020.
[55] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
[56] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008, 2017.
[57] Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, 2020.
[58] Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems, 33, 2020.
[59] Chang Xiao, Peilin Zhong, and Changxi Zheng. Enhancing adversarial defense by k-winners- take-all. In International Conference on Learning Representations, 2020.
[60] Cihang Xie and Alan Yuille. Intriguing properties of adversarial training at scale. In Interna- tional Conference on Learning Representations, 2020.
[61] Tien-Ju Yang. Yu-Hsin Chen, and Vivienne Sze. Designing energy-efficient convolutional neural networks using energy-aware pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5687-5693 2017.
[62] Shaokai Ye, Kaidi Xu, Sijia Liu, Hao Cheng, Jan-Henrik Lambrechts, Huan Zhang, Aojun Zhou, Kaisheng Ma, Yanzhi Wang, and Xue Lin. Adversarial robustness vs. model compression, or both. In The IEEE International Conference on Computer Vision (ICCV), volume 2, 2019.
[63] Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only prop- agate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pages 227-238, 2019.
[64] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pages 7472-7482, 2019.
[65] Tianyun Zhang, Shaokai Yc, Kaiqi Zhang, Jian Tang, Wujie Wen, Makan Fardad, and Yanzhi Wang. A systematic dnn weight pruning framework using alternating direction method of multipliers. In Proceedings of the European Conference on Computer Vision (ECCV), pages 184-199 2018
[66] Haizhong Zheng, Ziqi Zhang, Juncheng Gu, Honglak Lee, and Atul Prakash. Efficient adversar- ial training with transferable adversarial examples. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 1181-1190, 2020.
[67] Aojun Zhou, Anbang Yao. Yiwen Guo, Lin Xu, and Yurong Chen. Incremental network quantization: Towards lossless cnns with low-precision weights. In International Conference on Learning Representations, 2017.
[68] Hattie Zhou, Janice Lan, Rosanne Liu, and Jason Yosinski. Deconstructing lottery tickets: Zeros, signs, and the supermask. Advances in Neural Information Processing Systems, 32:3597-3607. 2019.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the pa- per's contributions and scope? [Yes] The paragraphs before the Notation part in the introduction demonstrate the findings and the contributions of this paper.
(b) Did you describe the limitations of your work? [Yes] This paper focuses on irregular pruning. We point out that we need hardware customization to fully take advantage of the efficiency improvement.
(c) Did you discuss any potential negative societal impacts of your work? [No] Our proposed algorithms are generic. This paper does not focus particular application.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] All the theoretical results are put in the appendix.
(b) Did you include complete proofs of all theoretical results? [Yes] We provide detailed proof' for all theorems we claim.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL.?? [Yes] It is in the supplemental material.
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] We highlight key settings in the experiment part. More details are put the appendix.
(c) Did you report error bars (c.g., with respect to the random seed after running exper- iments multiple times)? [No] We do not report the error bars in the main table that compares our method with baselines. As the ablation study in Table 2 shows, when we the last batch normalization layer and adaptive pruning. the performance variance for different initializations small, compared with the performance gap between our methods and baselines.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] We put these details in the appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether
how consent was obtained from people whose data you're
using/curating? [N/A] All the data use is from popular benchmarks
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "oZ9Gq7uY0N", "reviewer": "Reviewer_HVEC", "summary": "This paper presents a method for searching binary randomly initialized networks to generate pruned and robust models. The paper leverages the existing edge-popup algorithm for pruning untrained networks via learning important scores instead of model weights. The paper also proposes adding a last layer batchnorm to stabilize the training of their compressed robust models. The paper conducts several experiments to support their claim of achieving state-of-the-art robust binary networks.", "questions": "1) In the Signed Kaiming Constant initialization (SKC) in line 175, is the 1-r a typo? since the original paper does not have the pruning ratio in the initialization step. If it is not a typo, does that mean every pruning ratio requires a different randomly initialized network?\n\n2) Is the last normalization layer a standard batchnorm layer, or a homogenous (affine=False) batchnorm layer as used in other layers?\n\n3) The observation that more pruning leads to higher robustness is very intriguing, especially when existing pruning + adversarial training methods, such as HYDRA [1] and ADMM [2], typically show that robustness decreases with the pruning rate (as is expected). Moreover, the edge-pop up algorithm typically achieves its best performance (in the non adversarial setting) when r=0.5, as it maximizes the number of possible network choices. Could the authors shed more light on the reason why their method seems to contradict both these existing observations?\n\n4) If I understood correctly, the proposed method is able to generate 99% pruned binary networks, that achieve adversarial robustness that is higher than all other baselines, which includes full precision adversarially trained models and their pruned versions, as well as other trained binary models. For context, the state-of-the-art 99% pruned full precision baseline achieved by HYDRA suffers massive degradation in terms of robustness, when compared to an unpruned baseline. The results reported in this paper are very impressive and also very intriguing, as they seems too good to be true. I believe more experiments/analysis is warranted to explain why this method is so much more effective than prior work. One simple experiment perhaps is to initialize the weights to constant values between [-1,1] instead of the binary {-1,1}, and keep everything else the same. One would expect that pruning this baseline should have even better robust accuracy than the binarized version, and thus surpass that of any FP baseline. Another question is what would happen if we train the pruned models from scratch (that is take the pruning mask and initialize another model for adversarial training). It would be great if the authors can address this. \n\n[1] Sehwag, V., Wang, S., Mittal, P., & Jana, S. (2020). Hydra: Pruning adversarially robust neural networks. Advances in Neural Information Processing Systems, 33, 19655-19666.\n\n[2] Ye, S., Xu, K., Liu, S., Cheng, H., Lambrechts, J. H., Zhang, H., ... & Lin, X. (2019). Adversarial robustness vs. model compression, or both?. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 111-120).", "limitations": "The authors claim that their work is generic and does not suffer from limitations or potential negative societal impact.", "rating": 7, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is well motivated\n- The quality of writing is good, the ideas are well explained\n- The experimental results support the authors' claims, achieving state-of-the-art results for pruned binary robust models\n\nWeaknesses:\n- The novelty of this work is a bit limited, as it combines existing techniques (edge-popup + adversarial loss similar to [1] and [2]), that being said the end results are very impressive\n- I would have liked to see more analysis on why the method is so successful, as opposed to adversarial training and/or binarization/pruning schemes\n\n\n[1] Ramanujan, V., Wortsman, M., Kembhavi, A., Farhadi, A., & Rastegari, M. (2020). What's hidden in a randomly weighted neural network?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11893-11902).\n\n[2] Sehwag, V., Wang, S., Mittal, P., & Jana, S. (2020). Hydra: Pruning adversarially robust neural networks. Advances in Neural Information Processing Systems, 33, 19655-19666.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "XmrJtys65n", "reviewer": "Reviewer_WyYF", "summary": "The paper proposes a method for learning robust binary neural networks from random initialization. The proposed training scheme includes  adaptive pruning approach and an edge pop-up algorithm for training the binary model to obtain a binary mask for pruning using a PGD-like method. It shows clear improvement over other adversarial training methods for binary models in experiments. Some theoretical analysis for the adaptive pruning method is provided.", "questions": "- Can the authors provide some intuition or analysis regarding why this training scheme could work?", "limitations": "- It would be interesting to investigate whether the proposed training method can also be applied to training binary Transformers.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n- The training algorithm is clearly stated makes sense. \n- The experimental results are impressive with evident improvement over comparing methods.\n- Some theoretical analysis for the adaptive pruning method is provided.\n- The ablation study and sensitivity analysis are very clear and are consistent with intuition. For example, the ablation study for the hyper-parameter $p$ for adaptive pruning aligns very well with the analysis.\n\n### Weakness\n- Only one neural net model (ResNet34) is evaluated in the experiments.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "VgE5h7sGolQ", "reviewer": "Reviewer_ATBX", "summary": "This manuscript extends the Strong Lottery Ticket Hypothesis to binary networks and proposed a new method to obtain compact binary networks with better robustness. The proposed method consists of three main parts: a) adaptively pruning different network layers; b) an effective binary initialization scheme; c) adding the last batch normalization layer. Extensive experiments on CIFAR-10/100 and ImageNet-100 validate the effectiveness of the method.", "questions": "None", "limitations": "To some extent, this paper may help to build more secure and compact deep learning systems in the real world. But the real-world system faces all types of attacks, which are far beyond adversarial examples.\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "#### Strength:\n\n1. The paper is well written with sufficient details of the proposed methods.\n2. Experimental results on three datasets show superior performance against multiple baseline methods, including vanilla AT, Flying Bird, BCS, RST, HYDRA, and ATMC.\n3. Analysis of the derived sparse patterns is provided.\n\n#### Weakness:\n\n1. The proposed methods search for sparse binary subnetworks within randomly-initialized networks. It's unclear whether such a scheme is better than training the parameters directly. It seems the proposed methods (adaptive pruning, binary initialization, and last batch normalization layer) can also to used to optimize the parameters of randomly-initialized networks. \n2. As the proposed methods aim to obtain binary neural networks, which are both compact and robust. The comparison about training and inference cost (e.g., FLOPs) is important.\n3. It seems better to firstly report the comparison results in Section 4.2, and then conduct the ablation study of three proposed methods (Section 4.1).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "S1YCT_T2Gyr", "reviewer": "Reviewer_WydW", "summary": "For a robust binary model built with Strong Lottery Ticket Hypothesis, the paper proposes a model compress method with three strategies: adaptive layer-wise pruning, a binary initialization scheme, and adding a last batch normalization layer. Experiments on CIFAR-10 and CIFAR-100 show that the proposed method outperforms previous competitive methods such as ATMC [22].", "questions": "1) Are the proposed three techniques useful for regular DNNs in addition to the robust binary models [19]?\n2) Why do the methods in Table 4 use different $p$ value? Are the methods in Table 4 fairly compared with each other?\n3) Is the proposed method scalable to large-scale datasets such as tinyImageNet?", "limitations": "As mentioned above, a major limitation is that the paper's proposed three techniques are not systematical but auxiliary model compression strategies.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.\n1) In Section 1 and Section 3.1 (Equation (2)), the paper clearly introduces the base model [19] that the paper works on: a robust binary model built with Strong Lottery Ticket Hypothesis.\n2) In Section 3.2 and Section 3.2, introduction of the proposed three techniques is easy-to-follow, except too more \"In Appendix ...\". The proposed three techniques, adaptive layer-wise pruning, a binary initialization scheme, and adding a last batch normalization layer, are reasonable.\n3) Experiments on CIFAR-10 and CIFAR-100 show that the proposed method outperforms previous competitive methods such as ATMC [22]. The paper further provides necessary analysis on the subnetwork patterns, and sensitivity analysis on the pruning strategy and added last batch-normalization layer.\n\nWeaknesses.\n1) My major concern is that the paper's proposed three techniques are not systematical but auxiliary model compression strategies. For example, the proposed adaptive pruning is a straightforward trade-off between fixed pruning rate and fixed number of parameters, and the key parameter $p$ is manually but not automatically selected in all experiments. Moreover, the paper works on a base model [19] which is already a robust binary model. The paper focuses on model compression only but not further improving binarization and adversarial robustness. \n2) In Table 4, different methods use different $p$, e.g., $p=1.0$ for RST and $p=0.1$ for the proposed method. I think all methods should use the same $p$ value for fair comparison.\n3) Since tiny-scale datasets CIFAR-10 and CIFAR-100 are easily to be overfitting. I suggest the authors use large-scale datasets such as tinyImageNet for experiments. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept