SPARSITY WINNING TWICE: BETTER ROBUST GEN- ERALIZATION FROM MORE EFFICIENT TRAINING
Year: 2022

Abstract:
Recent studies demonstrate that deep networks, even robustified by the state-ofthe-art adversarial training (AT), still suffer from large robust generalization gaps, in addition to the much more expensive training costs than standard training. In this paper, we investigate this intriguing problem from a new perspective, i.e., injecting appropriate forms of sparsity during adversarial training. We introduce two alternatives for sparse adversarial training: (i) static sparsity, by leveraging recent results from the lottery ticket hypothesis to identify critical sparse subnetworks arising from the early training; (ii) dynamic sparsity, by allowing the sparse subnetwork to adaptively adjust its connectivity pattern (while sticking to the same sparsity ratio) throughout training. We find both static and dynamic sparse methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overfitting, meanwhile significantly saving training and inference FLOPs. Extensive experiments validate our proposals with multiple network architectures on diverse datasets, including CIFAR-10/100 and TinyImageNet. For example, our methods reduce robust generalization gap and overfitting by 34.44% and 4.02%, with comparable robust/standard accuracy boosts and 87.83%/87.82% training/inference FLOPs savings on CIFAR-100 with ResNet18. Besides, our approaches can be organically combined with existing regularizers, establishing new state-of-the-art results in AT. Codes are available in https: //github.com/VITA-Group/Sparsity-Win-Robust-Generalization.

text: 1 INTRODUCTION
Deep neural networks (DNNs) are notoriously vulnerable to maliciously crafted adversarial attacks. To conquer this fragility, numerous adversarial defense mechanisms are proposed to establish robust neural networks (Schmidt et al., 2018; Sun et al., 2019; Nakkiran, 2019; Raghunathan et al., 2019; Hu et al., 2019; Chen et al., 2020c; 2021e; Jiang et al., 2020). Among them, adversarial training (AT) based methods (Madry et al., 2017; Zhang et al., 2019) have maintained the state-of-the-art robustness. However, the AT training process usually comes with order-ofmagnitude higher computational costs than standard training, since multiple attack iterations are needed to construct strong adversarial examples (Madry et al., 2018b). Moreover, AT was recently revealed to incur severe robust generalization gaps (Rice et al., 2020), between its training and testing accuracies, as shown in Figure 1; and to require significantly more training samples (Schmidt et al., 2018) to generalize robustly.
*Equal Contribution.
1
In response to those challenges, Schmidt et al. (2018); Lee et al. (2020); Song et al. (2019) investigate the possibility of improving generalization by leveraging advanced data augmentation techniques, which further amplifies the training cost of AT. Recent studies (Rice et al., 2020; Chen et al., 2021e) found that early stopping, or several smoothness/flatness-aware regularizations (Chen et al., 2021e; Stutz et al., 2021; Singla et al., 2021), can bring effective mitigation.
In this paper, a new perspective has been explored to tackle the above challenges by enforcing appropriate sparsity patterns during AT. The connection between robust generalization and sparsity is mainly inspired by two facts. On one hand, sparsity can effectively regularize the learning of over-parameterized neural networks, hence potentially benefiting both standard and robust generalization (Balda et al., 2019). As demonstrated in Figure 1, with the increase of sparsity levels, the robust generalization gap is indeed substantially shrunk while the robust overfitting is alleviated. On the other hand, one key design philosophy that facilitates this consideration is the lottery ticket hypothesis (LTH) (Frankle & Carbin, 2019). The LTH advocates the existence of highly sparse and separately trainable subnetworks (a.k.a. winning tickets), which can be trained from the original initialization to match or even surpass the corresponding dense networks’ test accuracies. These facts point out a promising direction that utilizing proper sparsity is capable of boosting robust generalization while maintaining competitive standard and robust accuracy.
Although sparsity is beneficial, the current methods (Frankle & Carbin, 2019; Frankle et al., 2020; Renda et al., 2020) often empirically locate sparse critical subnetworks by Iterative Magnitude Pruning (IMP). It demands excessive computational cost even for standard training due to the iterative train-prune-retrain process. Recently, You et al. (2020) demonstrated that these intriguing subnetworks can be identified at the very early training stage using one-shot pruning, which they term as Early Bird (EB) tickets. We show the phenomenon also exists in the adversarial training scheme. More importantly, we take one leap further to reveal that even in adversarial training, EB tickets can be drawn from a cheap standard training stage, while still achieving solid robustness. In other words, the Early Bird is also a Robust Bird that yields an attractive win-win of efficiency and robustness - we name this finding as Robust Bird (RB) tickets.
Furthermore, we investigate the role of sparsity in a scene where the sparse connections of subnetworks change on the fly. Specifically, we initialize a subnetwork with random sparse connectivity and then optimize its weights and sparse typologies simultaneously, while sticking to the fixed small parameter budget. This training pipeline, called as Flying Bird (FB), is motivated by the latest sparse training approaches (Evci et al., 2020b) to further reduce robust generalization gap in AT, while ensuring low training costs. Moreover, an enhanced algorithm, i.e., Flying Bird+, is proposed to dynamically adjust the network capacity (or sparsity) to pursue superior robust generalization, at few extra prices of training efficiency. Our contributions can be summarized as follows:
• We perform a thorough investigation to reveal that introducing appropriate sparsity into AT is an appealing win-win, specifically: (1) substantially alleviating the robust generalization gap; (2) maintaining comparable or even better standard/robust accuracies; and (3) enhancing the AT efficiency by training only compact subnetworks.
• We explore two alternatives for sparse adversarial training: (i) the Robust Bird (RB) training that leverages static sparsity, by mining the critical sparse subnetwork at the early training stage, and using only the cheapest standard training; (ii) the Flying Bird (FB) training that allows for dynamic sparsity, which jointly optimizes both network weights and their sparse connectivity during AT, while sticking to the same sparsity level. We also discuss a FB variant called Flying Bird+ that adaptively adjusts the sparsity level on demand during AT.
• Extensive experiments are conducted on CIFAR-10, CIFAR-100, and Tiny-ImageNet with diverse network architectures. Specifically, our proposals obtain 80.16% ∼ 87.83% training FLOPs and 80.16% ∼ 87.83% inference FLOPs savings, shrink robust generalization from 28.00% ∼ 63.18% to 4.43% ∼ 34.44%, and boost the robust accuracy by up to 0.60% and the standard accuracy by up to 0.90%, across multiple datasets and architectures. Meanwhile, combining our sparse adversarial training frameworks with existing regularizations establishes the new state-of-the-art results.

heading: 2 RELATED WORK
text: Adversarial training and robust generalization/overfitting. Deep neural networks present vulnerability to imperceivable adversarial perturbations. To deal with this drawback, numerous defense
2
approaches have been proposed (Goodfellow et al., 2015; Kurakin et al., 2016; Madry et al., 2018a). Although many methods (Liao et al., 2018; Guo et al., 2018a; Xu et al., 2017; Dziugaite et al., 2016; Dhillon et al., 2018a; Xie et al., 2018; Jiang et al., 2020) were later found to result from obfuscated gradients (Athalye et al., 2018), adversarial training (AT) (Madry et al., 2018a), together with some of its variants (Zhang et al., 2019; Mosbach et al., 2018; Dong et al., 2018), remains as one of the most effective yet costly approaches.
A pitfall of AT, i.e., the poor robust generalization, was spotted recently. Schmidt et al. (2018) showed that AT intrinsically demands a larger sample complexity to identify well-generalizable robust solutions. Therefore, data augmentation (Lee et al., 2020; Song et al., 2019) is an effective remedy. Stutz et al. (2021); Singla et al. (2021) related robust generalization gap to curvature/flatness of loss landscapes. They introduced weight perturbing approaches and smooth activation functions to reshape the loss geometry and boost robust generalization ability. Meanwhile, the robust overfitting (Rice et al., 2020) in AT usually happens with or as a result of inferior generalization. Previous studies (Rice et al., 2020; Chen et al., 2021e) demonstrated that conventional regularization-based methods (e.g., weight decay and simple data augmentation) can not alleviate robust overfitting. Then, numerous advanced algorithms (Zhang et al., 2020; 2021b; Zhou et al., 2021; Bunk et al., 2021; Chen et al., 2021a; Dong et al., 2021; Zi et al., 2021; Tack et al., 2021; Zhang et al., 2021a) arose in the last half year to tackle the overfitting, using data manipulation, smoothened training, and else. Those methods work orthogonally to our proposal as evidenced in Section 4.
Another group of related literature lies in the field of sparse robust networks (Guo et al., 2018b). These works either treat model compression as a defense mechanism (Wang et al., 2018; Gao et al., 2017; Dhillon et al., 2018b) or pursue robust and efficient sub-models that can be deployed in resource-limited platforms (Gui et al., 2019; Ye et al., 2019; Sehwag et al., 2019). Compared to those inference-focused methods, our goal is fundamentally different: injecting sparsity during training to reduce the robust generalization gap while improving training efficiency.
Static pruning and dynamic sparse training. Pruning (LeCun et al., 1990; Han et al., 2015a) serves as a powerful technique to eliminate the weight redundancy in over-parameterized DNNs, which aims to obtain storage and computational savings with almost undamaged performance. It can roughly divided into two categories based on how to generate sparse patterns: (i) static pruning. It removes parameters (Han et al., 2015a; LeCun et al., 1990; Han et al., 2015b) or substructures (Liu et al., 2017; Zhou et al., 2016; He et al., 2017) based on optimized importance scores (Zhang et al., 2018; He et al., 2017) or some heuristics like weight magnitude (Han et al., 2015a), gradient (Molchanov et al., 2019), hessian (LeCun et al., 1990) statistics. The discarded elements usually will not participate in the next round of training or pruning. Static pruning can be flexibly applied prior to training, such as SNIP (Lee et al., 2019), GraSP (Wang et al., 2020) and SynFlow (Tanaka et al., 2020); during training (Zhang et al., 2018; He et al., 2017); and post training (Han et al., 2015a) for different trade-off between training cost and pruned models’ quality. (ii) dynamic sparse training. It updates model parameters and sparse connectivities at the same time, starting from a randomly sparsified subnetwork (Molchanov et al., 2017). During the training, the removed elements have chances to be grown back if they potentially benefit to predictions. Among the huge family of sparse training (Mocanu et al., 2016; Evci et al., 2019; Mostafa & Wang, 2019; Liu et al., 2021a; Dettmers & Zettlemoyer, 2019; Jayakumar et al., 2021; Raihan & Aamodt, 2020), the recent methods Evci et al. (2020a); Liu et al. (2021b) lead to the state-of-the-art performance.
A special case of static pruning, Lottery tickets hypothesis (LTH) (Frankle & Carbin, 2019), demonstrates the existence of sparse subnetworks in DNNs, which are capable of training in isolation and reach a comparable performance of their dense counterpart. The LTH indicates the great potential to train a sparse network from scratch without sacrificing expressiveness and has recently drawn lots of attention from diverse fields (Chen et al., 2020b;a; 2021g;f;d;c;b; 2022; Ding et al., 2022; Gan et al., 2021) beyond image recognition (Zhang et al., 2021d; Frankle et al., 2020; Redman et al., 2021).

heading: 3 METHODOLOGY

heading: 3.1 PRELIMINARIES
text: Adversarial training (AT). As one of the widely adopted defense mechanisms, adversarial training (Madry et al., 2018b) effectively tackles the vulnerability to maliciously crafted adversarial samples. As formulated in Equation 1, AT (specifically PGD-AT) replaces the original empirical risk minimization into a min-max optimization problem:
3
min θ
E(x,y)∈DL ( f(x; θ), y ) =⇒ min
θ E(x,y)∈D max ‖δ‖p≤ L ( f(x+ δ; θ), y ) , (1)
where f(x; θ) is a network with parameters θ. Input data x and its associated label y from training set D are used to first generate adversarial perturbations δ and then minimize the empirical classification loss L. To meet the imperceptible requirement, the `p norm of δ is constrained by a small constant . Projected Gradient Descent (PGD), i.e., δt+1 = projP [δ t + α · sgn ( ∇xL(f(x + δt; θ), y) ) ], is usually utilized to produce the adversarial perturbations with step size α, which works in an iterative manner leveraging the local first order information about the network (Madry et al., 2018b).
Sparse subnetworks. Following the routine notations in Frankle & Carbin (2019), f(x;m θ) donates a sparse subnetwork with a binary pruning mask m ∈ {0, 1}‖θ‖0 , where is the elementwise product. Intuitively, it is a copy of dense network f(x; θ) with a portion of fixed zero weights.

heading: 3.2 ROBUST BIRD FOR ADVERSARIAL TRAINING
text: Introducing Robust Bird. The primary goal of Robust Bird is to find a high-quality sparse subnetwork efficiently. As shown in Figure 2, it locates subnetworks quickly by detecting critical network structures arising in the early training, which later can be robustified with much less computation.
Specifically, for each epoch t during training, Robust Bird creates a sparsity mask mt by “masking out” the p% lowest-magnitude weights; then, Robust Bird tracks the corresponding mask dynamics. The key observation behind Robust Bird is that the sparsity mask mt does not change drastically beyond the early epochs of training (You et al., 2020) because high-level network connectivity patterns are learned during the initial stages (Achille et al., 2019). This indicates that (i) winning tickets emerge at a very early training stage, and (ii) that they can be identified efficiently.
Robust Bird exploits this observation by comparing the Hamming distance between sparsity masks found in consecutive epochs. For each epoch, the last l sparsity masks are stored. If all the stored masks are sufficiently close to each other, then the sparsity masks are not changing drastically over time and network connectivity patterns have emerged; thus, a Robust Bird ticket (RB ticket) is drawn. A detailed algorithmic implementation is provided in Algorithm 1 of Appendix A1. This is the RB ticket used in the second stage of adversarial training.
4
5 0 5 10 15 20 25 30 35
1 s t PC: 3 7 .6 7 %
20
10
0
10
20
30
2n d
PC : 1
9. 56
%
0 .500
1 .000
1 .500
2 .000
2.500
3 .0 00
3 .000
3 .500
3 .50 0
4.000 4 .500
5 .0 00
5.500 6 .000
6 .5 0
7.000 7.500
8 .0 00.59 .000.5
10.000 100 .000
1 000 .000
1 0 000 .000
100000 .000
1 000000 .000 10000000 .000 100000000 .000
5 0 5 10 15 20 25 30 35
1 s t PC: 3 6 .1 6 %
20
10
0
10
20
30
2n d
PC : 1
9. 17
%
1 .000 1 .5 00
2 .000 2 .500
3.000
3 .5 00
3 .500 4 .000
4 . 00 0
4 .5 00
5 .0 00
5 . 50 0 6 . 00 0
6. 50 0
7 .0 007 .5 008 .00 0 8 .50 09 .000.5
10 .0 00
1 00 .000
1000.000
1 0 000 .000
100000 .000
1 0 00000 .000
10000000 .000
1 00000000 .000
5 0 5 10 15 20 25 30 35
1 s t PC: 3 6 .9 4 %
20
10
0
10
20
30
2n d
PC : 1
9. 72
%
1 .500
2 .000
2 .500
2 .500
2 .500
3 .000
3 .000
3 .500
3 .500
4 .000 4 .5 0 5 .000
5 .500 6 .0 6 .500 7 .0 7 .500 8 .0 8 .500 9 .000
9 .50
10 .000
100 .000
1000 .000
1 0000 .000
100000 .0
1000000 .000
10000000 .000
000 .000
D en
se
R an
do m
P ru
ni ng
Fl yi
ng B
ir d+
Figure 3: Visualization of loss contours and training trajectories. We compare the dense network, randomly pruned sparse networks, and flying bird+ at 90% sparsity from ResNet-18 robustified on CIFAR-10.
Rationale of Robust Bird. Recent studies (Zhang et al., 2021c) present theoretical analyses that identified sparse winning tickets enlarge the convex region near the good local minima, leading to improved generalization. Our work also shows a related investigation in Figure A9 that, compared with dense models and random pruned subnetworks, RB tickets found by the standard training have much flatter loss landscapes, serving a high-quality starting point for further robustification. This occurs because flatness of the loss surface is often believed to indicate the standard generalization. Similarly, as advocated by Wu et al. (2020a); Hein & Andriushchenko (2017), a flatter adversarial loss landscape also effectively shrinks the robustness generalization gap. This “flatness preference” of adversarial robustness has been revealed by numerous empirical defense mechanisms, including Hessian/curvature-based regularization (Moosavi-Dezfooli et al., 2019), learned weight and logits smoothening (Chen et al., 2021e), gradient magnitude penalty (Wang & Zhang, 2019), smoothening with random noise (Liu et al., 2018), or entropy regularization (Jagatap et al., 2020).
These observations make the main cornerstone for our proposal and provide possible interpretations to the surprising finding that the RB tickets pruned from a non-robust model can be used for obtaining well-generalizable robust models in the followed robustification. Furthermore, unlike previous costly flatness regularizers (Moosavi-Dezfooli et al., 2019), our methods not only offer a flatter starting point but also obtain substantial computational savings due to the reduced model size.

heading: 3.3 FLYING BIRD FOR ADVERSARIAL TRAINING
text: Introducing Flying Bird(+). Since sparse subnetworks from static pruning are unable to regret for removed elements, they may be too aggressive to capture the pivotal structural patterns. Thus, we introduce Flying Bird (FB) to conduct a thorough exploration of dynamic sparsity, which allows pruned parameters to be grown back and engages in the next round of training or pruning, as demonstrated in Figure 2. Specifically, it starts from a sparse subnetwork f(x;m θ) with a random binary mask m, and then jointly optimize model parameters and sparse connectivities simultaneously. In other words, the subnetwork’s typologies are “on the fly”, decided dynamically based on current training status. Specifically, we update Flying Bird’s sparse connectivity every ∆t epochs of adversarial training, which consists of two continually applied operations: pruning and growing. For the pruning step, p% of model weights with the lowest magnitude will be eliminated, while g% weights with the largest gradient will be added back in the growth step. Note that newly added connections are not activated in the last sparse topology, and are initialized to zero since it establishes better performance as indicated in (Evci et al., 2020a; Liu et al., 2021b). Flying Bird maintains the sparsity ratio unchanged during the full training by keeping both pruning and growing ratio p%, g% equal k% that decays with a cosine annealing schedule.
We further propose Flying Bird+, an enhanced variant of FB, capable of adaptively adjusting the sparsity and learning the right parameterization level ”on demand” during training, as shown in Figure 2. To be specific, we first record the robust generalization gap and robust validation loss at each training epoch. An increasing generalization gap of the later training stage indicates a risk of overfitting, while a plateau validation loss implies underfitting. Hence, we then analyze the fitting status according to the upward/downward trend of those measurements. If most epochs (e.g., more than 3 out of the past 5 epochs in our case) tend to see enlarged robust generalization gaps, we raise the pruning ratio p% to further trim down the network capacity. Similarly, if the majority of epochs present unchanged validation loss, we will increase the growing ratio q% to enrich the subnetwork capacity. Detailed procedures are summarized in Algorithm 2 of Appendix A1.
Rationale of Flying Bird(+). As demonstrated in Evci et al. (2020a), allowing new connections to grow yields improved flexibility in navigating the loss surfaces, which creates the opportunity to
5
escape bad local minima and search for the optimal sparse connectivity Liu et al. (2021b). Flying Bird follows a similar design philosophy that excludes least important connections (Han et al., 2015a) while activating new connections with the highest potential to decrease the training loss fastest. Recent works (Wu et al., 2020c; Liu et al., 2019) have also found enabling network (re)growth can turn a poor local minima into a saddle point that facilitates further loss decrease. Flying Bird+ empowers the flexibility further by adaptive sparsity level control.
The flatness of loss geometry provides another view to dissect the robust generalization gain (Chen et al., 2021e; Stutz et al., 2021; Singla et al., 2021). Figure 3 compares the loss landscapes and training trajectories of dense, randomly pruned subnetworks, and Flying Brid+ robustified on CIFAR-10. We observe that Flying Bird+ converges to a wider loss valley with improved flatness, which usually suggests superior robust generalization (Wu et al., 2020a; Hein & Andriushchenko, 2017). Last but not the least, our approaches also significantly trim down both the training memory overhead and the computational complexity, enjoying extra bonus of efficient training and inference.

heading: 4 EXPERIMENT RESULTS
text: Datasets and architectures. Our experiments consider two popular architectures, ResNet-18 (He et al., 2016), VGG-16 (Simonyan & Zisserman, 2014) on three representative datasets, CIFAR-10, CIFAR-100 (Krizhevsky & Hinton, 2009) and Tiny-ImageNet (Deng et al., 2009). We randomly split one-tenth of the training samples as the validation dataset, and the performance is reported on the official testing dataset.
Training and evaluation details. We implement our experiments with the original PGD-based adversarial trainig (Madry et al., 2018b), in which we train the network against `∞ adversary with maximum perturbations of 8/255. 10-steps PGD for training and 20-steps PGD for evaluation are chosen with a step size α of 2/255, following Madry et al. (2018b); Chen et al. (2021e). In addition, we also use Auto-Attack (Croce & Hein, 2020) and CW Attack (Carlini & Wagner, 2017) for a more rigorous evaluation. More details are provided in Appendix A2. For each experiment, we train the network for 200 epochs with an SGD optimizer, whose momentum and weight decay are kept to 0.9 and 5× 10−4, respectively. The learning rate starts from 0.1 that decays by 10 times at 100,150 epoch and the batch size is 128, which follows Rice et al. (2020).
For Robust Bird, the threshold τ of mask distance is set as 0.1. In Flying Birds(+), we calculate the layer-wise sparsity by Ideal Gas Quotas (IGQ) (Vysogorets & Kempe, 2021) and then apply random pruning to initialize the sparse masks. FB updates the sparse connectivity per 2000 iterations of AT, with an update ratio k that starts from 50% and decays by cosine annealing. More details are referred to Appendix A2. Hyperparameters are either tuned by grid search or following Liu et al. (2021b).
Evaluation metrics. In general, we care about both the accuracy and efficiency of obtained sparse networks. To assess the accuracy, we consider both Robust Testing Accuracy (RA) and Standard Testing Accuracy (SA) which are computed on the perturbed and the original test sets, together with Robust Generalization Gap (RGG) (i.e., the gap of RA between train and test sets). Meantime, we report the floating point operations (FLOPs) of the whole training process and single image inference to measure the efficiency.

heading: 4.1 ROBUST BIRD IS A GOOD BIRD
text: In this section, we evaluate the effectiveness of static sparsity from diverse representative pruning approaches, including: (i) Random Pruning (RP), by randomly eliminating model parameters to the desired sparsity; (ii) One-shot Magnitude Pruning (OMP), which globally removes a certain ratio of lowest-magnitude weights; (iii) Pruning at Initialization algorithms. Three advanced methods, i.e., SNIP (Lee et al., 2019), GraSP (Wang et al., 2020) and SynFlow (Tanaka et al., 2020), are considered, which identify the subnetworks at initialization respect to certain criterion of gradient flow. (iv) Ideal Gas Quotas (IGS) (Vysogorets & Kempe, 2021). It adopts random pruning based on pre-calculated layer-wise sparsity which draws intuitive analogies from physics. (v) Robust Bird (RB), which can be regarded as an early stopped OMP. (vi) Small Dense. It is an important sanity check via considering smaller dense networks with the same parameter counts as the ones of sparse networks. Comprehensive results of these subnetworks at 80% and 90% sparsity are reported in Table 1, where the chosen sparsity follows routine options (Evci et al., 2020a; Liu et al., 2021b).
6
As shown in Table 1, we first observe the occurrence of poor robust generalization with 38.82% RA gap and robust overfitting with 7.49% RA degradation, when training the dense network (Baseline). Fortunately, coincided with our claims, injecting appropriate sparsity effectively tackle the issue. For instance, RB greatly shrinks the RGG by 15.45%/22.20% at 80/90% sparsity, while also mitigates robust overfitting by 2.53% ∼ 4.08%. Furthermore, comparing all static pruning methods, we find that (1) Small Dense and RP behave the worst, which suggests the identified sparse typologies play important roles rather than reduced network capacity only; (2) RB shows clear advantages to OMP in terms of all measurements, especially for 78.32% ∼ 84.80% training FLOPs savings. It validates our RB proposal that a few epochs of standard training are enough to learn a high-quality sparse structure for further robustification, and thus there is no need to complete the full training in the tickets finding stage like traditional OMP. (3) SynFlow and IGQ approaches have the best RA and SA, while RB obtains the superior robust generalization among static pruning approaches.
Finally, we explore the influence of training regimes during the RB ticket finding on CIFAR-100 with ResNet-18. Table A6 demonstrates that RB tickets perform best when found with the cheapest standard training. Specifically, at 90% and 95% sparsity, SGD RB tickets outperform both Fast AT (Wong et al., 2020) and PGD-10 RB tickets with up to 1.27% higher RA and 1.86% narrower RGG. Figure A7 offers a possible explanation for this phenomenon: the SGD training scheme more quickly develops high-level network connections, during the early epochs of training (Achille et al., 2019). As a result, RB Tickets pruned from the model trained with SGD achieve superior quality.

heading: 4.2 FLYING BIRD IS A BETTER BIRD
text: In this section, we discuss the advantages of dynamic sparsity and show that our Flying Bird(+) is a superior bird. Table 1 examines the effectiveness of FB(+) on CIFAR-10 with ResNet-18, and several consistent observations can be drawn: ¶ FB(+) achieve 9.92% ∼ 23.66% RGG reduction, 2.24% ∼ 5.88% decrease for robust overfitting, compared with the dense network. And FB+ at 80% sparsity even pushes the RA 0.60% higher. · Although the smaller dense network shows the leading performance w.r.t improving robust generalization, the robustness has been largely sacrificed, with up to 4.29% RA degradation, suggesting that only reducing models’ parameter counts is insufficient to keep satisfactory SA/RA. ¸ FB and FB+ achieve superior performance of RA for both the best and final checkpoints across all methods, including RB. ¹ Regardless of small dense and random pruning due to their poor robustness, FB+ reaches the most impressive robust generalization (rank #1 or #2) with the least training and inference costs. Precisely, FB+ obtains 84.46% ∼ 91.37% training FLOPs and 84.46% ∼ 93.36% inference FLOPs saving, i.e., Flying Bird+ is SUPER light-weight.
7
Superior performance across datasets and architectures. We further evaluate the performance of FB(+) across various datasets (CIFAR-10, CIFAR-100 and Tiny-ImageNet) and architectures (ResNet-18 and VGG-16). Table 2 and 3 display that both static and dynamic sparsity of our proposals serve effective remedies for improving robust generalization and mitigating robust overfitting, with 4.43% ∼ 15.45%, 14.99% ∼ 34.44% and 21.62% ∼ 23.60% RGG reduction across different architectures on CIFAR-10, CIFAR-100 and Tiny-ImageNet, respectively. Moveover, both RB and FB(+) gain significant efficiency, with up to 87.83% training and inference FLOPs savings.
Superior performance across improved attacks. Additionally, we verify both RB and FB(+) under improved attacks, i.e., Auto-Attack (Croce & Hein, 2020) and CW Attack (Carlini & Wagner, 2017). As shown in Table A8, our approaches shrink the robust generalization gap by up to 30.76% on CIFAR-10/100, and largely mitigate robust overfitting. This piece of evidence shows our proposal’s effectiveness sustained across diverse attacks.
Combining FB+ with existing start-of-the-art (SOTA) mitigation. Previous works (Chen et al., 2021e; Zhang et al., 2021a; Wu et al., 2020b) point out that smoothening regularizations (e.g., KD (Hinton et al., 2015) and SWA (Izmailov et al., 2018)) help robust generalization and lead to SOTA robust accuracies. We combine them with our FB+ and collect the robust accuracy on CIFAR-10 with ResNet-18 in Figure 4. The extra robustness gains from FB+ imply that they makes complementary contributions.
Excluding obfuscated gradients. A common “counterfeit” of robustness improvements is less effective adversarial examples resulted from obfuscated gradients (Athalye et al., 2018). Table A7 demonstrates the maintained enhanced robustness under unseen transfer attacks, which excludes the possibility of gradient masking. More are referred to Section A3.

heading: 4.3 ABLATION STUDY AND VISUALIZATION
text: Different sparse initialization and update frequency. As two major components in the dynamic sparsity exploration (Evci et al., 2020a), we conduct thorough ablation studies in Table 4 and 5. We found the performance of Flying Bird+ is more sensitive to different sparse initialization; using SNIP to produce initial layer-wise sparsity and updating the connections per 2000 iterations serves the superior configuration for FB+.
8
Table 4: Ablation of different sparse initialization in Flying Bird+. Subnetwroks at 80% initial sparsity are chosen on CIFAR-10 with ResNet-18.
Table 5: Ablation of different update frequency in Flying Bird+. Subnetworks at 80% initial sparsity are chosen on CIFAR-10 with ResNet-18.
Final checkpoint loss landscapes. From visualizations in Figure 5, FB and FB+ converge to much flatter loss valleys, which evidences their effectiveness in closing robust generalization gaps.
Attention and saliency maps. To visually inspect the benefits of our proposal, here we provide attention and saliency maps generated by Grad-GAM (Selvaraju et al., 2017) and tools in (Smilkov et al., 2017). Comparing the dense model to our “talented birds” (e.g., FB+), Figure 6 shows that our approaches have enhanced concentration on main objects, and are capable of capturing more local feature information, aligning better with human perception.
1
Dense

heading: Adversarial Samples
text: Random Pruning
SNIP
Flying Bird
Robust Bird
Flying Bird+
+
Heatmap Saliency Map
Figure 6: (Left) Visualization of attention heatmaps on adversarial images based on Grad-Cam (Selvaraju et al., 2017). (Right) Saliency map visualization on adversarial samples (Smilkov et al., 2017).

heading: 5 CONCLUSION
text: We show the adversarial training of dense DNNs incurs a severe robust generalization gap, which can be effectively and efficiently resolved by injecting appropriate sparsity. Our proposed Robust Bird and Flying Bird(+) with static and dynamic sparsity, significantly mitigate the robust generalization gap while retaining competitive standard/robust accuracy, besides substantially reduced computation. Our future works plan to investigate channel- and block-wise sparse structures.
9

heading: A1 MORE TECHNIQUE DETAILS
text: Algorithms of Robust Bird and Flying Bird(+). Here we present the detailed procedure to identify robust bird and flying bird(+), as summarized in algorithm 1 and 2. Note that for the increasing frequency on Line 10 and 11 in algorithm 2, we compare the measurements stored in the queue between two consequent epochs and calculate the frequency of increasing.
Algorithm 1: Finding a Robust Bird Input: f(x; θ0) w. initialization θ0, target sparsity s%, FIFO queue Q with length l, threshold τ Output: Robust bird f(x;mt∗ θT)
1 while t < tmax do 2 Update network parameters θt ← θt−1 via standard training 3 Apply static pruning towards target sparsity s% and obtain the sparse mask mt 4 Calculate the Hamming distance δH(mt,mt−1), append result to Q 5 t← t+ 1 6 if max(Q) < τ then 7 t∗ ← t 8 Rewind f(x;mt∗ θt∗)→ f(x;mt∗ θ0) 9 Training f(x;mt∗ θ0) via PGD-AT for T epochs
10 return f(x;mt∗ θT) 11 end 12 end
Algorithm 2: Finding a Flying Bird(+) Input: Initialization parameters θ0, sparse masks m of sparsity s%, FIFO queue Qp andQg
with length l, pruning and growth increasing ratio δp and δg , update threshold , optimize interval ∆t, parameter update ratio k%, ratio update starting point tstart
Output: Flying bird(+) f(x;m θT) 1 while t < T do 2 Update network parameters θt ← θt−1 via PGD-AT; 3 # Record training statistics 4 Add robust generalization gap between train and validation set to Qp 5 Add robust validation loss to Qg 6 # Update sparse masks m 7 if (t mod ∆t) == 0 then 8 |---Optional for Flying Bird+---| 9 # Update pruning and growth ratio p%, g%
10 if t > tstart and increasing frequency of Qp ≥ : p = (1 + δp)× k else p = k 11 if t > tstart and increasing frequency of Qg ≥ : g = (1 + δg)× k else g = k 12 |---Optional for Flying Bird+---| 13 Prune p% parameters with smallest weight magnitude 14 Grow g% parameters with largest gradient 15 Update sparse mask m accordingly 16 end 17 end

heading: A2 MORE IMPLEMENTATION DETAILS

heading: A2.1 OTHER COMMON DETAILS
text: We select two checkpoints during training: best, which has the best RA values on the validation set, and final, i.e., the last checkpoint. And we report both RA and SA of these two checkpoints on test sets. Apart from the robust generalization gap, we also show the extent of robust overfitting numerically by the difference of RA between best and final. Furthermore, we calculate the FLOPs
A17
at both training and inference stages to evaluate the prices of obtaining and exploiting the subnetworks respectively, in which we approximate the FLOPs of the back-propagation to be twice that of forwarding propagation (Yang et al., 2020).

heading: A2.2 MORE DETAILS ABOUT ROBUST BIRD
text: For the experiments of RB tickets finding, we comprehensively study three training regimes: standard training with stochastic gradient descent (SGD), adversarial training with PGD-10 AT (Madry et al., 2018b), and Fast AT (Wong et al., 2020). Following Pang et al. (2021), we train the network with an SGD optimizer of 0.9 momentum and 5 × 10−4 weight decay. We use a batch size of 128. For the experiments of PGD-10 AT, we adopt the `∞ PGD attack with a maximum perturbation = 8/255 and a step size α = 2/255. And the learning rate starts from 0.1, then decays by ten times at 50, 150 epoch. As for fast AT, we use a cyclic schedule with a maximum learning rate equals 0.2.

heading: A2.3 MORE DETAILS ABOUT FLYING BIRD(+)
text: For the experiments of Flying Bird+, the increasing ratio of pruning and growth δp, δq is kept default to 0.4% and 0.05%, respectively.

heading: A3 MORE EXPERIMENT RESULTS

heading: A3.1 MORE RESULTS ABOUT ROBUST BIRD
text: Accuracy during RB Tickets Finding Figure A7 shows the curve of standard test accuracy during the training phase of RB ticket finding. We can observe the SGD training scheme develops highlevel network connections much faster than the others, which provides a possible explanation for the superior quality of RB tickets from SGD.
0 5 10 15 20 25 30 Epoch
30 40 50 60 70 80 St an da rd A cc ur ac y %
RB Ticket Finding Performance
PGD-10 SGD FAST AT
Figure A7: Standard accuracy (SA) of PGD-10, SGD, and Fast AT during the RB ticket finding phase.
Mask Similarity Visualization. Figure A8 visualizes the dynamic similarity scores for each epoch among masks found via SGD, Fast AT, and PGD-10. Specifically, the similarity scores (You et al., 2020) reflect the Hamming distance between a pair of masks. We notice that masks found by SGD and PGD-10 share more common structures. A possible reason is that Fast AT usually adopts a cyclic learning rate schedule, while SGD and PGD use a multi-step decay schedule.
Different training regimes for finding RB tickets. We denote the subnetworks identified by standard training with SGD, adversarial training with Fast AT (Wong et al., 2020) and adversarial train-
A18
0 20 40 60 80 100
Fast AT Mask
0 20 40 60 80
100PG D-
10 M
as k
0 20 40 60 80 100
Fast AT Mask
0 20 40 60 80
100
SG D
M as
k
0 20 40 60 80 100
PGD-10 Mask
0 20 40 60 80
100
SG D
M as
k
0.50 0.55 0.60 0.65 0.70 0.75
Figure A8: Similarity scores by epoch among masks found via Fast AT, SGD, and PGD-10. A brighter color denotes higher similarity.
Table A6: Comparison results of different training regimes for RB ticket finding on CIFAR-100 with ResNet-18. The subnetworks at 90% and 95% are selected here.
Sparsity(%) Settings Roubst Accuarcy Standard Accuarcy Robust
Best Final Diff. Best Final Diff. Generalization
0 Baseline 26.93 19.62 7.31 52.03 53.91 −1.88 54.56
90
SGD tickets 25.83 23.40 2.43 49.35 53.51 −4.16 18.37↓ 36.19 Fast AT tickets 25.15 22.88 2.27 51.00 51.75 −0.75 20.23↓ 34.33 PGD-10 tickets 25.34 22.96 2.38 52.01 53.27 −1.26 20.03↓ 34.53
95
SGD tickets 24.77 24.12 0.65 49.88 50.89 −1.01 9.18↓ 45.38 Fast AT tickets 23.50 22.46 1.04 41.67 43.19 −1.52 9.53↓ 45.03 PGD-10 tickets 24.44 23.77 0.67 49.30 50.65 −1.35 9.86↓ 44.70
ing with PGD-10 AT as SGD tickets, Fast AT tickets, and PGD-10 tickets, respectively. Table A6 demonstrate the SGD tickets has the best performance.
Loss Landscape Visualization We visualize the loss landscape of the dense network, random pruned subnetwork, and robust bird tickets at 30% sparsity in Figure A9. Compared with the dense model and random pruned subnetwork, RB tickets found by the standard training shows much flatter loss landscapes, which provide a high-quality starting point for further robustification.

heading: A3.2 MORE RESULTS ABOUT FLYING BIRD(+)
text: Excluding Obfuscated Gradients. To exclude this possibility of gradient masking, we show that our methods maintain improved robustness under unseen transfer attacks. As shown in Table A7, the left part represents the testing accuracy of perturbed test samples from an unseen robust model, and the right part shows the transfer testing performance on an unseen robust model (here we use a separately robustified ResNet-50 with PGD-10 on CIFAR-100).
Performance under Improved Attacks. We report the performance of both RB and FB(+) under Auto-Attack (Croce & Hein, 2020) and CW Attack (Carlini & Wagner, 2017). For Auto-Attack, we keep the default setting with = 8255 . And for CW Attack we perform 1 search step on C with an initial constant of 0.1. And we use 100 iterations for each search step with the learning rate of 0.01. As shown in Table A8, both RB and FB(+) outperform the dense counterpart in terms of robust generalization. And FB+ achieves superior performance.
More Datasets and Architectures We report more results of different sparsification methods across diverse datasets and architectures at Table A9, A10, A11 and A12, from which we observe our approaches are capable of improving robust generalization and mitigating robust overfitting.
A19
Dense Models RB Tickets (30%) Random Pruning (30%)
Figure A9: Loss landscapes visualizations (Engstrom et al., 2018; Chen et al., 2021e) of the dense model (unpruned), random pruned subnetwork at 30% sparsity, and Robust Bird (RB) tickets at 30% sparsity found by the standard training. The ResNet-18 backbone with the same original initialization on CIFAR-10 is adopted here. Results demonstrate that RB tickets offer a smoother and flatter starting point for further robustification in the second stage.
Table A7: Transfer attack performance from/on an unseen non-robust model, where the attacks are generated by/applied to the non-robust model. The robust generalization gap is also calculated based on transfer attack accuracies between train and test sets. We use ResNet-18 on CIFAR-10/100 and sub-networks at 80% sparsity.
Dataset Settings Transfer Attack from Unseen Model Transfer Attack on Unseen Model
Accuracy Robust Accuracy Robust
Best Final Diff. Generalization Best Final Diff. Generalization
CIFAR-10 Baseline 79.68 82.03 −2.35 16.43 70.48 79.85 −9.37 11.84 Robust Bird 77.33 81.04 −3.71 12.18 73.17 77.03 −3.86 11.49 Flying Bird 79.13 82.17 −3.04 13.49 71.59 77.19 −5.60 11.88 Flying Bird+ 79.47 81.90 −2.43 11.85 70.43 76.00 −5.57 11.42
CIFAR-100 Baseline 50.51 52.15 −1.64 45.91 48.67 54.48 −5.81 36.98 Robust Bird 47.25 51.74 −4.49 28.80 47.47 50.90 −3.43 35.82 Flying Bird 51.80 53.52 −1.72 31.98 45.56 50.61 −5.05 35.39 Flying Bird+ 50.72 53.56 −2.84 25.09 47.04 49.43 −2.39 35.09
Distributions of Adopted Sparse Initialization. We report the layer-wise sparsity of different initial sparse masks. As shown in Figure A10, we observe that subnetworks generally have better performance when the top layers remain most of the parameters.
Training Curve of Flying Bird+. Figure A11 shows the training curve of Flying Bird+, in which the red dotted lines represent the time for increasing the pruning ratio and the green dotted lines for growth ratio. The detailed training curve demonstrates the flexibility of flying bird+ for dynamically adjusting the sparsity levels.

heading: A4 EXTRA RESULTS AND DISCUSSION
text: We sincerely appreciate all anonymous reviewers’ and area chairs’ constructive discussions for improving this paper. Extra results and discussions are presented in this section.
A20
Table A8: Evaluation under improved attacks (i.e., Auto-Attack and CW-Attack) on CIFAR-10/100 with ResNet-18 at 80% sparsity. The robust generalization gap is computed under improved attacks.
Dataset Settings Auto-Attack CW-Attack
Accuracy Robust Accuracy Robust
Best Final Diff. Generalization Best Final Diff. Generalization
CIFAR-10 Baseline 47.41 41.59 5.82 35.30 75.76 66.13 9.63 30.39 Robust Bird 45.90 42.45 3.45 21.58 ↓ 13.72 73.95 73.52 0.43 17.67 ↓ 12.72 Flying Bird 47.55 43.57 3.98 26.55 ↓ 8.75 75.30 72.08 3.22 21.77 ↓ 8.62 Flying Bird+ 47.06 44.09 3.17 21.73 ↓ 13.57 76.00 73.83 2.17 17.77 ↓ 12.62
CIFAR-100 Baseline 23.16 17.68 5.48 49.73 45.83 36.21 9.62 57.52 Robust Bird 21.29 18.00 3.29 21.72 ↓ 28.01 43.30 42.39 0.91 30.82 ↓ 26.70 Flying Bird 22.74 19.44 3.30 25.18 ↓ 24.55 46.23 42.36 3.87 35.50 ↓ 22.02 Flying Bird+ 22.90 20.31 2.59 19.05 ↓ 30.68 45.86 43.90 1.96 26.76 ↓ 30.76
Table A9: More results of different sparcification methods on CIFAR-10 with ResNet-18.
Sparsity(%) Settings Robust Accuracy Standard Accuracy Robust
Best Final Diff. Best Final Diff. Generalization
0 Baseline 51.10 43.61 7.49 81.15 83.38 −2.23 38.82
95
Small Dense 45.99 44.55 1.44 74.26 75.64 −1.38 7.87 ↓ 30.95 Random Pruning 45.64 44.18 1.46 75.20 75.20 0.00 7.96 ↓ 30.86 OMP 47.08 46.23 0.85 78.77 79.36 −0.59 12.01 ↓ 26.81 SNIP 48.18 46.72 1.46 78.55 79.21 −0.66 9.58 ↓ 29.24 GraSP 48.58 47.15 1.43 78.95 79.44 −0.49 10.37 ↓ 28.45 SynFlow 48.93 48.22 0.71 78.70 78.90 −0.20 8.25 ↓ 30.57 IGQ 48.82 47.56 1.26 79.44 79.76 −0.32 9.33 ↓ 29.49 Robust Bird 47.53 46.48 1.05 78.33 78.78 −0.45 9.20 ↓ 29.62 Flying Bird 49.62 48.46 1.16 78.12 81.43 −3.31 13.32 ↓ 25.52 Flying Bird+ 49.37 48.84 0.53 80.33 80.28 0.05 9.27 ↓ 29.55
co nv
1
la ye
r1 .0
.c on
v1
la ye
r1 .0
.c on
v2
la ye
r1 .1
.c on
v1
la ye
r1 .1
.c on
v2
la ye
r2 .0
.c on
v1
la ye
r2 .0
.c on
v2
la ye
r2 .0
.sh or
tc ut
.0
la ye
r2 .1
.c on
v1
la ye
r2 .1
.c on
v2
la ye
r3 .0
.c on
v1
la ye
r3 .0
.c on
v2
la ye
r3 .0
.sh or
tc ut
.0
la ye
r3 .1
.c on
v1
la ye
r3 .1
.c on
v2
la ye
r4 .0
.c on
v1
la ye
r4 .0
.c on
v2
la ye
r4 .0
.sh or
tc ut
.0
la ye
r4 .1
.c on
v1
la ye
r4 .1
.c on
v2
lin ea
r
Layer Name
0.0
0.2
0.4
0.6
0.8
1.0
Sp ar
si ty
Uniform GraSP SNIP SynFlow IGQ ERK
Figure A10: Layer-wise sparisty of different initial sparse masks with ResNet-18

heading: A4.1 MORE RESULTS OF DIFFERENT SPARSITY
text: We report more results of subnetworks with 40/60% sparsity on CIFAR-10/100 with ResNet-18 and VGG-16. As shown in Table A13, A14, A15 and A16, our flying bird(+) achieves consistent improvement than baseline unpruned networks, in terms of 2.45 ∼ 19.81% narrower robust generalization gaps with comparable RA and SA performance.

heading: A4.2 MORE RESULTS ON WIDERESNET
text: We further evaluate our flying bird(+) with WideResNet-34-10 on CIFAR-10 and report the results on Table A17. We can observe that compared with the dense network, our methods significantly shrink the robust generalization gap by up to 13.14% and maintain comparable RA/SA performance.
A21
Table A10: More results of different sparcification methods on CIFAR-10 with VGG-16.
Sparsity(%) Settings Robust Accuracy Standard Accuracy Robust
Best Final Diff. Best Final Diff. Generalization
0 Baseline 48.33 42.73 5.60 76.84 79.73 −2.89 28.00
80
Random Pruning 46.14 40.33 5.81 74.42 76.68 −2.26 21.01 ↓ 6.99 OMP 47.90 43.19 4.71 76.60 80.02 −3.42 24.97 ↓ 3.03 SNIP 48.03 43.17 4.86 76.68 80.08 −3.40 24.71 ↓ 3.29 GraSP 47.91 42.34 5.57 75.74 78.87 −3.13 23.65 ↓ 4.35 SynFlow 48.47 45.32 3.15 77.62 79.09 −1.47 20.17 ↓ 7.83 IGQ 48.57 44.25 4.32 77.51 80.01 −2.50 22.79 ↓ 5.21 Robust Bird 47.69 41.66 6.03 75.32 78.58 −3.26 23.57 ↓ 4.43 Flying Bird 48.43 44.65 3.78 77.53 79.72 −2.19 21.01 ↓ 6.99 Flying Bird+ 48.25 45.24 3.01 77.48 79.55 −2.07 17.75 ↓ 10.25
90
Random Pruning 44.33 40.33 4.00 71.27 74.46 −3.19 15.48 ↓ 12.52 OMP 47.84 43.34 4.50 75.60 79.10 −3.50 18.29 ↓ 9.71 SNIP 47.76 44.27 3.49 75.92 79.62 −3.70 17.85 ↓ 10.15 GraSP 45.96 42.12 3.84 75.19 77.03 −1.84 15.04 ↓ 12.96 SynFlow 47.54 45.79 1.75 78.43 78.70 −0.27 14.40 ↓ 13.60 IGQ 47.79 45.12 2.67 74.87 79.19 −4.32 16.06 ↓ 11.94 Robust Bird 47.09 44.13 2.96 75.53 78.36 −2.83 16.57 ↓ 11.43 Flying Bird 48.45 45.55 2.90 75.82 79.21 −3.39 16.56 ↓ 11.44 Flying Bird+ 48.39 46.26 2.13 78.73 79.12 −0.39 12.47 ↓ 15.53

heading: A4.3 COMPARISON WITH EFFICIENT ADVERSARIAL TRAINING METHODS
text: To elaborate more about training efficiency, we compare our methods with two efficient training methods. Shafahi et al. (2019) proposed Free Adversarial Training that improves training efficiency by reusing the gradient information, which is orthogonal to our approaches and can be easily combined with our methods to pursue more efficiency by replacing the PGD-10 training with Free AT.
A22
Sparsity: 80% Sparsity: 90%
Additionally, Li et al. (2020) uses magnitude pruning to locate sparse structures, which is similar to OMP reported in Table 1, except they use a smaller learning rate. Our methods achieve better performance and efficiency than OMP. Specifically, with 80% sparsity, our flying bird+ reaches a 4.49% narrower robust generalization gap and 1.54% higher RA yet only requires 87.58% less training FLOPs. Also, our methods can be easily combined with Fast AT for further training efficiency.
A23

heading: A4.4 COMPARISON WITH OTHER PRUNING AND SPARSE TRAINING METHODS
text: Compared with the recent work (Özdenizci & Legenstein, 2021), our flying bird(+) is different at both levels of goal and methodologies. Firstly, Özdenizci & Legenstein (2021) pursues a superior adversarial robust testing accuracy for sparsely connected networks. While we aim to investigate the relationship between sparsity and robust generalization, and demonstrate that introducing appropriate sparsity (e.g., LTH-based static sparsity or dynamic sparsity) into adversarial training
A24
substantially alleviates the robust generalization gap and maintains comparable or even better standard/robust accuracies. Secondly, Özdenizci & Legenstein (2021) samples network connectivity from a learned posterior to form a sparse subnetwork. However, our flying bird first removes the parameters with the lowest magnitude, which ensures a small term of the first-order Taylor approximation of the loss and thus limits the impact on the output of networks (Evci et al., 2020a). And then, it allows new connectivity with the largest gradient to grow to reduce the loss quickly (Evci et al., 2020a). Furthermore, we propose an enhanced variant of Flying Bird, i.e., Flying Bird+, which not only learns the sparse topologies but also is capable of adaptively adjusting the network capacity to determine the right parameterization level “on-demand” during training, while Özdenizci & Legenstein (2021) stick to a fixed parameter budget.
Another work, HYDRA (Sehwag et al., 2020) also has several differences from our robust birds. Specifically, HYDRA starts from a robust pre-trained dense network, which requires at least hundreds of epochs for adversarial training. However, our robust bird’s pre-training only needs a few epochs of standard training. Therefore, Sehwag et al. (2020) has significantly higher computational costs, compared to ours. Then, Sehwag et al. (2020) adopt TRADES (Zhang et al., 2019) for adversarial training, which also requires auxiliary inputs of clean images, while our methods follow the classical adversarial training (Madry et al., 2018b) and only take adversarial perturbed samples as input. Moreover, for CIFAR-10 experiments, Sehwag et al. (2020) uses 500k additional pseudolabeled images from the Tiny-ImageNet dataset with a robust semi-supervised training approach. However, all our methods and experiments do not leverage any external data.
Furthermore, one concurrent work (Fu et al., 2021) demonstrates that there exist subnetworks with inborn robustness. Such randomly initialized networks have matching or even superior robust accuracy of adversarially trained networks with similar parameter counts. It’s interesting to utilize this finding for further improvement of robust generalization, and we will investigate it in future works.
A25

Review summaries:
Recent studies demonstrate adversarial training suffers from severe overfitting besides getting very expensive. This paper proposes to handle the two problems organically altogether, with the tool of sparse training.   The authors show that injecting appropriate sparsity forms in training could substantially shrink the robust generalization gap and alleviate the robust overfitting, meanwhile significantly saving training and inference FLOPs. 

This paper deals with the problem of training a neural network so that it generalizes well over data unseen at training time. Namely, they address the particular case where a network is trained over an adversarial scheme. This paper proposes two methods for learning a sparse architecture called robust and flying bird. These methods aim at identifying sparse subnetworks arising during early training stages, so to get a pruning mask that eventually yields a sparse architecture (RobustBird). FlyingBird improves over Robust Bird in teh sense that the learning mask can be dynamically adjusted over time, i.e. pruned params may be recovered later on. The authors then experiment at training multiple architectures over different datasets in the experimental section, showing better generalization abilities and lower computational complexity (MACs) for their proposed methods Robust and Flying Birds. the authors conclude that sparsity help networks to generalize better, and as a byproduct it slashes computational complexity.