S PARSITY W INNING T WICE : B ETTER R OBUST G ENERALIZATION FROM M ORE E FFICIENT T RAINING
Anonymous authors
Paper under double-blind review
A BSTRACT
Recent studies demonstrate the deep networks, even robustiﬁed by the state-ofthe-art adversarial training (AT), still suffer from large robust generalization gaps, in addition to the much more expensive training costs than standard training.
In this paper, we investigate this intriguing problem from a new perspective, i.e., injecting appropriate forms of sparsity during adversarial training.
We introduce two alternatives for sparse adversarial training: (i) static sparsity , by leveraging recent results from the lottery ticket hypothesis to identify critical sparse subnetworks arising from the early training; (ii) dynamic sparsity , by allowing the sparse subnetwork to adaptively adjust its connectivity pattern (while sticking to the same sparsity ratio) throughout training.
We ﬁnd both static and dynamic sparse methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overﬁtting, meanwhile signiﬁcantly saving training and inference FLOPs.
Extensive experiments validate our proposals with multiple network architectures on diverse datasets, including CIFAR-10/100 and TinyImageNet.
For example, our methods reduce robust generalization gap and overﬁtting by 34 .
44% and 4 .
02% , with comparable robust/standard accuracy boosts and 87 .
82% training/inference FLOPs savings on CIFAR-100 with ResNet18.
Besides, our approaches can be organically combined with existing regularizers, establishing new state-of-the-art results in AT.
All codes are included.
I NTRODUCTION
robust generalization gap between training and testing accuracy is substantially narrowed.
Deep neural networks (DNNs) are notoriously vulnerable to maliciously crafted adversarial attacks.
To conquer this fragility, numerous adversarial defense mechanisms are proposed to establish robust neural networks (Schmidt et al., 2018;
Sun et al., 2019; Nakkiran, 2019; Stutz et al., 2019; Raghunathan et al., 2019; Hu et al., 2019; Chen et al., 2020d; 2021c; Jiang et al., 2020).
Among them, adversarial training (AT) based methods (Madry et al., 2017;
Zhang et al., 2019) have maintained the state-of-theart robustness.
However, the AT training process usually comes with order-of-magnitude higher computational costs than standard training, since multiple attack iterations are needed to construct strong adversarial examples (Madry et al., 2018b).
Moreover, AT was recently revealed to incur severe robust generalization gaps (Rice et al., 2020), between its training and testing accuracies, as shown in Figure 1; and to require signiﬁcantly more training samples (Schmidt et al., 2018) to generalize robustly.
In response to those challenges, Schmidt et al.
Lee et al.
Song et al.
(2019) investigate the possibility of improving generalization by leveraging advanced data augmentation techniques, which further ampliﬁes the training cost of AT.
Recent studies (Rice et al., 2020; Chen et al., 2021c) found that early stopping, or several smoothness/ﬂatness-aware regularizations (Chen et al., 2021c; Stutz et al., 2021; Singla et al., 2021), can bring effective mitigation.
In this paper, a new perspective has been explored to tackle the above challenges by enforcing appropriate sparsity patterns during AT.
The connection between robust generalization and sparsity is mainly inspired by two facts.
On one hand, sparsity can effectively regularize the learning of over-parameterized neural networks, hence potentially beneﬁting both standard and robust generalization (Balda et al., 2019).
As demonstrated in Figure 1, with the increase of sparsity levels, the robust generalization gap is indeed substantially shrunk while the robust overﬁtting is alleviated.
On the other hand, the key design philosophy that facilitates this consideration is the lottery ticket hypothesis (LTH) (Frankle & Carbin, 2019).
The LTH advocates the existence of highly sparse and separately trainable subnetworks (a.k.a.
winning tickets), which can be trained from the original initialization to match or even surpass the corresponding dense networks’ expressiveness.
These facts point out a promising direction that utilizing proper sparsity is capable of boosting robust generalization while maintaining competitive standard and robust accuracy.
Although sparsity is beneﬁcial, the current methods (Frankle & Carbin, 2019; Frankle et al., 2020;
Renda et al., 2020) often empirically locate sparse critical subnetworks by Iterative Magnitude Pruning (IMP).
It demands excessive computational cost even for standard training due to the iterative train-prune-retrain process.
Recently, You et al.
(2020) demonstrated that these intriguing subnetworks can be identiﬁed at the very early training stage using one-shot pruning, which they term as
Early Bird (EB) tickets.
We show the phenomenon also exists in the adversarial training scheme.
More importantly, we take one leap further to reveal that even in adversarial training, EB tickets can still be drawn from a cheap standard training stage, while still achieving solid robustness.
In other words, the Early Bird is also a Robust Bird that yields an attractive win-win of efﬁciency and robustness - we name this ﬁnding as Robust Bird (RB) tickets.
Furthermore, we investigate the role of sparsity in a scene where the sparse connections of subnetworks are on the ﬂy.
Speciﬁcally, we initialize a subnetwork with random sparse connectivity and then optimize its weights and sparse typologies simultaneously, while sticking to the ﬁxed small parameter budget.
This training pipeline, called as Flying Bird (FB), is motivated by the latest sparse training approaches (Evci et al., 2020b; Liu et al., 2021b) to further reduce robust generalization gap in AT, while ensuring the low training costs.
Moreover, an enhanced algorithm, i.e., Flying Bird+ , is proposed to dynamically adjust the network capacity (or sparsity) to pursue superior robust generalization and performance, at minor prices of training efﬁciency.
Our innovative contributions can be summarized as follows:
• We perform a thorough investigation to reveal that introducing appropriate sparsity into AT is an appealing win-win, specially: (1) substantially alleviating the robust generalization gap; (2) maintaining comparable or even better standard/robust accuracies; and (3) enhancing the AT efﬁciency by training only compact subnetworks.
• We explore two alternatives for sparse adversarial training: (i) the Robust Bird (RB) training that leverages static sparsity, by mining the critical sparse subnetwork at the early training stage, and using only the cheapest standard training; (ii) the Flying Bird (FB) training that allows for dynamic sparsity, which jointly optimizes both network weights and their sparse connectivity during AT, while sticking to the same sparsity level.
We also discuss a FB variant called Flying Bird+ that adaptively adjusts the sparsity level on demand during AT.
• Extensive experiments are conducted on CIFAR-10, CIFAR-100, and Tiny-ImageNet with diverse network architectures.
Speciﬁcally, our proposals obtain 80 .
16% ∼ 87 .
83% training FLOPs and 80 .
16% ∼ 87 .
83% inference FLOPs savings, shrink robust generalization from 28 .
% ∼ 63 .
18% to 4 .
43% ∼ 34 .
44% , and boost the robust accuracy by up to 0 .
60% and the standard accuracy by up to 0 .
90% , across multiple datasets and architectures.
Meanwhile, combining our sparse adversarial training frameworks with existing regularizations establishes the new state-of-the-art results.
R ELATED W ORK
Adversarial training and robust generalization/overﬁtting.
Deep neural networks present vulnerability to imperceivable adversarial perturbations.
To deal with this drawback, numerous defense approaches have been proposed (Goodfellow et al., 2015; Kurakin et al., 2016; Madry et al., 2018a).
Although many methods (Liao et al., 2018; Guo et al., 2018a; Xu et al., 2017; Dziugaite et al., 2016;
Dhillon et al., 2018a; Xie et al., 2018; Jiang et al., 2020) were later found to result from obfuscated gradients (Athalye et al., 2018), adversarial training (AT) (Madry et al., 2018a), together with some of its variants (Zhang et al., 2019; Mosbach et al., 2018; Dong et al., 2018), remains as one of the most effective yet costly approaches.
A pitfall of AT, i.e., the poor robust generalization, was howevr noticed recently.
Schmidt et al.
(2018) showed that AT intrinsically demands a larger sample complexity to identify wellgeneralizable robust solutions.
Therefore, data augmentation (Lee et al., 2020; Song et al., 2019) is an effective remedy.
Stutz et al.
; Singla et al.
(2021) related robust generalization gap to curvature/ﬂatness of loss landscapes.
They introduced weight perturbing approaches and smooth activation functions to reshape the loss geometry and boost robust generalization ability.
Meanwhile, the robust overﬁtting (Rice et al., 2020) in AT usually happens with or as a results of inferior generalization.
Previous studies (Rice et al., 2020; Chen et al., 2021c) demonstrated that conventional regularization-based methods (e.g., weight decay and simple data augmentation) can not alleviate robust overﬁtting.
Then, numerous advanced algorithms (Zhou et al., 2021; Bunk et al., 2021; Chen et al., 2021a; Dong et al., 2021b;a; Zi et al., 2021; Tack et al., 2021;
Zhang et al., 2021a) arose in the last half year to tackle the overﬁtting, using data manipulation, smoothened training, and else.
Those methods work orthogonally to our proposal as evidenced in Section 4.
Another group of related literature lies in the ﬁled of sparse robust networks (Guo et al., 2018b).
These works either treat model compression as a defense mechanism (Wang et al., 2018; Gao et al., 2017; Dhillon et al., 2018b) or pursue robust and efﬁcient sub-models that can be deployed in resource-limited platforms (Gui et al., 2019; Ye et al., 2019; Sehwag et al., 2019).
Compared to those inference-focused methods, our goal is fundamentally different: injecting sparsity during training to reduce the robust generalization gap while improving training efﬁciency.
Static pruning and dynamic sparse training.
Pruning (LeCun et al., 1990; Han et al., 2015a) serves as a powerful technique to eliminate the weight redundancy in over-parameterized DNNs, which aims to obtain storage and computational savings with almost undamaged performance.
It can roughly divided into two categories based on how to generate sparse patterns: ( i ) static pruning .
It removes parameters (Han et al., 2015a; LeCun et al., 1990; Han et al., 2015b) or substructures (Liu et al., 2017; Zhou et al., 2016;
He et al., 2017) based on optimized importance scores (Zhang et al., 2018; He et al., 2017) or some heuristics like weight magnitude (Han et al., 2015a), gradient (Molchanov et al., 2019), hessian (LeCun et al., 1990) statistics.
The discarded elements usually will not participate in the next round of training or pruning.
Static pruning can be ﬂexibly applied prior to training, such as SNIP (Lee et al., 2019), GraSP (Wang et al., 2020) and
SynFlow (Tanaka et al., 2020); during training (Zhang et al., 2018;
He et al., 2017); and post training (Han et al., 2015a) for different trade-off between training cost and pruned models’ quality.
( ii ) dynamic sparse training .
It updates model parameters and sparse connectivities at the same time, starting from a randomly sparsiﬁed subnetwork (Molchanov et al., 2017).
During the training, the removed elements have chances to be grown back if they potentially beneﬁt to predictions.
Among the huge family of sparse training (Mocanu et al., 2016;
Evci et al., 2019; Mostafa & Wang, 2019;
Liu et al., 2021a; Dettmers & Zettlemoyer, 2019; Jayakumar et al., 2021; Raihan & Aamodt, 2020), the recent methods Evci et al. (2020a); Liu et al.
(2021b) lead to the state-of-the-art performance.
A special case of static pruning, Lottery tickets hypothesis (LTH) (Frankle & Carbin, 2019), demonstrates the existence of sparse subnetworks in DNNs, which are capable of training in isolation and reach comparable performance of their dense counterpart.
The LTH indicates the great potential to train a sparse network from scratch without scarifying expressiveness and has recently drawn lots of attention from diverse ﬁelds (Chen et al., 2020b;c;a;
Yu et al., 2019; Chen et al., 2021d;b) beyond image recognition (Zhang et al., 2021c; Frankle et al., 2020; Renda et al., 2020).
M ETHODOLOGY 3. P RELIMINARIES
Adversarial training (AT).
As one of the widely adopted defense mechanisms, adversarial training (Madry et al., 2018b) effectively tackles the vulnerability to maliciously crafted adversarial samples.
As formulated in Equation 1, AT (speciﬁcally PGD-AT) replaces the original empirical risk minimization into a min-max optimization problem:
[Figure]
[Equation]
(1) where f ( x ; θ ) is a network with parameters θ .
Input data x and its associated label y from training set
D are used to ﬁrst generate adversarial perturbations δ and then minimize the empirical classiﬁcation loss L .
To meet the imperceptible requirement, the (cid:96) p norm of δ is constrained by a small constant (cid:15) .
Projected Gradient Descent (PGD), i.e., δ t +1 = proj
P
[ δ t + α · sgn (cid:0) ∇ x L ( f ( x + δ t ; θ ) , y ) (cid:1) ] , is usually utilized to produce the adversarial perturbations with step size α , which works in an iterative manner leveraging the local ﬁrst order information about the network (Madry et al., 2018b).
Sparse subnetworks.
Following the routine notations in Frankle & Carbin (2019), f ( x ; m (cid:12) θ ) donates a sparse subnetwork with a binary pruning mask m ∈ { 0 , 1 } (cid:107) θ (cid:107) 0 , where (cid:12) is the elementwise product.
Intuitively, it is a copy of dense network f
( x ; θ ) with a portion of ﬁxed zero weights.
3. R OBUST B IRD FOR A DVERSARIAL T RAINING
Introducing Robust Bird.
The primary goal of Robust Bird is to ﬁnd a high-quality sparse subnetwork efﬁciently.
As shown in Figure 2, it locates subnetworks quickly by detecting critical network structures arisen in the early training, which later can be robustiﬁed with much less computation.
Speciﬁcally, for each epoch t during training, Robust Bird creates a sparsity mask m t by “masking out” the p % lowest-magnitude weights; then, Robust Bird tracks the corresponding mask dynamics.
The key observation behind Robust Bird is that the sparsity mask m t does not change drastically beyond the early epochs of training (You et al., 2020) because high-level network connectivity patterns are learned during the initial stages (Achille et al., 2019).
This indicates that (i) winning tickets emerge at a very early training stage, and (ii) that they can be identiﬁed efﬁciently.
Robust Bird exploits this observation by comparing the Hamming distance between sparsity masks found in consecutive epochs.
For each epoch, the last l sparsity masks are stored.
If all the stored masks are sufﬁciently close to each other, then the sparsity masks are not changing drastically over time and network connectivity patterns have emerged; thus, a Robust Bird ticket (RB ticket) is drawn.
A detailed algorithmic implementation is provided in Algorithm 1 of Appendix A1.
This is the RB ticket used in the second stage of adversarial training.
[Figure]
D e n s e
R a nd o m P r un i n g
[Figure]
i n g
B i r d +
Rationale of Robust Bird.
Recent studies (Zhang et al., 2021b)
present theoretical analyses that identiﬁed sparse winning tickets enlarge the convex region near the good local minima, leading to improved generalization.
Our work also shows a related investigation in Figure A10 that, compared with dense models and random pruned subnetworks, RB tickets found by the standard training have much ﬂatter loss landscapes, serving a high-quality starting point for further robustiﬁcation.
This occurs because ﬂatness of the loss surface is often believed to indicate the standard generalization.
Similarly, as advocated by Wu et al.
(2020a);
Hein & Andriushchenko (2017), a ﬂatter adversarial loss landscape also effectively shrinks the robustness generalization gap.
This “ﬂatness preference” of adversarial robustness has been revealed by numerous empirical defense mechanisms, including
Hessian/curvature-based regularization (Moosavi-Dezfooli et al., 2019), learned weight and logits smoothening (Chen et al., 2021c), gradient magnitude penalty (Wang & Zhang, 2019), smoothening with random noise (Liu et al., 2018), or entropy regularization (Jagatap et al., 2020).
These observations make the main cornerstone for our proposal and provide possible interpretations to the surprising ﬁnding that the RB tickets pruned from a non-robust model can be used for obtaining well-generalizable robust models in the followed robustiﬁcation.
Furthermore, unlike previous costly ﬂatness regularizers (Moosavi-Dezfooli et al., 2019), our methods not only offer a ﬂatter starting point, but also obtain substantial computational savings due to the reduced model size.
3. F LYING B IRD FOR A DVERSARIAL T RAINING
Introducing Flying Bird(+).
Since sparse subnetworks from static pruning are unable to regret about removed elements, they maybe too aggressive to capture the pivotal structural patterns.
Thus, we introduce Flying Bird (FB) to conduct a thorough exploration of dynamic sparsity, which allows pruned parameters to be grown back and engages in the next round of training or pruning, as demonstrated in Figure 2.
Speciﬁcally, it starts from a sparse subnetwork f
( x ; m (cid:12) θ ) with a random binary mask m , and then jointly optimize model parameters and sparse connectivities simultaneously.
In other words, the subnetwork’s typologies are “on the ﬂy”, decided dynamically based on current training status.
Speciﬁcally, we update Flying Bird ’s sparse connectivity every ∆ t epochs of adversarial training, which consists of two continually applied operations: pruning and growing.
For the pruning step, p % of model weights with the lowest magnitude will be eliminated, while g % weights with the largest gradient will be added back in the growth step.
Note that newly added connections are not activated in the last sparse topology, and are initialized to zero since it establishes better performance as indicated in (Evci et al., 2020a; Liu et al., 2021b).
Flying Bird maintains the sparsity ratio unchanged during the full training by keeping both pruning and growing ratio p % , g % equal k % that decays with a cosine annealing schedule.
We further propose Flying Bird+ , an enhanced variant of FB, capable of adaptively adjusting the sparsity and learning the right parameterization level “on demand” during training, as shown in
Figure 2.
To be speciﬁc, we ﬁrst record the robust generalization gap and robust validation loss at each training epoch.
An increasing generalization gap of the latter training stage indicates a risk of overﬁtting, while a plateau validation loss implies underﬁtting.
Hence, we then analyse the ﬁtting status according to the upward/downward trend of those measurements.
If most epochs (e.g., more than 3 out of the past 5 epochs in our case) tend to see enlarged robust generalization gaps, we raise the pruning ratio p % to further trim down the network capacity.
Similarly, if the majority of epochs present unchanged validation loss, we will increase the growing ratio q % to enrich the subnetwork capacity.
Detailed procedures are summarized in Algorithm 2 of Appendix A1.
Rationale of Flying Bird(+).
As demonstrated in Evci et al.
(2020a), allowing new connections to grow yields improved ﬂexibility in navigating the loss surfaces, which creates the opportunity to escape bad local minima and search for the optimal sparse connectivity Liu et al.
(2021b).
Flying
Bird follows a similar design philosophy that excludes least important connections (Han et al., 2015a), while activating new connections with the highest potential to decrease the training loss fastest.
Recent works (Wu et al., 2020c; Liu et al., 2019) have also found enabling network (re- )growth can turn a poor local minima into a saddle point that facilitates further loss decrease.
Flying
Bird + empowers the ﬂexibility further by adaptive sparsity level control.
The ﬂatness of loss geometry provides another view to dissect the robust generalization gain (Chen et al., 2021c; Stutz et al., 2021;
Singla et al., 2021).
Figure 3 compares the loss landscapes and training trajectories of dense, randomly pruned subnetworks, and Flying Brid+ robustiﬁed on CIFAR-10.
We observe that Flying Bird + converges to a wider loss valley with improved ﬂatness, which usually suggests superior robust generalization (Wu et al., 2020a; Hein & Andriushchenko, 2017).
Last but not the least, our approaches also signiﬁcantly trim down both the training memory overhead and the computational complexity, enjoying extra bonus of efﬁcient training and inference.
E XPERIMENT R ESULTS
Datasets and architectures.
Our experiments consider two popular architectures, ResNet-18 (He et al., 2016), VGG-16 (Simonyan & Zisserman, 2014) on three representative datasets, CIFAR-10,
CIFAR-100 (Krizhevsky & Hinton, 2009) and Tiny-ImageNet (Deng et al., 2009).
We randomly split one-tenth of the training samples as the validation dataset, and the performance is reported on the ofﬁcial testing dataset.
Training and evaluation details.
We implement our experiments with the original PGD-based adversarial trainig (Madry et al., 2018b), in which we train the network against (cid:96) ∞ adversary with maximum perturbations (cid:15) of 8 / 255 .
10 -steps PGD for training and 20 -steps PGD for evaluation are chosen with a step size α of 2 / 255 , following Madry et al.
(2018b)
; Chen et al.
(2021c).
In addition, we also use Auto-Attack (Croce & Hein, 2020) and CW Attack (Carlini & Wagner, 2017) for a more rigorous evaluation.
More details are provided in the Appendix A2.
For each experiment, we train the network for 200 epochs with an SGD optimizer, whose momentum and weight decay are kept to 0 .
9 and 5 × 10 − 4 , respectively.
The learning rate starts from 0 .
1 that decays by 10 times at 100 ,epoch and the batch size is 128 , which follows Rice et al.
For Robust Bird , the threshold τ of mask distance is set as 0 . 1 .
In Flying Birds (+), we calculate the layer-wise sparsity by Ideal Gas Quotas (IGQ) (Vysogorets & Kempe, 2021) and then apply random pruning to initialize the sparse masks.
FB updates the sparse connectivity per 2000 iterations of AT, with a update ratio k that starts from 50% and decays by cosine annealing.
More details are referred to Appendix A2.
Hyperparameters are either tuned by grid search or following Liu et al.
(2021b).
Evaluation metrics.
In general, we care about both the accuracy and efﬁciency of obtained sparse networks.
To assess the accuracy, we consider both Robust Testing Accuracy ( RA ) and Standard
Testing Accuracy ( SA ) which are computed on the perturbed and the original test sets, together with
Robust Generalization Gap ( RGG ) (i.e., the gap of RA between train and test sets).
Meantime, we report the ﬂoating point operations ( FLOPs) of the whole training process and single image inference to measure the efﬁciency.
4. R OBUST B IRD IS A G OOD B IRD
In this section, we evaluate the effectiveness of static sparsity from diverse representative pruning approaches, including: (i) Random Pruning (RP), by randomly eliminating model parameters to the desired sparsity; (ii) One-shot Magnitude Pruning (OMP), which globally removes a certain ratio of lowest-magnitude weights; (iii) Pruning at Initialization algorithms.
Three advanced methods, i.e., SNIP (Lee et al., 2019), GraSP (Wang et al., 2020) and SynFlow (Tanaka et al., 2020), are considered, which identify the subnetworks at initialization respect to certain criterion of gradient ﬂow.
(iv) Ideal Gas Quotas (IGS) (Vysogorets & Kempe, 2021).
It adopts random pruning based on pre-calculated layer-wise sparsity which draws intuitive analogies from physics.
(v) Robust Bird (RB), which can be regarded as a early stopped OMP.
(vi) Small Dense .
It is an important sanity check via considering a smaller dense networks with the same parameter counts as the ones of sparse networks.
Comprehensive results of these subnetworks at 80% and 90% sparsity are reported in Table 1, where the chosen sparsity follows routine options (Evci et al., 2020a;
Liu et al., 2021b).
[Table]
As shown in Table 1, we ﬁrst observe the occurrence of poor robust generalization with 38 .
82% RA gap and robust overﬁtting with 7 . 49% RA degradation, when training the dense network (Baseline).
Fortunately, coincided with our claims, injecting appropriate sparsity effectively tackle the issue.
For instance, RB greatly shrinks the RGG by 15 .
20% at 80 / 90% sparsity, while also mitigates robust overﬁtting by 2 .
53% ∼ 4 .
Furthermore , comparing all static pruning methods, we ﬁnd that (1) Small Dense and RP behave the worst, which suggests the identiﬁed sparse typologies play important roles rather than reduced network capacity only; (2) RB shows clear advantages to OMP in terms of all measurements, especially for 78 .
32% ∼ 84 .
80% training FLOPs savings.
It validates our RB proposal that a few epochs of standard training are enough to learn a high-quality sparse structure for further robustiﬁcation, and thus there is no need to complete the full training in tickets ﬁnding stage like traditional OMP.
(3) SynFlow and IGQ approaches have the best RA and SA, while RB obtains the superior robust generalization among static pruning approaches.
Finally, we explore the inﬂuence of training regimes during the RB ticket ﬁnding on CIFARwith ResNet-18.
Table A8 demonstrates that RB tickets perform best when found with the cheapest standard training.
Speciﬁcally, at 90% and 95% sparsity, SGD RB tickets outperform both Fast
AT (Wong et al., 2020) and PGD- 10 RB tickets with up to 1 .
27% higher RA and 1 .
86% narrower
RGG.
Figure A8 offers a possible explanation for this phenomenon: the SGD training scheme more quickly develops high-level network connections, during the early epochs of training (Achille et al., 2019).
As a result, RB Tickets pruned from model trained with SGD achieve the superior quality.
4. F LYING B IRD IS A B ETTER B IRD
In this section, we discuss the advantages of dynamic sparsity and show that our Flying Bird(+) is a superior bird.
Table 1 examines the effectiveness of FB(+) on CIFAR-10 with ResNet-18, and several consistent observations can be drawn: (cid:182) FB(+) achieve 9 .
92% ∼ 23 .
66% RGG reduction, 2 .
24% ∼ 5 .
88% decrease for robust overﬁtting, compared with the dense network.
And FB+ at 80% sparsity even pushes the RA 0 .
60% higher.
(cid:183)
Although the smaller dense network shows the leading performance w.r.t improving robust generalization, the robustness has been largely sacriﬁced, with up to 4 .
29% RA degradation, suggesting that only reducing models’ parameter counts is insufﬁcient to keep satisfactory SA/RA.
(cid:184) FB and FB+ achieve superior performance of RA for both the best and ﬁnal checkpoints across all methods, including RB.
(cid:185)
Regardless of small dense and random pruning due to their poor robustness, FB+ reaches the most impressive robust generalization (rank # or #2) with the least training and inference costs.
Precisely, FB+ obtains 84 .
46% ∼ 91 .
37% training
FLOPs and 84 .
46% ∼ 93 .
36% inference FLOPs saving, i.e., Flying Bird+ is SUPER light-weight .
[Table] [Table]
Superior performance across datasets and architectures.
We further evaluate the performance of FB(+) across various datasets (CIFAR-10, CIFAR-100 and Tiny-ImageNet) and architectures (ResNet-18 and VGG-16).
Table 2 and 3 display that both static and dynamic sparsity of our proposals serve effective remedies for improving robust generalization and mitigating robust overﬁtting, with 4 .
43% ∼ 15 .
45% , 14 . 99% ∼ 34 .
44% and 21 .
62% ∼ 23 .
60% RGG reduction across different architectures on CIFAR-10, CIFAR-100 and Tiny-ImageNet, respectively.
Moveover, both RB and
FB(+) gain signiﬁcant efﬁciency, with up to 87 .
83% training and inference FLOPs savings.
Superior performance across improved attacks.
Additionally, we verify both RB and FB(+) under imporved attacks, i.e., Auto-Attack (Croce & Hein, 2020) and CW Attack (Carlini & Wagner, 2017).
As shown in Table A7, our approaches shrink the robust generalization gap by up to 30 .
76% on CIFAR-10/100, and largely mitigate robust overﬁtting.
This piece of evidence shows our proposal’s effectiveness sustained across diverse attacks.
KD
SWA
SWA&KD
R obu s t A cc u r ac y ( % )
DenseFB+
Combining FB+ with existing start-of-the-art (SOTA) mitigation.
Previous works (Chen et al., 2021c; Zhang et al., 2021a; Wu et al., 2020b) point out that smoothening regularizations (e.g., KD (Hinton et al., 2015) and SWA (Izmailov et al., 2018)) help robust generalization and lead to SOTA robust accuracies.
We combine them with our
FB+ and collect the robust accuracy on CIFAR-10 with ResNet-18 in
Figure 4.
The extra robustness gains from FB+ imply that they makes complementary contributions.
Excluding obfuscated gradients.
A common “counterfeit” of robustness improvements is less effective adversarial examples resulted from obfuscated gradients (Athalye et al., 2018).
Table A demonstrates the maintained enhanced robustness under unseen transfer attacks, which excludes the possibility of gradient masking.
More are referred to Section A3.
4. A BLATION S TUDY AND V ISUALIZATION
Different sparse initialization and update frequency.
As two major components in the dynamic sparsity exploration (Evci et al., 2020a), we conduct thorough ablation studies in Table 4 and 5.
We found the performance of Flying Bird + is more sensitive to different sparse initialization; using
SNIP to produce initial layer-wise sparsity and updating the connections per 2000 iterations serves the superior conﬁguration for FB+.
[Table] [Table]
[Figure]
Final checkpoint loss landscapes.
From visualizations in Figure 5, FB and FB+ converge to much ﬂatter loss valleys, which evidences their effectiveness in closing robust generalization gaps.
Attention and saliency maps.
To visually inspect the beneﬁts of our proposal, here we provide attention and saliency maps generated by Grad-GAM (Selvaraju et al., 2017) and tools in (Smilkov et al., 2017).
Comparing the dense model to our “talented birds” (e
[Figure]
g., FB+), Figure 6 shows that our approaches have enhanced concentration on main objects, and are capable of capturing more local feature information, aligning better with human perception
[Figure]
RandomPruning
SNIP
FlyingBird
RobustBird
FlyingBird+
[Figure]
C ONCLUSION
We show the adversarial training of dense DNNs incurs severe robust generalization gap, which can be effectively and efﬁciently resolved by injecting appropriate sparsity.
Our proposed robust birds and ﬂying birds(+) with static and dynamic sparsity, signiﬁcantly mitigate the robust generalization gap while retain competitive standard/robust accuracy with substantially reduced computation.
In the future works, we plan to investigate more sparse structures like channel- and block-wise patterns.
A M ORE T ECHNIQUE D ETAILS
Algorithms of robust bird and ﬂying bird(+).
Here we present the detailed procedure to identify robust bird and ﬂying bird(+), as summarized in algorithm 1 and 2.
Note that for the increasing frequency on Line 10 and 11 in algorithm 2, we compare the measurements stored in the queue between two consequent epochs and calculate the frequency of an increasing.
Algorithm 1:
Input: f ( x ; θ 0 ) w. initialization θ 0 , target sparsity s % , FIFO queue Q with length l , threshold τ
Output: Robust bird f ( x ; m t ∗ (cid:12) θ T ) 1 while t < t max do
Update network parameters θ t ← θ t − 1 via standard training
Apply static pruning towards target sparsity s % and obtain the sparse mask m t
Calculate the Hamming distance δ H ( m t , m t − 1 ) , append result to Q t ← t + if max( Q ) <
τ then t ∗ ← t
Rewind f
( x ; m t ∗ (cid:12) θ t ∗ ) →
f ( x ; m t ∗ (cid:12) θ 0 )
Training f
( x ; m t ∗ (cid:12) θ 0 ) via PGD-AT for T epochs return f ( x ; m t ∗ (cid:12) θ T ) end 12 end
Algorithm
Input start
Output
Flying bird(+)
f ( x ; m (cid:12) θ
) 1 while t < T do
Update network parameters via
# Record training statistics
Add robust generalization gap between train and validation set to Q p
Add robust validation loss to Q g
# Update sparse masks m
if t mod ∆ t ) =
=
then
Grow g % parameters
end 17 end
A M ORE I MPLEMENTATION D ETAILS A2. O THER C OMMON D ETAILS
We select two checkpoints during training: best , which has the best RA values on the validation set, and ﬁnal , i.e., the last checkpoint.
And we report both RA and SA of these two checkpoints on test sets.
Apart from robust generalization gap, we also show the extent of robust overﬁtting numerically by the difference of RA between best and ﬁnal .
Furthermore, we calculate the FLOPs at both training and inference stage to evaluate the prices of obtaining and exploiting the subnetworks respectively, in which we approximate the FLOPs of the back-propagation to be twice that of forward propagation (Yang et al., 2020).
A2. M ORE D ETAILS ABOUT R OBUST B IRD
For the experiments of RB tickets ﬁnding, we comprehensively study three training regimes: standard training with stochastic gradient descent (SGD), adversarial training with PGD-10 AT (Madry et al., 2018b), and Fast AT (Wong et al., 2020).
Following Pang et al.
(2021), we train the network with an SGD optimizer of 0 .
9 momentum and 5 × 10 − 4 weight decay.
We use a batch size of 128 .
For the experiments of PGD-10 AT, we adopt the (cid:96) ∞ PGD attack with a maximum perturbation (cid:15) = 8 / 255 and a step size α = 2 / 255 .
And the learning rate starts from 0 .
1 , then decay by ten times at 50 , 150 epoch.
As for fast AT, we use a cyclic schedule with a maximum learning rate equals 0 . 2 .
A2. M ORE D ETAILS ABOUT F LYING B IRD (+)
For the experiments of Flying Bird+, the increasing ratio of pruning and growth δ p , δ q is kept default to 0 .
4% and 0 .
05% , respectively.
Distributions of adopted sparse initialization.
We report the layer-wise sparsity of different initial sparse masks.
As shown in Figure A7, we observe that subnetworks generally have better performance when the top layers remains most of the parameters.
c onv l a y e r 1 .. c onv l a y e r 1 ..
c onv l a y e r 1 .. c onv l a y e r 1 ..
c onv l a y e r 2 .. c onv l a y e r 2 ..
c onv l a y e r 2 .. s ho
r t c u t .l
a y e r 2 ..
c onv l a y e r 2 ..
c onv l a y e r 3 .. c onv l a y e r 3 ..
c onv l a y e r 3 .. s ho
r t c u t .l
a y e r 3 ..
c onv l a y e r 3 ..
c onv l a y e r 4 .. c onv l a y e r 4 ..
c onv l a y e r 4 .. s ho
r t c u t .l
a y e r 4 ..
c onv l a y e r 4 ..
c onv2 li n ea r
[Figure]
S p a r s it y
[Figure]
Excluding Obfuscated Gradients.
To exclude this possibility of gredient masking, we show that our methods maintain improved robustness under unseen transfer attacks.
The left table transfers the attack from an unseen robust model to the above four methods; and the right table shows the transfer testing performance on an unseen robust model (here we use a separately robustiﬁed ResNet-50 with
PGD
CIFAR
A M ORE E XPERIMENT R ESULTS
[Figure]
Mask Similarity Visualization.
Figure A9 visualizes the dynamic similarity scores for each epoch among masks found via SGD, Fast AT, and PGD-10.
Speciﬁcally, the similarity scores (You et al., 2020) reﬂect the Hamming distance between a pair of masks.
We notice that masks found by SGD and PGD-10 share more common structures.
A possible reason is that Fast AT usually adopts a cyclic learning rate schedule, while SGD and PGD use a multi-step decay schedule.
Different training regimes for ﬁnding RB tickets.
We denote the subnetworks identiﬁed by standard training with SGD, adversarial training with Fast AT (Wong et al., 2020) and adversarial training with PGD- 10 AT as SGD tickets, Fast AT tickets and PGD-10 tickets, respectively.
Table A demonstrate the SGD tickets has the best performance.
[Table] [Table]
Epoch
S t a n d a r d
A cc u r a c y %
RB Ticket Finding Performance
PGDSGD
FAST AT
M ORE R ESULTS ABOUT F LYING B IRD (+)
We report more results of different sparsiﬁcation methods across diverse datasets and architectures at Table A9, A10, A11 and A12, from which we observe our approaches are capable of improving robust generalization and mitigating robust overﬁtting.
A
[Figure]
PGD
[Figure]
[Figure]
G D
M
[Figure]
s k
[Figure]
[Figure]
[Table]
{+tickets 25 .22 .2 .
+}
.53 .−
03 ↓ 34 .
[Table]
tickets 24 .24 .
[-Random Pruning 46 .40 .
-]
[-.74 .76 .− 01 ↓ 6 . OMP 47 .43 .
 .76 .80-].50 .−
[-3 .24 . 97 ↓ 3 . SNIP 48 .43 .
 .76 .80 .− 71-]18 ↓ 45 .
[-GraSP 47 .42 .5 .
 .78 .− 65 ↓ 4 . SynFlow 48 .45 .3 .
-]{+Fast AT tickets 
+}
[-.79 .− 17 ↓ 7 . IGQ 48 .44 .
 .77 .80-].43 .−
[-Robust Bird 47 .41 .6 .
-]
53 ↓ 45 .
PGD- 10 tickets 24 .23 .
[-Random Pruning 44 .40 .4 .
-]
[-.74 .− 48 ↓ 12 . OMP 47 .43 .4 .
 .79-].49 .50 .−
86 ↓ [-9 . SNIP 47 .44 .3 .
 .79 .− 3 .17 . 85 ↓ 10-]44 .[-GraSP 45 .42 .3 .
-]
[-SynFlow 47 .45 .
-]
[-.78 .78 .− 
 40 ↓ 13 . IGQ 47 .45 .2 .74 .79 .− 06 ↓ 11 .-]A
[-Robust Bird 47 .44 .
-]
[-.78 .− 57 ↓ 11 . Flying Bird 48 .45 .2 .
 .79 .−-][Figure]
[-Flying Bird+ 48 .46 .
 .78 .79 .− 
 47 ↓ 15 . A-][Table]
[Table]
[Table] [Table]
5 .46 .48 .−
1 .40 . 42 ↓ 22 .
GraSP 23 .16 .6 .43 .46 .−
71 ↓ 13 .
SynFlow 23 .17 .5 .45 .47 .−
70 ↓ 21 .
IGQ 23 .17 .6 .45 .47 .−
18 ↓ 15 .
Robust Bird 23 .17 .5 .
.47 .−
19 ↓ 15 .
Flying Bird 22 .17 .
.47 .−
11 ↓ 15 .
Flying Bird+ 22 .19 .
.48 .− 1 .34 .
63 ↓ 28 .
Random Pruning 21 .16 .
.43 .44 .−
34 ↓ 31 .
OMP 22 .17 .
.44 .45 .−
91 ↓ 24 .
SNIP
.48 .49 .−
02 ↓ 43 .
GraSP 22 .17 .
.47 .−
76 ↓ 33 .
SynFlow 22 .18 .
.46 .−
96 ↓ 38 .
IGQ 22 .18 .
.44 .48 .−
91 ↓ 35 .
Robust Bird 22 .19 .3 .
.48 .−
46 ↓ 36 .
Flying Bird 23 .18 .
.46 .48 .−
05 ↓ 29 .
Flying Bird+ 23 .20 .
.45 .48 .−
16 ↓ 41 .
A
A N EW R ESULTS AND D ISCUSSION A4. M ORE RESULTS OF DIFFERENT SPARSITY
[Table]
[Table]
[Table]
[Table]
[Table]
Flying Bird+
[Table]
We report more results of subnetworks with 40 / 60% sparsity on CIFAR-10/100 with ResNetand VGG-16.
As shown in Table A13, A15, A14 and A16, our ﬂying bird(+) achieves consistent
[Table]
improvement than baseline unpruned networks, in terms of
[Table]
45 ∼
[Table]
81% narrower robust generalization gaps with comparable RA and SA performance.
A4. M ORE RESULTS ON W IDE R ES N ET
We further evaluate our ﬂying bird(+) with WideResNet-34-10 on CIFAR-10 and report the results on Table A17. We can observe that compared with the dense network, our methods signiﬁcantly shrink the robust generalization gap by up to 13 . 14% and maintain comparable RA/SA performance . A4. C OMPARISON WITH EFFICIENT ADVERSARIAL TRAINING METHODS
To elaborate more about training efﬁciency, we compare our methods with two efﬁcient training methods. Shafahi et al. (2019) proposed Free Adversarial Training that improve training efﬁciency by reusing the gradient information, which is orthogonal to our approaches and can be easily combined with our methods to pursue more efﬁciency by replacing the PGD-10 training with Free AT. Additionally, Li et al.
(2020) uses magnitude pruning to locate sparse structures, which is similar to
OMP reported in Table 1, except they use a smaller learning rate. Our methods achieve better performance and efﬁciency than OMP. Speciﬁcally, with 80% sparsity, our ﬂying bird+ reaches a 4 . 49% narrower robust generalization gap and 1 .
54% higher RA yet only requires 87 . 58% less training FLOPs. Also, our methods can be easily combined with Fast AT for further training efﬁciency.