H OLISTIC A DVERSARIALLY R OBUST P RUNING
A BSTRACT
Neural networks can be drastically shrunk in size by removing redundant param- eters.
While crucial for the deployment on resource-constraint hardware, oftentimes, compression comes with a severe drop in accuracy and lack of adversarial robustness.
Despite recent advances, counteracting both aspects has only succeeded for moderate compression rates so far.
We propose a novel method, H ARP , that copes with aggressive pruning significantly better than prior work.
For this, we consider the network holistically.
We learn a global compression strategy that optimizes how many parameters (compression rate) and which parameters (scoring connections) to prune specific to each layer individually .
Our method fine-tunes an existing model with dynamic regularization, that follows a step-wise incremental function balancing the different objectives.
It starts by favoring robustness before shifting focus on reaching the target compression rate and only then handles the objectives equally.
The learned compression strategies allow us to maintain the pre-trained model’s natural accuracy and its adversarial robustness for a reduction by 99 % of the network’s original size.
Moreover, we observe a crucial influence of non-uniform compression across layers.
The implementation of H ARP is publicly available at https://intellisec.de/research/harp .
I NTRODUCTION
Deep neural networks (DNNs) yield remarkable performances in classification tasks in various domains (He et al., 2016; Cakir & Dogdu, 2018;
Schroff et al., 2015; Li et al., 2022) but are vulnerable to input manipulation attacks such as adversarial examples (Szegedy et al., 2014).
Small perturbations to benign inputs can cause worst-case errors in prediction.
To date, the most promising defensive approach against this sort of attacks is adversarial training as introduced by Madry et al.
(2018) and further refined ever since (Shafahi et al., 2019; Zhang et al., 2019; Wang et al., 2020).
It introduces adversarial examples into the training process, diminishing the generalization gap between natural performance and adversarial robustness.
However, there is evidence indicating that higher robustness requires over-parameterized networks that have wider layers and higher structural complexity (Madry et al., 2018; Zhang et al., 2019; Wu et al., 2021), rendering the task of combining both objectives—compression and robustness—inherently difficult.
Neural network pruning (Han et al., 2016; Yang et al., 2017;
He et al., 2017), for instance, has been proven to be an extraordinary valuable tool for compressing neural networks.
The model can be reduced to a fraction of its size by removing redundancy at different structural granularity (Li et al., 2017; Mao et al., 2017; Kollek et al., 2021).
However, pruning inflicts a certain recession in model accuracy (Han et al., 2016) and adversarial robustness (Timpl et al., 2021) that is unavoidable the stronger the model is compressed.
The aim of adversarial robust pruning hence is to maintain the accuracy and robustness of an already adversarially pre-trained model as good as possible.
Despite great efforts (Ye et al., 2019; Sehwag et al., 2020; Madaan et al., 2020; Özdenizci & Legenstein, 2021; Lee et al., 2022), this dual objective has only been achieved for moderate compression so far.
In this paper, we start from the hypothesis that effective adversarially robust pruning requires a nonuniform compression strategy with learnable pruning masks, and propose our method H ARP .
We follow the three-stage pruning pipeline proposed by Han et al.
(2015) to improve upon pre-trained models, where we jointly optimize score-based pruning masks and layer-wise compression rates during fine-tuning.
As high robustness challenges the compactness objective (Timpl et al., 2021), we employ a step-wise increasing weighted-control of the number of weights to be pruned, such that we can learn masks and rates simultaneously.
Our approach explores a global pruning strategy that
[Figure]
In summary
[Figure]
we make the following contributions:
• Novel pruning technique for pre-trained models.
We optimize how many parameters and which parameters to prune for each layer individually , resulting in a global but non-uniform pruning strategy.
That is, the overall network is reduced by a predetermined rate governed by the target hardware’s limits, but layers are compressed varyingly strongly.
We show that both aspects are needed for H ARP to take full effect (cf.
Section 4.1).
• Significant improvement over related work.
An overview of our method’s performance is presented in Fig.
1 for a PGD-AT trained VGG16 model learned on CIFAR-10 (left) and SVHN (right), providing a first glimpse of the yield advances.
No less importantly, we conduct experiments with small (CIFAR-10, SVHN) as well as large-scale (ImageNet) datasets across different robust training methods and various attacks (cf.
Section 4.2).
• Importance of non-uniform pruning for adversarial robustness.
We emphasize the superiority of non-uniform strategies ( ? ) by extending existing adversarially robust pruningtechniques.
We demonstrate that H YDRA (Sehwag et al., 2020) and Robust-ADMM (Ye et al., 2019) yield better results
when used with non-uniform strategies determined by ERK (Evci et al., 2020) and LAMP (Lee et al., 2021)
(cf.
Section 4.3).
N EURAL N ETWORK P RUNING
Removing redundant parameters reduces a network’s overall memory footprint and necessary computations, allowing for a demand-oriented adaptation of neural networks to resource-constrained environments (Han et al., 2015; 2016; Wen et al., 2016; Huang et al., 2018;
He et al., 2018; 2017; Li et al., 2017; Mao et al., 2017; Molchanov et al., 2017).
Neural network pruning attempts to find a binary mask M ( l ) for each layer l of a network with L layers represented by its parameters θ and θ ( l ) , respectively.
The overall pruning-mask thus is denoted as M =
( M (1) , . . .
, M ( l ) , . .
, M ( L ) ) .
These masks specify which parameters of the layer θ ( l ) to remove (zero out) and which to keep, yielding a reduced parameter set ˆ θθ ( l ) = θ ( l ) ⊙ M ( l ) , where ⊙ is the Hadamard product.
Based on this, we define the overall compression rate a as the ratio of parameters preserved after pruning, Θ ̸ =0 , to the number of parameters in the model, Θ .
Compression rates for individual layers a ( l ) are defined analogously.
Note, that a network’s sparsity is defined inversely, meaning, a 99 . 9 % sparsity refers to a compression of 0 .
In further follow, we consider θ ( l ) ∈ R c
( l ) i × c
( l ) o × k ( l ) × k
( l ) where c ( l ) i and c ( l ) o represent input and output channels, respectively, and k ( l ) is the kernel size.
Han et al.
(2015) propose a three stage pipeline for network pruning, starting with (1) training an over-parameterized network, followed by (2) removing redundancy per layer according to some pruning criterion, before (3) recovering network performance via fine-tuning.
The actual pruning strategy, that decides on the pruning mask M , is obtained in the second step.
While integrated approaches (pruning during model training) exist (e.g., Vemparala et al., 2021; Özdenizci & Legenstein, 2021), the staged process remains most common (Liu et al., 2019; Sehwag et al., 2020; Lee et al., 2022) as it allows to benefit from recent advantages in adversarial training (Shafahi et al., 2019; Zhang et al., 2019;
Wang et al., 2020) out-of-the-box.
H OLISTIC A DVERSARIALLY R OBUST P RUNING
Two aspects are crucial when pruning neural networks: How many and which parameters to prune .
Current state-of-the-art methods for adversarially robust pruning focus on the latter, allocating a fixed compression budget governed by the target hardware’s limits and using it uniformly for each layer (e.g., Ye et al., 2019; Sehwag et al., 2020).
We argue that learning the optimal pruning amount—the compression rate per layer—is equally crucial.
H ARP thus integrates measures for compression rates and scoring connections in an empirical min-max optimization problem based on adversarial training to learn a suitable pruning mask M for a pre-trained model θ :
[Equation]
The inner maximization generates adversarial examples from a benign input x with label y from a distribution D = { x , y } by adding noise δ , and incorporates these when minimizing the training loss L ce (Madry et al., 2018).
The exact formulation of L robust depends on the pre-trained model, for which we evaluate PGD-AT (Madry et al., 2018), TRADES-AT (Zhang et al., 2019), and MARTAT (Wang et al., 2020).
Simultaneously, the model is subject to a pruning mask M that involves two parameters:
First, the compression quota r , which is a learnable representation of compression rates, and second, scores S for determining the importance of the network’s connections.
In the following, we discuss H ARP ’s global compression control and describe how pruning strategies are learned, before we elaborate on how to dynamically regularize the training process using γ by following a step-wise incremental function balancing the different objectives.
Global compression control.
The compression control is encoded as an additional loss term L hw , that considers compression for the entire network to reach a specific target compression a t .
However, it explicitly allows for layer-specific rates, yielding an overall non-uniform compression strategy:
We measure the number of currently preserved (i.e., non-zero) weights across all L layers of the network, Θ ̸ =0 = (cid:80) Ll =1 ∥ ( 1 w ̸ =0 ) w ∈ ˆ θ ( l ) ∥ 1 , relative to the targeted total number of parameters at a compression rate a t :
[Equation]
Additionally, we clip rates that would exceed the targeted compression-rate. Constraining the loss to positive values ensures to not encourage even lower compression rates that would potentially harm accuracy and robustness unnecessarily.
For learning the layer-specific compression rates, moreover, it is crucial that we constrain the reduction by a minimal compression a min = 0 . 1 × a t as we need to prevent a layer from been removed completely.
We thus limit the layers’ compression a ( l ) to [ a min , 1] .
In order to better control learning of the optimal pruning rate in this range, we introduce a trainable parameter r ( l ) ∈ R , which we refer to as compression quota , and constrain it by an activation function g : R → [ a min , 1] .
For the sake of the continuous derivability on r , we use the sigmoid function squeezed to the desired output range, yielding a layer-wise compression rate a ( l )
=
g ( r ( l ) ) , where g : r (cid:55)→ (1 − a min ) · sig( r ) + a min .
Connection importance.
For learning the actual pruning mask, in turn, we introduce another learnable parameter S , the connection importance-score matrix, to rate each pruning connection.
In weight pruning, S ( l ) originates R c ( l ) i × c
( l ) o × k ( l ) × k
( l ) , and is assigned values from activation
function g .
Additionally, we use P ( α ( l ) , S ( l ) ) to represent the function that determines the α -th percentile on layer l as the cutoff threshold for pruning parameters in accordance with compression rate a ( l ) , that is: α ( l ) = 1 − a ( l ) .
Due to the relation with S ( l ) (and therefore also with r ( l ) via compression rate a ( l ) ), the binary pruning mask M ( l ) is thus trainable and defined as:
[Equation]
Pruning the l -th layer can hence be expressed by the Hadamard product of the mask and the model’s parameters θ yielding the pruned model parameters ˆ θθ ( l )
= θ ( l ) ⊙ M ( l ) .
Learning pruning strategies.
Binarizing importance scores to obtain a pruning mask as described above is a non-differentiable operation and prevents us from performing gradient descent over the masks.
We, thus, follow the “straight through estimation” (STE) strategy (Hubara et al., 2016) as proposed by Kusupati et al.
(2020) to assign the updated gradients to the importance scores S ( l ) directly and proceed similarly for compression quotas r ( l ) :
[Equation]
Finally, also the initialization of the connection importance-scores is crucial.
We follow the lead of
Sehwag et al.
and initialize the scores proportional to the weights of the pre-trained model and use a scaling factor η = (cid:112) k / fan-in ( l ) with k = 6 and fan-in being the product of the receptive field size and the number of input channels.
Our initialization for pruning weights, hence, follows the order of weight magnitudes (Ye et al., 2019):
[Equation]
H ARP does not use a fixed ratio for pruning parameters, but learns the optimal compression rate per layer.
However, for initializing the compression quota r , we first align with a uniform compression strategy that sets the same rate a init on each layer and let the optimizer improve it (cf.
Appendix A. for more details).
For the activation function g ( · ) defined above, we thus yield:
[Equation]
Balancing pruning objectives.
H ARP strives to strike a balance between natural accuracy, adversarial robustness and compactness.
For this, we employ a step-wise incremental function to adapt the regularization parameter γ throughout the learning process.
We start by favoring robustness before shifting focus on reaching the target compression rate and only then handle the objectives equally.
[Figure]
L r o b u s t
L h w
Fig.
2 shows the relation of γ and the two different losses, L robust against PGD-10 and L hw , on the y-axis over the epochs on the x-axis, exemplarily for weight pruning VGG
(learned on CIFAR-10) with a target compression a t = 0 .
01 , that is, removing 99 % of the parameters.
The model starts off from the uniform strategy with a init = 0 .
1 , learning to be more robust while being only slightly penalized by L hw .
As the model has been adversarially pretrained, L robust is low already.
Over time, we put more focus on L hw up until we have reached our target compression ( γ = 0 .
11 in this example), causing it to decrease while L robust increases.
Afterwards, the model aims for higher robustness under the found L hw penalty, balancing both objectives.
In
Appendix A.1, we study the influence of different step sizes and elaborate on the progression of the model’s natural accuracy and adversarial robustness in Appendix A.3.
E VALUATION
We resume to demonstrate H ARP ’s effectivity in pruning neural networks by comparing to existing state-of-the-art methods in Section 4.2.
In Section 4.1, we then perform an ablation study regarding the learnable parameters used by our method.
In particular, we show the influence of learning compression quotas r and importance scores S , separately, to demonstrate that both are indeed needed for successful, holistic pruning.
In Section 4.3 and Appendix A.3, we then discuss the non-uniform nature of H ARP ’s pruning strategy and compare to novel extensions of H YDRA and Robust-ADMM to non-uniform compression.
In the appendix, we additionally present the extended comparison to related work (Appendix A.2), elaborate on the pruned model’s parameter distribution (Appendix A.4), show H ARP ’s performance on naturally trained models (Appendix A.5), and extend H ARP to channel pruning (Appendix A.6).
Experimental setup.
We evaluate H ARP on two small-scale datasets, CIFAR-10 (Krizhevsky et al.) and SVHN (Netzer et al., 2011).
While the first is balanced, the second is not.
Consequently, we use accuracy (ACC) as performance measure for the former, and the balanced accuracy (BACC) for the latter (Brodersen et al., 2010).
Each dataset is learned with a VGG16 (Simonyan & Zisserman, 2015) and
ResNet
(He et al., 2016) model.
Additionally, we show the performance on the large-scale, balanced ImageNet dataset (Deng et al., 2009) trained with a ResNet50 network.
We apply γ = 0 .
01 for small-scale datasets, and increase γ to 0 .
1 for ImageNet to guarantee the arrival at the target compression rate a t .
In the pruning and fine-tuning phase of H ARP , we train for 20 epochs and 100 epochs, respectively.
Adversarial training.
We adversarially train on the small-scale datasets, CIFAR-10 and SVHN, with PGD-AT (Madry et al., 2018), TRADES-AT (Zhang et al., 2019), and MART-AT (Wang et al., 2020).
For each, we use l ∞ PGD-10 attacks with random initialization and a perturbation strength ϵ = 8 / 255 with step size α = 2 / 255 .
{+We adopt stochastic gradient descent with a cosine learning-rate schedule (Loshchilov & Hutter, 2016) and a weight decay of 0 . 
+}
(Pang et al., 2021).
For TRADES and MART, we follow the originally proposed regularization of λ = 6 .
and λ = 5 . 0 , respectively.
To accelerate the training process for the large-scale ImageNet experiments, we resort to F REE -AT (Shafahi et al., 2019) with 4 replays, reducing the pruning and fine-tuning phase to 5 epochs and 25 epochs, and use a perturbation strength ϵ = 4 / 255 with step size α = 1 / 255 .[-For all, we use stochastic gradient descent with a cos. learning-rate schedule (Loshchilov & Hutter, 2016) and a weight decay of 0 . 
 (Pang et al., 2021).-]
Adversarial robustness evaluation.
To validate the pruned networks’ adversarial robustness, we use C&W ∞ (Carlini & Wagner, 2017) optimized by PGD and PGD-10 with the same settings used for training.
Furthermore, we implement A UTO PGD (APGD) (Croce & Hein, 2020a) with crossentropy (CE) loss, 50 steps, and 5 restarts, and A UTO A TTACK (AA) (Croce & Hein, 2020a) using the standard ensemble out of A UTO PGD with CE-loss, Targeted-A UTO PGD DLR , FAB (Croce & Hein, 2020b), and S QUARE A TTACK (Croce et al., 2019).
All attacks are carried out on the respective complete test dataset.
4. A BLATION S TUDY
H ARP features two important components:
Learning compression quotas and learning what connections to prune.
While we do argue that both aspects are crucial for effective adversarial robust pruning, in this section, we set out to analyze whether this indeed is the case.
We perform an ablation study regarding the compression quota to be learned r and determining importance scores S .
For the experiments reported in Table 1, we thus either learn compression quotas or importance scores, while keeping the other one constant—consequently, we also do not balance pruning objectives here.
Note that for the optimization on compression quotas r , importance scores S remain as initialized by Eq.
(9), while during investigating the influence of scores S , we use uniform compression.
We denote our method with compression-rate optimization only as H ARP - r , and with importance-score optimization only as H ARP - S .
As already observed in Fig.
1, moderate compression manages to maintain natural accuracy and adversarial robustness very well, such that in this study, we focus on 99 % and 99 . 9 % sparsity.
[Table]
Interestingly, H ARP - r appears to be particular beneficial for moderate to high compression, while
H ARP - S is increasingly important for aggressive pruning.
In particular for VGG16 with its cascading network architecture, this tendency is apparent for different pre-training strategies.
However, at either sparsity level the combination of both, that is, optimizing compression rates r and importance scores S , allows H ARP to excel.
4. R OBUST P RUNING WITH H ARP
In this section, we report results of our comparative evaluation with Robust-ADMM (Ye et al., 2019),
H YDRA (Sehwag et al., 2020), BCS-P (Özdenizci & Legenstein, 2021), and MAD (Lee et al., 2022) in pruning a neural network’s weights.
In particular, for aggressive compression yielding high sparsity, H ARP excels and outperforms prior work.
For the comparison to Robust-ADMM and
H YDRA , we perform a systematic evaluation with CIFAR-10, SVHN, and ImageNet.
For BCS-P and MAD, we revert to the respective settings reported in the original papers.
Comparison to Robust-ADMM.
The method by Ye et al.
(2019) is designed to robustly prune neural networks on fine-grained (weights) and coarse-grained granularity (channels/filters).
We focus on pruning weights at this point and present further results on pruning channels in Appendix A.6.
Tables 2 to 4 summarize the results for pruning models on CIFAR-10, SVHN, and ImageNet, respectively.
Robust-ADMM performs well for pruning a network’s weights to a sparsity of 99 % for
CIFAR-10 as well as SVHN.
However, its performance drops drastically when increasing sparsity to 99 .
For SVHN, the models are even completely damaged, yielding a balanced accuracy of 10 % .
Also for ImageNet, Robust-ADMM significantly harms the model robustness and natural performance.
H ARP , in turn, exceeds the performance of Robust-ADMM distinctively.
Models pruned by our method preserve natural accuracy and robustness significantly better for high compression rates:
At a sparsity of 99 % , the natural accuracy and the adversarial robustness is at a comparable level to the original model on CIFAR-10 and SVHN.
Even for aggressive pruning to a sparsity of 99 . 9 % , the models show substantial resistance against adversarial inputs across training methods and considered attackers.
On ImageNet, H ARP outperforms Robust-ADMM in moderate and aggressive pruning with 20 percentage points for natural accuracy and 10 percentage points for robustness against various attacks.
[Table]
[Table] [Table] [Table]
[Table]
Comparison to H YDRA .
Sehwag et al.
(2020) demonstrate remarkable performance on robustly pruning a model’s weights, which we are able to confirm in our experiments.
The results are presented in Tables 2 to 4 for CIFAR-10, SVHN
[Table]
and ImageNet.
H YDRA consistently improves upon
Robust-ADMM for the sparsity levels considered in our experiments.
However, while H YDRA maintains its natural accuracy and robustness for a sparsity of 99 % well, for 99 . 9 % its performance degrades heavily (similarly to Robust-ADMM).
Differently, H ARP is able to maintain both accuracy metrics significantly better for aggressive pruning.
In most cases, our method doubles the accuracy of H YDRA at a sparsity of 99 . 9 % , yielding an unprecedentedly competitive performance for such highly compressed networks.
Also, the additionally gained robustness stemming from using
MART-AT rather than TRADES-AT nicely transfers to models pruned by H ARP , while H YDRA cannot benefit from this advantage at the same level.
For ImageNet, H ARP is also able to outperform
H YDRA noticeably and has the highest effectiveness in suppressing the model performance recession.
However, the drop in performance from 90 % to 99 % sparsity is comparable to H YDRA and remains significant.
Limiting the decay even further for large-scale datasets remains a challenge.
Comparison to BCS-P. Similar to our approach, BCS-P (Özdenizci & Legenstein, 2021) strives for a non-uniform compression strategy.
To do so, they introduce “Bayesian Connectivity Sampling” for selecting important network elements from a global view.
In contrast to H ARP , it however does not follow the staged pruning-pipeline of Han et al.
but achieves robust pruning via an end-to-end sparse training-procedure.
In Fig. 1, we present results of BCS-P for a VGG16 model trained with PGD-AT on CIFAR-10, and explicitly investigate its performance at sparsity 99 % and 99 . 9 % on CIFAR-10 and SVHN in Tables 2 and 3. While it does not allow for channel pruning, BCS-P outperforms both H YDRA and Robust-ADMM in pruning VGG16’s weights, while remains inferior to H YDRA in pruning ResNet18.
H ARP , however, surpasses BCS-P significantly with increasing model sparsity and yields a functional model at 99 .
9 % sparsity, while BCS-P does not.
In Section 4.3, we analyze the strategies and attribute the observed collapse to a heavily pruned input layer.
More comparative results and experiments on ResNet18 are presented in Appendix A.2.
Comparison to MAD.
Lee et al.
(2022) propose to compute an adversarial saliency map of a robustly pre-trained model via “Kronecker-Factored Approximate Curvature” (Martens & Grosse, 2015) and mask out weights that are least relevant for robustness.
Unfortunately, we have not been able to fully reproduce the MAD’s results.
We thus present the numbers reported by the authors of MAD along-side results for H ARP in the experimental setup described in the original publication and adjust pre-training accordingly for CIFAR-10 and SVHN.
Lee et al.
(2022) use a PGD step-size of 0 . 0069 for adversarial pre-training over 60 epochs and a 2 × 10 − 4 weight decay.
At moderate compression ( < 90 % sparsity) shown in Table 5, MAD and H ARP are on a par.
Neither one harms the performance of the neural network for CIFAR-10 and SVHN.
[Table]
As indicated in Fig.
3, showing the PGDadversarial robustness on CIFAR-10 and
SVHN, this changes for 99 % sparsity.
While our method maintains robustness, MAD drops in performance significantly.
In particular, it is noteworthy that at a sparsity of 99 . 9 % H ARP achieves similar adversarial robustness than MAD at a sparsity of 99 % , meaning, the network pruned by MAD is an order of magnitude larger.
[Figure]
4. S TRATEGY A NALYSIS
Also in conventional network pruning, non-uniform compression strategies have been proven effective, for instance, ERK by Evci et al.
(2020) and LAMP by Lee et al.
While we are the first to show an equivalent benefit for adversarial robust pruning, in this section, we investigate (a) if these approaches can also be applied to adversarial robust pruning, and (b) if they find similar compression strategies in comparison to H ARP .
Both ERK and LAMP focus on model parameters and support weight pruning only.
We adjust H YDRA to accept different compression rates per layer and
[Table]
ResNet
[Table]
assign the strategies determined by ERK and LAMP.
The results for pruning a PGD-AT pre-trained VGG16 model are presented in Table 6.
LAMP in particular produces very promising compression strategies that, however, are surpassed by H ARP .
For ImageNet learned with ResNet50, in turn, using LAMP and ERK with H YDRA impacts performance negatively, unlike H ARP (cf. Tablein Appendix A.3).
In Fig.
4, we present the non-uniform compression strategies determined by the different approaches as (a) compression rates and (b) overall preserved parameters per layer.
Additionally, we show a uniform compression yielding 99 .
9 % sparsity as reference.
Note, that due to the layer’s difference in size the absolute number of preserved parameters varies for the uniform strategy in Fig.
4b also.
It is interesting to see that LAMP puts more focus on layers in the middle sacrificing input/output information, while H ARP tends to focus on front and back layer stronger.
ERK, in turn, even maintains large portions of fc1 .
Interestingly, BCS-P yields an almost uniform strategy although it strives for non-uniformity.
Even worse, only two out of 1 , 728 parameters in VGG16’s input layer are preserved at 99 . 9 % sparsity, severely impacting performance.
[Figure]
[Table]
Layer
P r e s e r v e d P a r a m
[Table]
C ONCLUSIONS
For pruning neural networks it is crucial to decide how many and which parameters to prune .
This does not only affect natural accuracy, but also the robustness against adversarial input manipulations.
Maintaining adversarial robustness is crucial for safety-critical applications, such as autonomous driving and edge AI.
Our method, H ARP , incorporates a global view of the network’s compression and scores for gauging the importance of network connections into a dynamically regularized loss formulation that allows to significantly outperform related work.
We are the first to reach competitive performance for highly aggressive pruning, aiming at up to 99 .
9 % network sparsity.
Therewith, we show that learning a global, but layer-specific and, thus, non-uniform compression strategy is at least as important as deciding on what connections to prune.
Limitations.
Similarly to the majority of related work in the field, our method is of empirical nature and, thus, can benefit from future work on theoretic analyses of how and why our pruning approach arrives at a particular compression strategy.
In our evaluation, we show that H ARP ’s compression is close to the theoretically founded strategies of LAMP, raising the hope that a rigorous theoretic justification is possible.
Ideally this extends to a theoretically determined, one-shot calculation that spares the computational effort of the optimization problem considered for H ARP . A CKNOWLEDGEMENTS
The authors thank the anonymous reviewers for their valuable suggestions and gratefully acknowledge funding by the Helmholtz Association (HGF) within topic “46.23 Engineering Secure Systems" and by SAP S.E. under project DE-2020-021.
E THICS S TATEMENT
Deep neural networks are used in a wide variety of applications. While these applications may have ethical implications, our method itself does not. HARP aims to compress an existing model, maintaining accuracy and resistance against input manipulation attacks.
As such, our method is agnostic to the underlying application. Moreover, we do not reveal any vulnerabilities in the robust model compression, do not use any personalized data and do not involve human subjects in our experiments. R EPRODUCIBILITY S TATEMENT For the sake of reproducibility and to foster future research we make the implementations of H ARP for holistic adversarially robust pruning publicly available at: https://intellisec.de/research/harp
A A PPENDIX
We proceed to analyze the influence of different step sizes to regularize H ARP (Appendix A.1), before we extend upon the empirical evidence of our methods performance: We broaden our comparison to related work by inspecting the performance of BCS-P and MAD on ResNet18 and additionally considering DNR (Appendix A.2).
We also extend upon the analysis of H ARP ’s pruning strategies (Appendix A.3), inspect the parameter distribution of the pruned models (Appendix A.4, and evaluate H ARP ’s performance based on natural training (Appendix A.5).
Moreover, we demonstrate an extension of our method to channel/structural pruning (Appendix A.6) and present a comparison of training consumption for all methods in our evaluation (Appendix A.7).
A. I NFLUENCE OF DYNAMIC γ ON H ARP ’ S PRUNING
In this section, we present results for different regularization step-sizes of H ARP when pruning
VGG16 to a sparsity of 99 . 9 % .
As illustrated in Figs.
5a and 5b, H ARP exhibits strong oscillation for small step sizes ( γ ≤ 0 . 1 ), while larger steps ( γ = 1 . 0 ) restrict performance improvements.
A step size of γ = 0 .
01 , in turn, helps the model to better converge.
H ARP performs similarly for a slightly changed step size of γ = 0 .
005 , but yielding the target compression is delayed.
When [Figure] lowering the step size further to 0 .
001 , the target compression cannot be reached anymore as shown in Fig.
5c.
Hence, small step sizes shorten the time for adversarial training but cannot guarantee yielding the target compression.
In Fig.
5d, we summarize the generated strategies that satisfy the target compression.
With decreasing γ , H ARP preserves layer in front and at the end stronger.
Model parameters in the middle layers are sacrificed to preserve higher performance.
A. E XTENDED C OMPARISON TO R ELATED W ORK
Comparison to BCS-P. Fig.
6 shows extended experiments for ResNet18.
At sparsity 99 .
9 % , model collapse appears after BCS-P’s pruning, particularly on SVHN.
Fig.
7 visualizes pruning strategies for ResNet18 on SVHN.
Similar to Fig.
4a, BCS-P exhibits an almost uniform pruning strategy, compressing the input layer down to 0 . 0005 .
This reduction refers to one out of 1 , 728 parameters which clearly cannot yield any meaningful prediction.
Additionally, we extend to initialize
BCS-P with LAMP non-uniform strategy (cf.
Table 8).
Similar to the effect on H YDRA , BCS-P significantly benefit from the non-uniform initialization. Especially, the plight of BCS-P at sparsity 99 .
9 % is solved.
However, H YDRA shows a better adaptability to LAMP’s initialization. At the same time, H ARP remains overall the best performance on both 99 % and 99 . 9 % sparsity.
[Figure] [Figure]
[Table]
Comparison to MAD.
We maintain the same training setting in Fig.
Similarly, MAD presents a promising performance on SVHN (cf Fig.
8) to further enhance adversarial resistance by mod- erately pruning.
At the same time, H ARP presents robustness improvement as well in moderate pruning.
In distinction from MAD, however, H ARP remains the stable performance on pruning a higher sparsity while MAD performs less efficiently at aggressive pruning
[Figure]
Comparison to DNR.
Kundu et al.
(2021) propose a robust pruning framework, DNR, to learn compact and robust neural network through dynamic network rewiring.
To conduct the comparison, we follow the authors training setting (Rakin et al., 2018) and use the authors evaluation metrics.
Results are reported in Table 9.
At a high compression ( 50 × )
H ARP plays its strengths.
For CIFAR-10, our method yields robustness scores comparable to DNR at a compression of ≈ 20 × .
The natural accuracy is slightly inferior, though.
For the large-scale dataset, Tiny-ImageNet (Le & Yang, 2015), in turn, H ARP yields higher robustness than DNR for compression up to 50 × , but falls behind in natural accuracy again.
In summary, H ARP yields higher robustness while DNR seems favors natural accuracy.
[Table]
A. E XTENDED S TRATEGY A NALYSIS
In this section
[Table]
we investigate the impact of non-uniform strategies on robust weight pruning.
Firstly, we compare different strategies used to initialize H ARP
’s pruning in order to present the insensibility on H ARP ’ layer-wise rate initialization.
In the next step
[Table]
we analyze the end-to-end learning process with regard to the stability of training with and without using non-uniform strategies on H YDRA .
Secondly
[Table]
we assess H YDRA ’s performance by using ERK and LAMP for ImageNet, demonstrating that the positive impact of the strategies’ non-uniformity is not consistent for different datasets.
Analysis of H ARP ’s strategy initialization.
As demonstrated by Sehwag et al.
(2020), score initialization is key to realize a improvement on the weight pruning.
In Fig.
[Table] we provide the learning processes of H ARP ’s pruning at sparsity that are initialized by ERK, LAMP and the default uniform strategy.
In comparison with ERK and LAMP, uniform strategy leads to a dramatic performance degradation after initialization.
However, the training on scores S W and rates r shows the cape of [Figure] recovering the network performance, resulting in a largely similar network performance to ERK and LAMP after the first epoch training.
In the subsequent epochs, the learning curves on different initializations are essentially overlap.
By further evaluating the final pruned networks after finetuning, results in Table 10 show that the difference between uniform adn non-uniform strategies are negligible for the layer compression rate initialization.
Confirming, that H ARP is insensitive to different strategies initialization.
Therewith, we rely on the uniform strategy as the default setting in our experiments.
[Table]
Comparison of end-to-end compression pipelines.
In Fig.
10, we presents a comparison between different strategies on H YDRA and H ARP across the end-to-end compression pipeline.
For pruning 99 % weights, the potential of the original H YDRA is impaired by the use of a uniform strategy in the pruning phase, limiting the potential for improving via fine-tuning.
At the same time, the non-uniform strategies ERK and LAMP prove their positive impact in the pruning stage.
Without fine-tuning, they achieve already better results than H YDRA .
In H ARP ’s pruning, we observe a down-up performance curve, indicating that the network performance recovers after the robust networks is compressed.
Further improved by fine-tuning, H ARP achieves slightly better results than
H YDRA with ERK and LAMP.
In pruning a 99 . 9 % sparsity, H YDRA produces a dramatic decrease in performance that can, however, be compensated by using LAMP and ERK.
Using H ARP during pruning results in the highest network performance, approximating the final result after fine-tuning.
Comparing non-uniform strategies for ImageNet.
Presented in Table 6, H YDRA is significantly improved by applying ERK and LAMP.
In Table 11, we further investigate their impact on the largescale dataset ImageNet.
Compared to original H YDRA , using ERK and LAMP strategies leads to a performance degradation, and with LAMP yields more harm than ERK at sparsity 99 % .
Fig.visualizes the strategies of sparsity 99 % for ImageNet experiments.
Similar to Fig.
4, LAMP and
ERK tend to preserve parameters, leading to a global parameter distribution that approximates a uniform shape.
However, since H ARP preserves more parameters in the former layers and the last fully connected layer, more input and output information are preserved. [Figure] [Table]
9.51 (−1.81) 8.42 (−2.90) 12.42 APGD 31. 20.38 (−2.59) 20.52 (−2.45) 25. 10.12 (−2.40) 9.56 (−2.96) 13.47 AA 28.
17.42 (−2.34) 16.95 (−2.81) 22.
8.64 (−1.37) 7.78 (−2.23) 11.
[Figure] A. P ARAMETER D ISTRIBUTION H ARP prunes neural networks by incorporating connection importance-scores for learning pruning masks and the layers’ compression rates into the optimization procedure.
The pruned models particularly benefit from the non-uniformity of H ARP ’s compression strategy, yielding high natural accuracy and high adversarial robustness.
Ye et al.
(2019) argue that wider parameter distributions promise higher robustness, while Sehwag et al.
(2020) show that parameters close to zero are crucial to preserve the adversarial robustness.
In this section, we take VGG16 learned on CIFAR-10 and
SVHN as examples to analyze the difference in parameter distributions along the network’s layers and compare H ARP with Robust-ADMM and H YDRA .
Parameter distribution when pruning weights.
Fig. 12a and Fig.
12b visualize the layer-wise weight distribution of VGG16 pruned by Robust-ADMM (blue), H YDRA (green), and H ARP (orange) on datasets CIFAR-10 and SVHN, respectively.
Concatenated architectures such as VGG possess relatively high sparsity in the middle layers (Lee et al., 2021;
Evci et al., 2020).
This phenomenon is also captured by H ARP , which learns to preserve the first six layers and the last layer.
[Figure] Another intriguing but interesting observation concerns non-zero parameters: Robust-ADMM tends to favor middle layers that contain many zero parameters.
In contrast, H YDRA tries to push the model’s weights “away” from zero, which reduces sparsity and thus achieves higher robustness than
Robust-ADMM.
Pruning weights with H ARP , however, converges to a different distribution.
All but the first and last layers, have weights with (mostly) smaller magnitude than for models pruned with H YDRA —few of them, however, remain zero-valued.
For instance, in the penultimate layer, preserved weights are close to zero and the distribution is more compact than those of H YDRA and
Robust-ADMM.
By comparing the parameter distribution of middle layers in CIFAR-10 and SVHN, we observe that H YDRA tends to have a rather close-zero distribution in pruning networks for SVHN than its behavior in CIFAR-10.
Recall Table 2 and Table 3, the performance gap between H YDRA and H ARP in SVHN is smaller than in CIFAR-10.
This implies an interesting phenomenon that encouraging a close-zero distribution yields a positive impact on pruning robust networks.
A. P ERFORMANCE WITH NATURAL TRAINING
We extend our experiments to pruning naturally pre-trained networks with datasets CIFAR-10 and
SVHN, and consider H YDRA , H YDRA with LAMP, and the original LAMP pruning in the comparison.
In Table 12, there exists no significant difference between the pruning methods at sparsity 90 % .
For CIFAR-10, H YDRA yields a larger performance drop at sparsity 99 % .
In contrast, using
LAMP directly or using H YDRA with LAMP’s strategy yields better results.
With regards to the highest sparsity 99 . 9 % , LAMP pruning and H YDRA with LAMP strategy have significantly better performance than original H YDRA .
Meanwhile, H ARP shows the highest compatibility to prune networks for CIFAR-10.
Additionally, in pruning a sparsity 99 .
9 % for SVHN, H ARP presents the pruning performance that deviates from the best result by no more than 0 .
Conclusively, pruning naturally pre-trained networks benefits more from the layer-wise specific non-uniform strategies.
At the same time, H ARP offers more significant improvement about concerning pruning robust networks than other approaches.
[Table]
A. E XTENSION TO C HANNEL P RUNING
Structurally pruning neural networks has better compatibility with the hardware deployment, while its coarse pruning granularity leads to larger performance degradation than weight pruning, itself increasing the difficulty of pruning robust networks.
In this section, we extend our method H ARP on pruning network layer channels.
To map H ARP on channel pruning, we use FLOPs estimation (denoted as function FLOPS ( · ) ) to replace the sparsity in L hw (expressed as Eq.
Moreover, we determine a t by considering network global FLOPs reduction, in order for the pruned model to have the same size as a uniform strategy would have in FLOPs. [Equation] 
[Equation]
Different from weight pruning, importance scores S ( l ) originates R c ( l ) i for channel pruning.
In addition, we use Eq.
(9) to initialize S ( l ) in channel pruning and follow the order of weight magnitude (Ye et al., 2019) by summing up the weights of each input channel (expressed as csum( · ) ) and normalizing by the maximal channel-wise summation.
[Equation]
Algorithm 1 presents the complete implementation of H ARP .
Note that weight pruning is largely independent of the network architecture as it does not change the layer’s size (we merely zero out weights).
However, pruning channels/filters requires special attention.
When facing ResNet-like architectures, we do not train the compression rates of all shortcut layers directly but update them by assigning the rate of the connected input layer in the residual block instead.
This way, the pruned input layer aligns with the channel dimensionality of connected pruned shortcut-layers.
Comparing channel pruning with related work.
Ye et al.
(2019) have shown the capability of
Robust-ADMM in structurally pruning robust networks, controlling the network FLOPs straightforwardly by a global structural compression rate.
Similarly, H YDRA (Sehwag et al., 2020) extends to structural pruning and improves over magnitude-based criterion (Han et al., 2015).
For channel pruning with H ARP , we start off from the uniform strategy with a init = 1 .
and use γ = 0 .to
ensure the arrival at the target compression rate a t .
As presented experiment results in Fig. 13,
H ARP shows promising performance in preserving a robust model while pruning channels.
RobustADMM possesses a similar performance on pruning up to 4 .
xFLOPs, while experiencing higher model performance degradation at xFLOPs of 10 and 20 .
Channel pruning with H YDRA , in turn, significantly harms model performance at 2 .
xFLOPs compression.
[Figure]
Comparison to ANP-VS.
Madaan et al.
(2020) propose to suppress the latent feature level vulnerability in neural networks by gradually pruning a model {+during training based on a regularized loss function. In the open-source implementation of ANP-VS 1 , pre-processing CIFAR-10 data is done using image standardization (e.g., tf.image.per_image_standardization() function). Standardization transforms the input to have mean 0 . 
+} and variance 1 . 0 , that is, input features may fall outside of an [0 . 0 , 1 . 0] interval.
However, ANP-VS clips adversarial perturbations to [0 . 0 , 1 . 0] , resulting in overall weaker adversarial examples used for the evaluation as it would be the case for 0-1 rescaling the inputs.
Note, the latter is used for evaluating H ARP in all experiments.
We investigate the influence of input ranges by comparing training with ANP-VS using both pre-processing
[Table] steps (standardization and simple 0-1 re-scaling) under the otherwise exact same PGD settings. The original publication reports an original accuracy of 88 . 18 % adversarial accuracy of 56 . 21 % for VGG on CIFAR-10, which match our reproduced results with standardization (cf.
Table 13).
We even yield higher compression in our experiments. With the correct input pre-processing step (0-1 rescaling), however, ANP-VS yields a significantly worse result, showing the criticality of wrongly used preprocessing. With thus refrain from comparing H ARP to ANP-VS in pruning channels. Input: pre-trained model and its parameters θ , number layers L , target compression rate a t , minimal compression rate a min , training data-set D
Output: Compressed network (cid:101) θ ⊙ M ∗
Step 1: Initialization of trainable pruning parameters
For each layer l , initialize learnable rates ( l ) = log (cid:18) [Equation] init − [Equation]
min
[Equation]
(cid:19) and importance scores:
[Equation]
Step 2: Strategy search via robust training
[Equation]
, with M = ( M (1) , . . .
, M ( l ) , . . , M ( L ) ) and M ( l ) : = (cid:16)
[Equation]
P
[Equation]
Step 3: Fine-tuning of the pruned network ˆ θ = θ ⊙ M
[Equation]
A. T RAINING CONSUMPTION
In this section, we elaborate on the training (time) consumption of the pruning approaches considered in our evaluation.
As every method operates on the complete training dataset, we estimate the consumption in training epochs rather than wall-clock time as the latter varies with the used hardware and its load.
We report the results in Table 14.
The methods either follow a three-stage pipeline (Han et al., 2015) or train the model from scratch with pruning considerations in their optimization.
Consequently, the distribution of training effort per stage varies.
Under this metric, MAD is most efficient.
However, when targeting a sparsity of 99 % , H ARP arrives close its to best performance right after pruning ( 20 epochs) as can be seen in Fig.
At 99 . 9 % sparsity, our method uses 100 epochs for fine-tuning to recover natural accuracy but does not improve robustness anymore.
Thus, H ARP ’s pruning stage turns out to be most decisive but helps for aggressive pruning.
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
A. CO 2 E MISSION
We have conducted all our experiments on Nvidia RTX-3090 GPU cards and have consumed about 10 , 198 GPU hours in total.
This amounts to an estimated total CO 2 emissions of 2 , 177 . 27 kgCO2eq when using Google Cloud Platform in region europe-west3 .
However, our university consumes 100 % renewable-energy, such that our specific CO 2 emissions for the project amounts to 1 . 74 kgCO2eq .
Estimates are conducted using the “Machine Learning Impact Calculator” (Lacoste et al., 2019).