FaShapley: Fast and Approximated Shapley Based Model Pruning Towards Certifiably Robust DNNs
Abstract —Despite the great success achieved by deep neural networks (DNNs) recently, several concerns have been raised regarding their robustness against adversarial perturbations as well as large model size in resource-constrained environments.
Recent studies on robust learning indicate that there is a tradeoff between robustness and model size.
For instance, larger smoothed models would provide higher robustness certification.
Recent works have tried to weaken such a tradeoff by training small models via optimized pruning.
However, these methods usually do not directly take specific neuron properties such as their importance into account.
In this paper, we focus on designing a quantitative criterion, neuron Shapley, to evaluate the neuron weight/filter importance within DNNs, leading to effective unstructured/structured pruning strategies to improve the certified robustness of the pruned models.
However, directly computing Shapley value for neurons is of exponential computational complexity, and thus we propose a fast and approximated Shapley (FaShapley) method via gradient-based approximation and optimized sample-size.
Theoretically, we analyze the desired properties (e.g, linearity and symmetry) and sample complexity of FaShapley.
Empirically, we conduct extensive experiments on different datasets with both unstructured pruning and structured pruning.
The results on several DNN architectures trained with different robust learning algorithms show that FaShapley achieves state-of-the-art certified robustness under different settings.
I. I NTRODUCTION
Deep neural networks (DNNs) have been deployed in a variety of real-word applications such as medical imaging analysis and language understanding [13, 57].
However, several studies have shown that DNNs are vulnerable to adversarial attacks, which are stealthy to humans but very effective in misleading machine learning models [3, 8, 14, 40, 41, 55, 63, 71, 80].
Therefore, it is critical to understand, evaluate, and most importantly, certify the robustness of DNNs before massive production and deployment of safety-critical applications.
Meanwhile, an almost equally important factor before realworld deployment of DNNs is the model size , especially in resource-limited environments [5, 49].
Recent works show that there is a tradeoff implicitly between these two factors (i.e., robustness and model size): oversparsified DNNs are vulnerable [23], and over-parameterization is important for achieving decent robustness [47].
For instance, empirically, models with larger size would achieve higher robustness after adversarial training [76].
Similarly, the certified robustness of DNNs trained via randomized smoothing also indicates that large models with 13 × more parameters would achieve 10% better certification robustness than the smaller counterpart [10, Section 4]. 99 Pruning
[Figure] [Figure]
99 Pruning Ratio
[Figure]
[Figure] To weaken such a tradeoff in practice, several efforts have been made on model pruning for model robustness [23, 46, 56, 59, 76], but these methods do not directly take specific neuron properties such as their importance into account.
Note that compared to empirical robustness [7, 21, 47], the certified robustness [10, 17, 79] can provide a lower bound of prediction accuracy under arbitrary attacks under certain conditions.
In particular, deterministic verification [ ? ] provides a lower bound to the certified accuracy with the guarantee of no false positive predictions and is more meaningful on pruned networks with bound propagation based certification approaches, and therefore we focus on deterministic verification and evaluate on standard model architectures following the literature [64, 73, 78].
In this paper, we aim to ask: Can we design an efficient criterion based on neuron properties to evaluate the importance of neuron weights and filters for unstructured and structured pruning, respectively?
Can we improve the certified robustness for the pruned models with smaller size to relax the tradeoff?
Shapley value (SV) [60] has been successfully applied to a variety of ML tasks, including data summarization, data evaluation, and active data acquisition [18, 28, 29] due to its ability to evaluate the importance of data samples.
In this work, we aim to leverage the power of SV to evaluate the neuron weight/filter importance towards the certified robustness.
Concretely, we view the problem of model pruning as a cooperative game with the objective of achieving high certified robustness, and each weight or filter is viewed as a player in the game.
Recent works [1, 2, 19, 20, 20] apply SV to neuron importance evaluation, but they only explore filterlevel evaluation and do not consider the certified robustness of models.
In contrast, our proposed FaShapley enables SV to be applied to weight-level evaluation to prune models towards high certified robustness.
In our work, the SV of each neuron weight/filter is calculated as the criterion to evaluate the “non-important” ones for pruning.
The criterion has four advantages: (1) desired properties (linearity, symmetry, nullity, efficiency), (2) awareness of the certified robustness objective by using specific utility functions, (3) effectiveness due to the consideration from a global view, (4) interpretability as a neuron selection criteria.
However, there are additional challenges of directly utilizing SV for model pruning towards certified robustness, 1) exponential computational complexity and 2) less effective without enough sampling .
Calculating the exact SV is a #Pcomplete problem, and therefore it is usually approximated with a sampling-based method [18].
Even so, when SV is applied to unstructured pruning for models with millions of neurons, it is impossible to calculate the value of every weight with feasible computation.
To reduce the computational costs, we propose a fast and approximated SV method (FaShapley), leveraging the neuron properties by multiplying the norm of weights with gradients which is able to scale up to weight-level evaluation for unstructured pruning.
Besides reducing computational complexity, FaShapley also demonstrates effectiveness on model pruning towards certifiably robust DNNs with our sample-size optimization.
As shown in Figure 1, it is clear that
FaShapley achieves much higher certified accuracy than the state-of-the-art algorithm HYDRA [59].
In addition, we theoretically analyze (1) desired properties of our FaShapley, and (2) sample complexity of our method.
We prove that the sample complexity of FaShapley is a function of the sample-size (i.e., size of the sampled subgroup of neurons), and based on the empirical observations of model pruning literature, we conclude that our bound should be tight for general DNNs.
Technical Contributions.
In this paper, we take the first step towards designing an effective neuron importance evaluation method for model pruning to achieve high certified robustness.
We make the following contributions
• We propose a fast and approximated Shapley method (FaShapley) for model pruning to achieve high certified robustness for small models (e.g., up to 99% pruning), which is applicable to both unstructured and structured pruning.
• Theoretically, we analyze the desired properties of FaShapley (e.g. linearity and symmetry) and its sample complexity.
We also demonstrate the efficiency of FaShapley compared to existing Shapley approximation.
• Empirically, we conduct extensive experiments on three datasets with both unstructured pruning and structured pruning.
The results on three DNN architectures trained with different robust learning algorithms show that FaShapley achieves the state-of-the-art certified robustness under different pruning ratios (e
g., FaShapley achieves up to 35.8% higher certified robustness than baselines under 99% pruning on CIFAR-10 via CROWN-IBP verification).
Model pruning aims to compress neural networks for deployment in resource-constrained environments.
Network pruning can generally be categorized into unstructured and structured methods.
Unstructured methods [22, 24, 25] prune unimportant weights in the model, while structured pruning [34, 68] methods remove the whole filters or channels with low importance.
There have been pioneering works [11, 12, 23, 38, 46, 54, 56, 59, 72, 76] that explore the potential of pruning on top of robust DNNs, but this line of research mainly focuses on the training strategy and does not directly take specific neuron properties such as their importance into account.
In the literature of model pruning, there are different effective criteria designed to evaluate the importance of neurons using norm of weights, gradients, or
BN scaling factors [26, 36, 39, 44, 62].
Compared to these criteria, FaShapley is motivated by powerful Shapley value in cooperative game theorey and is more explainable by using the expected marginal contributions of neuron weights/filters as the importance score.
Therefore, FaShapley guarantees desired properties such as linearity and symmetry as an evaluation criterion and shows significant performance improvements over the heuristic criteria.
Shapley value (SV) [60] is a classic concept in cooperative game theory and demonstrates great power in a range of
ML tasks including data summarization, data evaluation, and active data acquisition [18, 28, 29].
However, SV calculates the marginal contributions of players based on all combinations and requires exponential computational costs.
To overcome the challenge of large computational costs of calculating the exact Shapley value, existing works approximate the value based on Monte Carlo sampling [18] or properties of utility functions [15].
Recent works [1, 2, 19, 20] leverage SV to evaluate the neuron importance, which differ from ours in two perspectives, (1) they do not explore weight-level importance evaluation, (2) they do not apply it to achieve certified robustness on pruned DNNs.
In this paper, we propose a fast
Shapley approximation based on the gradients and optimized sample-size for effectiveness and further computational costs reduction.
We take the first step to leverage SV for weight-level evaluation and unleash its power for model pruning towards certified robustness.
Certified robustness provides provable robustness guarantees under a given perturbation budget.
Certifiably robust approaches include robust training and verification methods [4, 10, 42, 43, 61, 67, 74, 75].
A recent survey categorizes verification approaches into deterministic and probabilistic verification [ ? ].
Both lines of research are faced with the challenge of high verification costs due to millions of neurons in DNNs.
In particular, deterministic verification [ ? ] provides a lower bound to the certified accuracy with the guarantee of no false positive predictions and is more meaningful on pruned networks with bound propagation based certification approaches, which are the focus in this paper.
Specifically, we mainly evaluate certified robustness against ℓ ∞ -bounded perturbations, which is a particularly challenging [6, 32] and commonly used threat model, and the methods for this threat model are extensible for handling other threat models [50, 69, 70].
Our proposed
FaShapley pruning algorithms can be directly leveraged on certifiably robust DNNs and weaken the tradeoff between robustness and the model size.
III. F A S HAPLEY F OR M ODEL P RUNING
In this section, we introduce our proposed FaShapley and the complete algorithm, leveraging which we achieve high certified robustness after model pruning.
Section III-A formulates the problem of model pruning to achieve certified robustness.
Section III-B introduces our sample-size optimization and the fast approximation of Shapley based on weights and gradients.
Section III-C illustrates the complete pruning algorithm to achieve certified robustness for unstructured and structured pruning.
A. Problem Formulation
Given a training dataset D and a network with weights θ , the goal of our model pruning is to prune the least important weights (or filters) and maintain high certified robustness.
Similar with [59], we introduce a binary mask m b in which each bit corresponds to a neuron weight or filter and all bits in m b are initialized as 1.
One weight (or filter) can be pruned by setting its mask to 0.
Therefore, the objective of unstructured pruning can be formulated as follows:
[Equation]
where x and y denote the images and labels in the dataset
D .
θ ⊙ m refers to the element-wise multiplication of weights θ with the mask m b , L cer is the loss function of the certifiably robust training approach, p is the pruning ratio, N is the number of weights, and | m b | is the ℓ 0 norm of m b .
For structured pruning, the only difference is that m b in Equation 1 has an extra constraint that the masks corresponding to one filter must be set to 0 or 1 simultaneously.
B
FaShapley
Based Importance Evaluation
Shapley value (SV) has been applied to different domains given desirable properties (linearity, symmetry, nullity, efficiency) and interpretability.
However, computing SV is of exponential computational complexity and thus calculating SV exactly is infeasible.
Even with sampling-based approximation, we show that approximating the marginal contributions of millions of neurons weights still cannot be completed in reasonable time.
Directly applying SV to importance evaluation also introduces effectiveness concerns especially for a small sample-size which is illustrated in detail in this section.
To solve these challenges, we propose 1) sample-size optimization to make the calculation effective , and 2) a gradient-based optimization strategy to approximate the marginal contribution of neurons efficiently .
Sample-Size Optimization.
To evaluate the importance of neuron weights or filters, we can view each weight or filter as a player in the cooperative game and calculate their SV as pruning criteria.
Suppose that θ = { θ i } Ni =1 is the weights of a neuron network and U ( S ) is the utility function, representing the value calculated based on the additive aggregation of { θ i } i ∈ S and
S ⊆
I = { 1 , ..., N } .
The SV s i ( U ) of weight/filter i can be formulated as follows:
[Equation]
To make the evaluation aware of the training objective to achieve certified robustness, we utilize the negative robustness loss − L cer ( θ, x, y ) as the utility function U .
Note that the power of SV as an importance evaluation criterion lies in the fact that it aggregates the marginal contribution considering all combinations for the rest of the players (i.e., weights/filters), which also leads to the high computational complexity.
Let k = | S | be the sample-size , the calculation of SV views the importance scores of players from a global view by traversing all possible sample-sizes (i.e., all integers in the range [0 , N − 1] ):
[Equation]
where s ik ( U ) denotes the marginal contribution of player i given a fixed sample-size k based on utility function U .
Note that the sample-size k belongs to the set { i } N − 1 i =0 .
From Equation 3, we can see that the SV calculation takes the average of the terms s ik ( U ) with different sample-sizes.
In addition, we observe that the term s ik ( U ) with a relatively large sample-size k plays a more effective role in the evaluation.
The results that confirm our hypothesis are provided in Table XVII.
There are two possible reasons for such observation: (1) s ik ( U ) with a large sample-size k introduces more fair and stable comparisons since this way the expectation of the size of the intersection set of sampled subsets S is larger ; (2) the utility of a small number of players is usually low, thus making the marginal contribution calculated based on it less useful.
In light of this observation, we calculate s i k ( U ) with a large sample-size k to evaluate the importance of neuron weights/filters.
Note that if the sample-size k is set N (the number of neurons), it is reduced to Leave-One-Out (LOO) [29].
LOO is shown to lose the advantage of considering the importance from multiple combinations, which enables a much better evaluation from a global view (more detailed evaluations provided in Table IX and Table X).
Without special specification in the experiments, we fix the sample-size k as ⌈ 0 .
9 N ⌉ in this work.
Fast and Approximated Shapley Calculation.
The challenge in calculating SV lies in its exponential computational costs.
Calculating the exact SV in Equation 2 involves computing the marginal utility of every neuron for each subsample, which is O (2 N ) with N the total number of neurons.
To overcome this, existing works calculate the approximation of the value based on Monte Carlo sampling [18], or design algorithms for the special properties of utility functions [15].
In this work, we propose to leverage the norm of the multiplication of weights and gradients to approximate the marginal contribution of weights/filters as follows:
[Equation]
In the Taylor expansion of the loss function L , we approximate by only taking the first-order terms into consideration and such approximation is also leveraged in [35].
Since the marginal contribution of a neuron weight is the difference of utility before pruning it and utility after pruning it, we have: | dθ i | =
| θ i − 0 | = | θ i | .
In this way, we can calculate the approximation of the marginal contribution of all neurons in one backward pass , in contrast to the standard SV calculation where the marginal contribution of one neuron requires two forward passes.
Thus, FaShapley is applicable to the weightlevel importance evaluation.
C. FaShapley Based Model Pruning
We design a three-step model pruning pipeline based on FaShapley to achieve certified robustness: pre-training a network with robust training methods, model pruning, and finetuning (FT).
The pre-training step generates DNNs based on specific loss functions.
The pruning step masks out a particular ratio of weights or filters and attempts to maintain the certified robustness to the largest extent.
The fine-tuning step refines the remaining weights/filters to further minimize the loss.
Our proposed FaShapley serves as an effective criterion to be applied in the pruning step.
Concretely, FaShapley assigns each weight an importance score and next the weights with small importance scores are pruned.
In algorithm 1, we provide the detailed steps to calculate the importance of each weight and prune the model with a predefined pruning ratio via unstructured pruning.
V stores
FaShapley values.
In one iteration, it is updated by the norm of the multiplication of weights and gradients with a randomly selected group of weights being masked out.
Finally, the weights with low FaShapley values are pruned.
For structured pruning, the only difference is that each filter is viewed as a unit for evaluation.
IV. T HEORETICAL A NALYSIS FOR F A S HAPLEY
Shapley value’s power of contribution evaluation benefits from its four desired properties (linearity, symmetry, nullity, efficiency).
However, SV fails to be exactly calculated for millions of neuron weights due to large computational costs, thus inducing extensive consideration of sample complexity for SV approximation methods.
In this section, we provide theoretical analysis on these desired properties of our proposed
FaShapley approximation as well as its sample complexity.
From the analysis, we conclude that (1) FaShapley can reserve four desired properties of SV (Section IV-A), (2) FaShapley demonstrates lower sample complexity than existing samplingbased SV (Section IV-B).
Input: model M , data loader , sample complexity m , samplesize k , neuron weights players of size N , loss function L cer , number of pruning units p
Output: model M o
Initialize V [ ] = 0 { store FaShapley values of neuron weights } for x , y in loader do
Initialize times =repeat times = times +
S = sample( players , N − k ) { randomly sample (N-k) neuron weights to be pruned }
M’ =
prune( M , S ) { prune the sampled neuron weights
S in model M } pred =
M’ ( x ) loss =
L cer ( pred , y ) loss .backward( ) for player in players do
V [ player ] += ∥ player .gradient ∗ player .weight
∥ end for until times = m end for
M o = prune( M ,argsort( V )
[: p ]) { prune based
on
V }
A. Desired Properties of FaShapley
It is proved that SV (including its continuous variants like Aumann-Shapley and Serial Cost methods) is the only way of assigning attributions to players that satisfies four properties [60]: linearity, symmetry, nullity, and efficiency.
These four desired properties are defined as follows: (1) linearity: if the utility function can be seen as a linear combination of two other utility functions, then any SV should also be a linear combination of the SVs computed with other utility functions; (2) symmetry: if the utility function value depends on two players but not their order, then they receive the same SV; (3) nullity: if the utility function value does not depend on a particular player, then its SV is zero; (4) efficiency: the overall reward can be assigned to all players.
We theoretically prove that expectation of FaShapley value can preserve these properties.
In other words, we do not consider the negligible error induced by the gradient-based approximation.
Formally, we present the FaShapley of player i with samplesize k as
[Equation]
where S is the randomly sampled group from all players
I =
{ i } Ni =1 and U ( S ) denotes the utility function.
Theorem 1.
FaShapley F i ( U, k ) of player i with sample-size k satisfies four desired properties as follows:
(1) linearity:
if U ( S ) = t 1 U 1 ( S ) + t 2 U 2 ( S ) , then F i ( U, k ) =
t 1 F i ( U 1 , k ) + t 2 F i ( U 2 , k ) .
(2) symmetry: if ∀ S ⊆ I \{ i, j } and | S |
= k, U ( S ∪ { i } ) =
U ( S ∪ { j } ) , then F i ( U, k ) =
F j ( U, k ) .
𝑝
N 𝑂
U 𝑡𝑖𝑙𝑖𝑡𝑦 𝑈(𝑆) 𝑈 !"
# 𝑈 𝑆 = 𝑏|𝑆|
𝑈 𝑆 = 𝑎|𝑆|
[Figure]
3) nullity: if ∀ S ⊆
I \{ i } and | S |
=
k, U ( S ∪ { i } ) =
U ( S )
+ U ( { i } ) , then F i ( U, k ) =
U ( { i } ) − U ( { ϕ } )
(4) efficiency: (cid:80)
[Equation]
i
[Equation]
Proof Sketch.
For the proof of linearity, we can decompose the utility function in the definition of FaShapley with the linear combination of two utility functions and reorder the expression to compose FaShapley regarding the new utility functions.
For the proof of symmetry, we only need to apply the identity of utility functions regarding player i , j to the definition and directly get the identity of FaShapley.
For the proof of nullity,
U ( S ∪ { i } ) can be equalized to the addition of utility and the point is that the utility of the empty set ϕ is 0.
For the proof of nullity, the key observation is that the expectation of the utility U ( S ∪ { i } ) where S is sampled from I/ { i } with size k connects to U ( S ) where S is sampled from I with size k +with a scaling factor N .
The complete proof is provided in
Appendix B.
Remark.
These properties of FaShapley benefit neuron importance evaluation towards certified robustness.
Linearity indicates that the utility can be viewed as a linear combination of classification utility (e.g., CE loss) and robustness utility (e.g., robust loss).
Symmetry suggests that neurons with the same utility receive the same FaShapley value.
Nullity indicates neurons with no contribution receive zero value.
Efficiency guarantees that the total utility is distributed to all neurons.
B. Analysis of Sample Complexity
Direct Shapley calculation fails to be scaled up to weightlevel evaluation on a large architecture and the sample complexity of approximating Shapley value is of great importance.
Given the nice structure of FaShapley, here we present the theoretical analysis of its sample complexity and demonstrate the efficiency of FaShapley compared to the naive samplingbased Shapley calculation.
When applying FaShapley to neuron evaluation on a model, we use the utility function − L cer ( θ, x, y ) where
L cer ( θ, x, y ) is loss function of robust learning.
Let L max = max( L cer ( θ, x, y )) be the maximum value of the loss function.
For simplicity, we denote U ( θ ) =
− L cer ( θ, x, y )
+ L max as the utility function to make the range of utility U be [0 , + ∞ ) .
N denotes the number of neuron weights/filters.
S is the weights/filters sampled from I
= { i } Ni =1 and | S | is the sample-size of set S .
U max = max( U ( S )) , ∀ S ⊆
I refers to maximum utility.
We view FaShapley value F i ( U, k ) as a continuous function given sample-size k ( k ∈ [0 , N − 1]) .
Fact IV.1.
There exist numbers a, b (0 < a < b ) , which satisfy the following inequality,
[Equation]
Remark.
We illustrate the bound of FaShapley in Figure 2.
First, the utility U ( S ) is positively correlated with | S | , and therefore can be bounded using a | S | ≤ U ( S ) ≤ b | S | (Equation (6) in [48]).
Second, U ( S ) has an upper bound as U max and the inflection point p =
⌊ U max /b ⌋ in Figure 2 is relatively small since existing neural network pruning techniques have shown that it is possible to reduce the parameters of trained networks by over 90% without compromising the accuracy [16].
In other words, Fact IV.1 is aligned with the existing empirical observations in literature.
Theorem 2.
With probability at least δ and estimation error ϵ , the sample complexity m for FaShapley is lower bounded as follows:
[Equation]
With probability at least δ and estimation error ϵ , the sample complexity m sv for Shapley value approximation has the lower bound as follows:
[Equation]
Proof Sketch.
We first derive the range of the utility function
U ( S ) regarding the sample-size | S | with a piecewise function according to Fact IV.1 and the illustratetion in Figure 2.
Then we can leverage Hoeffding’s inequality and plug in the range of the utility function U ( S ) , sample complexity m , and the estimation error ϵ .
Solving the inequality, we can finally get the lower bound of sample complexity m for FaShapley.
For direct Shapley value appriximation, the range of the utility function is U max and we can similarly leverage Hoeffding’s inequality to derive the lower bound of sample complexity m sv .
The complete proof is provided in Appendix B.
Remark.
Theorem 2 indicates that (1) FaShapley is more computationally efficient than direct Shapley approximation because the lower bound of m sv in Inequality 8 is larger than that of m in Inequality 7, (2) when sample-size k is smaller than p =
⌊ U max /b ⌋ , the sample complexity is positively correlated with the sample-size k ; while if the sample-size k exceeds p , the sample complexity is negatively correlated to the samplesize k .
This phenomenon is very interesting, since intuitively, when the sample-size grows up to p , there is more and more information involved, while after the sample-size exceeds p , there is a large overlap between samples and therefore the information gain is limited among samples.
We believe our theoretical analysis provides another novel perspective for the meaning of p , which represents the standard empirical “optimal” pruning ratio in model pruning literature [22, 24, 25].
A. Experimental Setup
Dataset.
We conduct extensive experiments on three datasets:
CIFAR-10 [31], SVHN [53], and Tiny-ImageNet [33].
We use
CIFAR-10 and SVHN for fair comparisons with the SOTA certified robustness pruning method [59].
We further evaluate on Tiny-ImageNet to perform evaluation on a large-scale dataset (notice that existing deterministic robustness verification methods cannot scale up to ImageNet).
CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes including transportation and animals, with 6000 images per class.
SVHN dataset is used for digit recognition in the real-world scenario and incorporates extensive labeled data (over 600,000 digit images).
Tiny-ImageNet is a subset of the ImageNet dataset and contains 100,000 images of 200 classes (500 for each class) downsized to 64×64 colored images.
Model Architecture.
Our proposed FaShapley based pruning algorithm is evaluated for three verifiable robust training methods: CROWN-IBP [78], MixTrain [64], and Auto-LiRPA [73].
For CROWN-IBP and MixTrain, we select CNN-4 (0.3M params) and CNN-7 (2.2M params) [70] as the backbone, while for Tiny-ImageNet, WideResNet 28-10 [77] (36.5M params) is selected.
The results on three architectures under multiple pruning budgets demonstrate the effectiveness of FaShapley towards achieving certified robustness on pruned networks.
For CIFAR-10 and SVHN, we evaluate on CNN-4 and CNN7 following HYDRA [59] for fair comparisons.
For TinyImageNet, we evaluate on WideResNet 28-10 network mainly because the SOTA verification LiRPA achieves the highest certified robustness with the type of network (Table 2 in [73]), and thus we use the same architecture for fair comparison purposes.
Note that Tiny-ImageNet with WideResNet 28-10 is the large-scale evaluation we can achieve with deterministic verification, which requires the bound propagation in neurons and is extremely time-consuming.
In this paper, our aim is to reduce the computation costs and memory overhead through pruning on different model architectures viable with deterministic verification.
The details of the model structures are formulated as follows:
• CNN-4: 2 Conv-ReLU layers with { 16 , 32 } filters, 2 linear layers with { 2048 , 100 } neurons.
• CNN-7: 4 Conv-ReLU layers with { 32 , 32 , 64 , 64 } filters, 3 linear layers with { 4096 , 512 , 512 } neurons.
• WideResNet 28-10: 3 wide basic blocks ( 6 Conv-ReLUBN layers) with widen factor = 10 and an additional linear layer with 512 neurons.
Evaluation Metrics.
The ℓ ∞ perturbation budget is 2 /for CIFAR-10, SVHN, and 1 / 255 for Tiny-ImageNet.
Both unstructured pruning and structured pruning are explored in all three datasets.
A larger perturbation ℓ ∞ = 8 / 255 in CIFAR10 is also explored for better comparisons.
We use clean accuracy (Cl-Acc) and certified accuracy (Cr-Acc) to evaluate the performance of the network.
Clean accuracy represents the percentage of correctly classified images without perturbation, while certified robust accuracy is the lower bound of the clean accuracy under given perturbation budgets.
Robust Training and Certification Methods.
For verified robust training, we leverage three state-of-the-art approaches: (1) CROWN-IBP [78] which combines the fast IBP bounds in a forward pass and a tight linear relaxation bound CROWN in a backward pass, (2) MixTrain [64] which balances standard accuracy and certified robustness by applying robustnessoriented regularization on selected training inputs, and (3) AutoLiRPA training [73] which optimizes efficiently computable bounds of the training loss.
For robust verification, deterministic verification provides a lower bound to the certified accuracy with the guarantee of no false positive predictions and is more meaningful on pruned networks with bound propagation based certification approaches, and therefore we focus on deterministic verification and evaluate on standard model architectures following the literature [64, 73, 78].
Concretely, we use IBP , symbolic interval analysis , and Auto-LiRPA verification to perform verification of models trained with
CROWN-IBP, MixTrain, and Auto-LiRPA, respectively.
Selection of Pruning Ratios for Fair Comparisons.
To demonstrate the effectiveness of FaShapley compared with
SOTA robust pruning methods and different criteria, we aim to evaluate with a wide range of pruning ratios.
In different settings, we follow the criteria that the comparison is done under an arithmetic sequence of pruning ratios such that
FaShapley achieves comparable certified accuracy and clean accuracy as the pre-trained model for the smallest pruning ratio and the baselines perform poorly for the largest pruning ratio.
Therefore, different methods can be compared in a wide and meaningful range of pruning ratios.
The clean / certified accuracy of pre-trained models before pruning is summarized as follows:
• CNNon CIFARbased on CROWN-IBP: 54 . 0% / 42 .
5% ; CNN-7 on CIFAR-10 based on CROWNIBP:
8% • CNN-4 on SVHN based on CROWN-IBP: 61 .
5% ; CNN-7 on SVHN based on CROWN-IBP: 69 .
9% • CNN-4 on CIFAR-10 based on MixTrain: 62 .
8% ; CNN-7 on CIFAR-10 based on MixTrain:
7% • CNN-4 on SVHN based on MixTrain: 72 .
4% ; CNN-7 on SVHN based on MixTrain:
5% • WideResNet on Tiny-ImageNet based on Auto-LiRPA: 27 . 0% / 15 .
The certified accuracy is computed through IBP, symbolic interval analysis, and Auto-LiRPA verification for CROWNIBP training, MixTrain, and Auto-LiRPA, respectively.
Baselines.
We compare FaShapley with four types of baselines, 1) state-of-the-art model pruning method to achieve certified robustness , 2) different neuron selection criteria in the literature for model pruning, 3) SOTA model pruning method , and 4) different Shapley based pruning approaches .
Concretely, HYDRA [59] is the state-of-the-art algorithm for pruning certifiably robust neural networks.
HYDRA optimizes a binary mask of weights as parameters based on its robustness
[Table] [Table]
objective.
Through extensive experiments on three datasets with different training methods, we demonstrate that our FaShapley based pruning algorithm outperforms HYDRA significantly.
We also compare FaShapley with other criteria used to evaluate the importance of weights/filters in DNNs: weight-based protocol (LWM) [76], gradient-based protocol (Grad) [51],
Taylor-based protocol (Taylor) [35, 52, 58]
[Table]
and stabilitybased protocol (Stab) .
The weight-based protocol selects the importance score as the magnitude of the neuron weights.
The gradient-based protocol selects the importance score as the gradients of weights.
The Taylor-based protocol selects the importance score as the magnitude of the multiplication of the neuron weights and the gradients of weights.
Note that existing studies show that neuron stability is highly correlated with the DNN certified robustness [78], and thus we consider the stability-based protocol, which prunes unstable neurons, as another strong baseline.
Notice that all the importance scores including our FaShapley are applied the criteria of one-shot pruning after convergence for fair comparisons.
Furthermore,
SOTA model pruning methods Flying Bird (FB)
[9] and
BAR [37] are the state-of-the-art unstructured pruning and structured pruning methods, respectively.
We demonstrate that our FaShapley can ourperform them towards achieving certified robustness with both unstructured pruning and structured pruning.
We also compare our FaShapley with different Shapley based pruning approaches, SVPrune [2] and ShapPrune [20], from which we show that only FaShapley can scale up to unstructured pruning and FaShapley also demonstrates significant performance improvements for structured pruning.
Experiment Details.
We leverage a three-step model pruning pipeline involving pre-training a network, pruning it, and later fine-tuning it.
In the pre-training step of CROWN-IBP, we follow the standard setting [78].
We set the scheduling length to 60 epochs, during which we gradually decrease the
[Table]
[Table]
portion of verifiable robust loss obtained by CROWN-IBP while increasing the portion obtained by IBP for each training batch.
For the rest of the epochs after the scheduling epochs, only
IBP contributes to the verifiable robust loss.
IBP is used for evaluation.
In the pre-training step of MixTrain, we use the best training setup [64] for both CIFAR-10 and SVHN.
Concretely, we use sampling numbers 5 and 1 for CNN-4 and CNN-7, respectively.
We select the balance factor α = 0 .
8 to balance between regular loss and verifiable robust loss.
The trained networks are evaluated with symbolic interval analysis [65, 66].
In the pre-training step of Auto-LiRPA, we follow the best training setup [73].
The networks were trained using the Adam [30] optimizer with an initial learning rate of 5 × 10 − 4 .
Also, gradient clipping with a maximum ℓ 2 norm of 8 is applied.
We gradually increase ϵ within a fixed epoch length of 400 .
We uniformly divide the epoch length with a factor 0.4, and exponentially increase ϵ during the former interval and linearly increase ϵ during the latter interval, to avoid a sudden growth of ϵ at the beginning stage.
A hyperparameter β to balance
LiRPA bounds and IBP bounds for the output layer is set and gradually decreases from 1 to 0 ( 1 for only using LiRPA bounds and 0 for only using IBP bounds), as per the same schedule of ϵ , and the end ϵ for training is set to 10% higher than the one in test.
Our proposed FaShapley serves as an effective criterion to be applied in the pruning step.
1 , 000 randomly selected images are used for the calculation of FaShapley values due to computational cost consideration.
We set the samplesize k (normalized to [0 , 1] based on the number of neuron weights/filters) to be 0 .
9 and the sample complexity m to be 30 for experiments on CIFAR-10 and SVHN.
The sample-size k is selected as 0 .
999 for Tiny-ImageNet.
The utility function is set to be the opposite of the loss function for CROWNIBP and MixTrain.
For the model trained with Auto-LiRPA, we simply use the opposite of Cross-Entropy (CE) loss as the utility function for efficiency concern.
To further boost the performance of the pruned model via structured pruning on Tiny-ImageNet, we set a threshold 0 .
35 for all layers to avoid over-pruning in some layers under the pruning ratios of 20% , 25% , and 30% .
In the fine-tuning step, we train the pruned model on CIFAR-10 and SVHN based on CROWN-IBP training for 60 epochs using the learning rate 1 × 10 − 5 .
We train 30 epochs on the pruned models based on MixTrain training using the learning rate 1 × 10 − 5 .
We train 10 epochs with the learning rate 5 × 10 − 4 on Tiny-ImageNet based on AutoLiRPA training.
The codes we used follow the MIT license.
All experiments are conducted on a 1080 Ti GPU with 11, memory.
B. Evaluation Results
Certified Robustness via Unstructured Pruning.
Through extensive experiments for two architectures (CNN-4 and CNN7) on two datasets (CIFAR-10 and SVHN), we demonstrate that our algorithm FaShapley outperforms the state-of-the-art method HYDRA significantly with different robust training methods.
We provide the results of CROWN-IBP and MixTrain robust training methods in Table I and Table II, respectively.
[Table]
The certified accuracy are verified through IBP and symbolic interval analysis, respectively.
To directly compare the effectiveness of the pruning step with HYDRA, we also present the results before fine-tuning, from which we can see that our algorithm can maintain the certified robustness of the pruned model even without fine-tuning for a large pruning ratio such as 95% , while HYDRA relies more on the fine-tuning step to recover the performance (still lower than FaShapley).
Results of pruning on a model trained with CROWN-IBP with a larger perturbation budget ( ϵ = 8 / 255 ) are given in Table IV.
From the results of extensive evaluation via unstructured pruning, we can conclude that 1) FaShapley outperforms the SOTA certified robustness pruning method HYDRA in terms of both certified accuracy and clean accuracy via different certification methods (CROWN-IBP and MixTrain) under different pruning ratios, 2)
FaShapley can maintain the certified robustness of the pruned model even without fine-tuning for a large pruning ratio such as 95% via CROWN-IBP, while HYDRA relies more on the fine-tuning step to recover the performance (still lower than
FaShapley), 3) FaShapley enables the compressed network to achieve comparable or higher certified robustness than the pretrained model (e.g., 95% pruning via CROWN-IBP), and 4) the effectiveness of FaShapley over HYDRA is demonstrated under larger perturbations ( ℓ ∞ = 8 / 255 ).
Certified Robustness via Structured Pruning.
FaShapley can be easily applied to structured pruning for which we only need to treat the filters/channels as the players and evaluate their importance scores.
Then we perform filter and channel pruning based on the importance scores.
To the best of our knowledge, we are the first to provide results of the certified robustness via structured pruning.
As the state-of-the-art algorithm for unstructured pruning, HYDRA also provides the empirical robust accuracy for structured pruning.
Therefore, we choose it as a strong baseline and conduct extensive comparisons on
CIFAR-10 and SVHN based on two architectures (CNN-4 and
CNN-7) and three pruning ratios in each setting.
The results in Table III and Table V show that for structured pruning, our proposed FaShapley can achieve much higher certified robustness compared to HYDRA with different robust training methods (CROWN-IBP and MixTrain) both with and without finue-tuning the weights after pruning.
[Table] [Table]
[Table]
Certified Robustness on Tiny-ImageNet.
We also evaluate on Tiny-ImageNet [33], which is known challenging to be certified under the ℓ ∞ norm bounded perturbations.
In this paper
[Table]
we focus on deterministic verification which provides a lower bound to the certified accuracy and is more meaningful on pruned networks with bound propagation based certification approaches; while HYDRA applies randomized smoothing to large models to get the probabilistic robustness certification.
To the best of our knowledge, existing deterministic verification methods cannot scale up to large-scale datasets such as
ImageNet , so besides CIFAR-10 and SVHN, we evaluate
[Table] [Table]
[Table] [Table]
on Tiny-ImageNet to compare with the SOTA verification results.
Auto-LiRPA is the first work to provide the ℓ ∞ -certified robustness scaling up to Tiny-ImageNet.
We leverage AutoLiRPA as the certification method and take the first step to provide certified robustness for the pruned DNNs on TinyImageNet.
For better comparisons with HYDRA on a larger model and dataset, we conduct experiments using HYDRA and
FaShapley by pruning a pre-trained WideResNet 28-10 network which is the most effective network in this setting.
The results in
Table VI show that for both structured pruning and unstructured pruning
[Table]
FaShapley outperforms HYDRA to achieve high certified robustness on pruned networks on Tiny-ImageNet with and without fine-tuning via auto-LiRPA verification.
In addition
[Table]
we compare FaShapley with a set of neuron importance selection baselines, including the weight-based protocol (LWM), gradient-based protocol (Grad)
[Table]
and Taylorbased protocol (Taylor).
The results in Table VII demonstrate that FaShapley can scale up to Tiny-ImageNet and outperform the strong baselines for unstructured pruning.
The effectiveness of FaShapley on Tiny-ImageNet for structured pruning is demonstrated in Table VIII.
Comparison with Different Neuron Selection Criteria.
In the literature of model pruning, there are many competitive criteria which aim to evaluate the importance of weights/filters in DNNs: weight-based (LWM) [76], L2 norm-based [26],
[Table] [Table]
[Table]
gradient-based (Grad) [51], and Taylor-based (Taylor) [35, 52, 58] protocols.
The weight-based protocol selects the importance score as the magnitude of the neuron weights.
The gradientbased protocol selects the importance score as the gradients of weights.
The Taylor-based protocol selects the importance score as the magnitude of the multiplication of the neuron weights and the gradients of weights.
To demonstrate the effectiveness of FaShapley and differentiate it from score-based neuron evaluation methods, we compare FaShapley with these criteria.
The evaluations of neuron importance with different criteria are all performed after convergence for fair comparisons.
For structured pruning, we further propose a new baseline to consider the stability of neurons , which is an important factor for certified robustness of pruned models.
The new criterion prunes unstable neurons (a neuron is unstable when the post-ReLU value can be both zero and positive for a specific perturbation budget of the input), which do harm to the certified robustness since this type of neurons can loosen the bound in the robustness verification.
The results in Table IX and Table X show that our proposed FaShapley is much more effective for model pruning toward high certified robustness than these baselines for both unstructured pruning and structured pruning.
We think that stability does not bring about expected results due to these two reasons: (1) it does not benefits the clean accuracy of the model since APoZ [27] prunes the neurons with large probability of being zero based on the assumption that these neurons influence little on the final prediction; (2) some unstable neurons could improve the lower bound but are removed in this case.
Note that LWM and L2 norm give the same importance ranking for unstructured pruning, so we only report the results of LWM in Table IX.
Comparisons of different neuron selection criteria on SVHN are provided in Table XI.
Our extensive evaluation on different datasets shows that the effectiveness of FaShapley is significant under different pruning ratios.
We analyze the advantage of FaShapley over Taylor-based criteria in detail as follows.
The Taylor-based pruning criterion is based on the leave-one-out (LOO) strategy and calculates the saliency scores (i.e., the influence of removing one neuron) without sampling.
LOO can be viewed as a special case of
Shapley value and thus the Taylor-based criterion is actually a special case of our proposed FaShapley.
However, with a
[Table]
reasonable sample-size, our FaShapley computes more effective marginal contributions, which consider multiple combinations and enables a more effective evaluation from a global view [29].
We also have a related discussion about the advantages of
Shapley value over LOO in Section III-B, from which we can understand why calculating the saliency scores from multiple combinations is much more effective.
Comparison with the SOTA Model Pruning Algorithm.
We also compare FaShapley with SOTA algorithms for both unstructured pruning and structured pruning in the general pruning field.
Concretely, we select the SOTA unstructured pruning method Flying Bird (FB) [9] and SOTA structured pruning method BAR [37] as our baselines.
Flying bird introduces a dynamic way to perform model pruning during training the masks which allows pruned parameters to be grown back and engaged in the next round of training or pruning.
To make Flying Bird pruning adapt to the scenario of achieving high certified robustness, we plug in the certified training loss during the training.
The results in Table XII demonstrate that FaShapley outperforms Flying Bird towards high certified robustness under different pruning ratios for unstructured pruning.
BAR performs structured pruning via training the masks of channels and filters.
Concretely, BAR adds the budget aware regularization and knowledge distillation loss to optimize the learnable mask.
Similarly, we also additionally consider corresponding certified training loss for fair comparisons.
The results in Table XIII suggest that FaShapley outperforms BAR in terms of both certified robustness and clean accuracy under different pruning ratios for structured pruning.
We only report the results of FaShapley after fine-tuning here since both FB and
BAR perform pruning and fine-tuning simultaneously.
We deem that the reason why FaShapley as a score-based criterion can outperform SOTA pruning methods is that leveraging the power of Shapley value for importance evaluation, our FaShapleybased neuron importance metric is stable and indicative of the location of good subnetworks, whereas in most SOTA pruning methods, the training of the learnable masks that only take binary values often suffers from bad initialization, which is also demonstrated by HYDRA (details in Figure 2 of [59])
[Table]
Comparison with SV-Based Pruning Approaches.
In the literature of model pruning, there exist works [1, 2, 19, 20] which leverage Shapley value approximation to evaluate the importance of filters/channels and perform structured pruning according to it.
To demonstrate the effectiveness of our proposed sample-size optimization and the computational efficiency of our gradient-based approximation, we compare
[Table] [Table]
[Table]
FaShapley with two strong baselines, SVPrune [2] and ShapPrune [20].
SVPrune analyzes the problem of estimating the contribution of hidden units with Shapley values as a principled ranking metric for this task.
It leverages the Monte-Carlo methods to approximate the Shapley values and utilizes the addition of Shapley value and its deviation as the importance scores based on empirical observations.
ShapePrune utilizes a discarding threshold to ignore marginal contributions based on subgroups with low utility and ϵ -greedy selection which samples useful players more times to estimate Shapley in backdoor defense.
We adapt these SV-based neuron importance evaluation methods to our setting by using the same utility function (i.e., the negative of the certified robustness objective) as ours.
We perform comparisons without fine-tuning after the pruning step for direct and fair comparisons.
The results in
Table XIV suggest that 1) for unstructured pruning, SVPrune and ShapPrune cannot scale up to large architectures such as
CNN-7, while FaShapley requires significantly less runtime and shows the effectiveness of high certified robustness of pruned networks, and 2) for structured pruning, FaShapley not only demonstrates significant performance improvements but also requires less runtime.
C. Ablation Studies How to design the utility function for Shapley evaluation?
There are two natural ways of designing the utility function, namely using the negative of the training objective and directly using the nagative of cross-entropy (CE) loss.
Generally, selecting the negative of the robust training objective as the utility function can make the process of evaluation aware of the certified robustness objective and thus outperforming only using CE loss.
Only in some extreme cases where the clean accuracy of the model decreases dramatically after pruning, directly using CE loss can help maintain the ability of the classifier, thus boosting the certified robustness.
The results
[Table] [Table]
[Table]
in Table XV indicate that using the negative of the robust training loss can generally outperform only using CE loss, except for extreme cases where the clean accuracy decreases much ( 99 . 5% pruning on CNN-4 and CNN-7).
Therefore, for consistency, we leverage the negative of the robust training loss as the utility function.
The results of our FaShapley converge fast.
To avoid the exponential computational complexity of exactly calculating marginal contributions, a sampling-based approximation is leveraged, therefore inducing concerns of large sample complexity.
To verify our sample complexity analysis, we evaluate
FaShapley with different sample-sizes and show the results in Table XVI.
It is shown that for both CNN-4 and CNN7, FaShapley can converge within a small sample-size (i.e., 30 samples in the experiment), and therefore we set samplesize to be 30 for all evaluation without specification.
From the runtime results in Table XIV, we can see that with the reduction of sample complexity, FaShapley can scale up to unstructured pruning for which the importance of millions of neurons weights is evaluated.
A relatively larger sample-size is more effective.
After fixing the sample complexity as 30 , we further explore the results with different sample-sizes and show the corresponding results in Table XVII.
We can see that a large sample-size benefits the evaluation of weights.
The results align with our analysis from two perspectives: (1) terms s ik ( U ) with a large sample-size k introduces fairer comparisons among the players since the expectation of the size of the intersection set of sampled subsets S becomes larger, and (2) the utility function value of a small number of players is usually low, thus making the marginal contribution calculated based on it meaningless.
Without specification, we set the sample-size k (normalized to [0 , 1] ) to 0 .
9 for the experiments.
VI. C ONCLUSION
In this paper, we design an efficient criterion based on neuron properties to weaken the tradeoff between the certified robustness and the model size.
We propose a fast and approximated
Shapley method (FaShapley) via gradient-based approximation and sample-size optimization.
The method inherits desired properties from Shapley value and overcomes the challenge of expensive computational costs of Shapley approximation.
Through theoretical analysis and extensive evaluation, we demonstrate that FaShapley is both computationally efficient and empirically effective to achieve high certified robustness for pruned DNNs.
A. Revision Summary
Feasibility with adversarial empirical training.
We compare FaShapley with HYDRA in the empirical adversarial training setting.
The models are pre-trained by adversarial training methods in [47] and during the pruning, we select the utility function as the minus of the training loss.
The results in Table XVIII demonstrate that FaShapley also achieves higher empirical robustness than HYDRA on different architectures. The effectiveness of FaShapley is not restricted in one task, and as long as we define specific utility functions for different tasks we can perform effective importance evaluation for neurons. [Table]
Clarification of handling extremely large pruning ratios.
For structured pruning with extremely large pruning ratios (e.g.
90%), we observe that constraining that each layer remains at least 5% channels benefits the pruning and thus we adopt the constraints.
Since FaShapley can perform neuron importance evaluation from a global view, we do not observe that all the neuron weights in one layer are assigned low values and pruned, and thus we do not set any constraints for unstructured pruning.
Additional results of computational cost comparisons.
We theoretically demonstrate that FaShapley has small sample complexity compared to direct Shapley calculation in Section
IV.B and also empirically show the efficiency of FaShapley compared to SV-based pruning approaches in Table XIV.
Following the suggestion, we further compare the runtime of FaShapley calculation with HYDRA optimization to show the effeciency of FaShapley in Table XXI.
The results in Table
XXI also suggest that the computation cost of pruning is much smaller compared with the standard training of the model.
The evaluation is done on a single RTX 2080 Ti GPU.
[Table]
Visualization of the distribution of remaining weights in convolutional layers.
We evaluate the distribution of remaining weights in different channels of the convolution layer.
The results in Figure 3 suggest that a large fraction of channels are of low importance and most weights are pruned in them.
The
[Table]
[Figure]
observation aligns with the structured pruning evaluation in which the performance only drops a little even with pruning 80% channels (see CNN-7 on CIFAR-10 in Table III).
More explanations of Theorem 2.
m in Equation 7 is the sample complexity of FaShapley while m sv in Equation 8 is the sample complexity of Monte-Carlo Shapley approximation.
As stated in the remark of Theorem 2, since m sv in Equationis larger than m , we theoretically prove that FaShapley is more computationally efficient than direct Shapley approximation.
Further results on the computational time over multiple instances and datasets.
We evaluate FaShapley on CNN-4,
CNN-7 following HYDRA for fair comparisons and also provide results of pruning WideResNet-28-10 on Tiny-ImageNet with SOTA LiRPA verification.
We further report certified robustness and computational costs of pruning MobieNet-v in Table XIX to show that the effectiveness and efficiency of
FaShapley generalizes to different architectures.
Additional evaluation of the speedup of FaShapley.
We add evaluation to show the speedup of FaShapley from three perspectives: 1) influence of sample-size optimization and gradient approximation on runtime of calculation, 2) results of runtime for different sample-size k, and 3) runtime of pruning compared to HYDRA.
The results in Table XX suggest the influence of sample-size optimization and gradientapproximation separately.
We demonstrate that the gradient approximation benefits the calculation efficiency a lot and the sample-size optimization further improves the speed-up and the effectiveness.
The results in Table XXII show runtime for
[Table]
[Table]
[Table]
different sample-sizes k .
We observe that a large sample-size benefits the efficiency because for large sample-sizes we only need to remove a small fraction of neurons which induces less computational cost.
Notice that in baseline SVPrune with permutation-based Monte-Carlo approximation the expected sample size k is 0.5 and thus being less efficient that FaShapley.
In Table XXIII, we further compare the runtime of FaShapley calculation with HYDRA optimization to show the effeciency of FaShapley.
Discussion of relations to SHAP [45].
There are several differences between SHAP and Shapley values.
(1) SHAP [45] approximates the **feature** Shapley (importance) of a trained model under the assumption of feature independence and model linearity.
Since it evaluates the feature importance during test time, meaning that the model doesn’t need to be retrained, it’s less computationally expensive.
On the contrary, Shapley value by definition evaluates the importance of “training instances”, meaning that for every combination of selected instances the model needs to be retrained which is much more expensive.
(2) SHAP uses a linear explanation model to locally approximate the original model to satisfy the linearity assumption, and then approximates the importance of features regarding the linear model.
However, our gradientbased Shapley approximation is directly derived from the
Taylor expansion of the utility function and does not depend on any assumptions of the model.
Therefore, SHAP and
FaShapley approximate Shapley from two different perspectives, special function properties (i.e. linearity) and function value approximation.
Since the gradients with respect to the features are computable, the efficiency of SHAP can be also improved using gradient expectation when computing the conditional expectation of the utility.
With the model linearity assumption, the second-order error of Taylor expression is exactly zero, and thus the approximation error only comes from the sampling error.
Discussion of
the
motivation and application of results without fine-tuning and the influence of fine-tuning for
FaShapley in more detail.
Pruning without fine-tuning only requires access to a small validation set (i.e. 1000 images for CIFAR-10, SVHN, TIny-ImageNet) instead of the full training set in the fine-tuning step.
Fine-tuning is also timeconsuming compared to the pruning step.
Furthermore, since we propose a novel neuron evaluation criteria in the pruning step, results without fine-tuning exclude the influence of fine-tuning and directly show the effectiveness of pruning.
To conclude, we report the results without fine-tuning for their meaning in resource-limited scenarios and for fair comparisons with baselines which report the pruning w/o fine-tuning setting.
In general, fine-tuning can improve the performance of pruning in terms of clean accuracy and certified accuracy.
[Figure]
For low performance of pruning, fine-tuning brings large improvements (e.g. under 99% pruning on CIFAR-10 for CNN4 in Table I), while for high performance close to the upper bound (i.e. performance without pruning), fine-tuning brings no improvements (e.g. under 99% pruning on SVHN for CNNin Table I). Explanations of certain results in Table I. For 97% and 95% unstructured pruning on CNN-7 on SVHN with CROWN-IBP, the clean and certified accuracy after pruning with FaShapley (i.e. 69.8/48.0) reaches the upper bound (i.e. performance of pre-trained model without pruning 69.5/47.9).
It is reasonable and observable that for pruning ratios with which we can achieve the upper bound performance the results are of low variance (e.g. CNN-7 on CIFAR-10 in Table I).
Improved presentation of results.
We present the results in Table V systematically in Figure 4 within a consistent range of pruning ratios.
From Figure 4, we can see that FaShapley outperforms HYDRA in terms of certified accuracy for different architectures and pruning ratios on CIFAR-10 and SVHN.
Theorem 3.
FaShapley F i ( U, k ) of player i with sample-size k satisfies four desired properties as follows:
Proof.
We give the proof of linearity, symmetry, and nullity sequentially as follows: (1) Proof of linearity:
[Equation]
(2) Proof of symmetry:
[Equation]
(3) Proof of nullity:
[Equation]
(4) Proof of efficiency:
[Equation]
N
[Equation]
Fact A.1.
There exist numbers a, b (0 < a < b ) , which satisfy the following inequality,
[Equation]
Theorem 4.
With probability at least δ and estimation error ϵ , the sample complexity m for FaShapley is lower bounded as follows: [Equation] ≥
[Equation]
With probability at least δ and estimation error ϵ , the sample complexity m sv for Shapley value approximation has the lower bound as follows:
[Equation]
Proof.
Define r ki = max( U ( S ∪ { i } ) − min( U ( S )) for S ⊆
I/ { i } and | S | =
k .
Then according to Fact A.1, we have:
[Equation]
If X is the sum of l independent random variables x 1 , ..., x l , each of which is bounded by the lower bound a i and the upper bound b i .
For all t> 0 , the hoeffding’s inequality can be written as follows,
[Equation]
[Equation]
Let q ik ( U ) =
U ( S ∪{ i } ) − U ( S ) for S ⊆
I and | S | =
k be a random variable of the marginal contribution.
For simplicity, we denote q ik ( U ) as q .
Suppose that we have m samples q 1 , ..., q m and Q = (cid:80) mj
=1 q j .
Then the approximated FaShapley value is F i ( U, k )
= 1 m (cid:80) mi =
q i .
We also denote F i ( U, k ) as F for simplicity.
Let X =
Q , x i
= q i , and l = m in Inequality 17, we have:
[Equation]
Equality r ik = b i − a
i holds by definition.
Therefore, we have the upper bound of Pr ( F − E ( F ) ≥ ϵ ) as follows :
[Equation]
Solving the inequality, we can get the bound of sample complexity m as follows,
[Equation]
For direct Shapley value approximation, define r sv = max S ⊆
I U ( S ) − min S ⊆
I U ( S ) .
According to Fact A.1, r sv =
U max .
Plugging r sv = U max into Inequality 18 and solve it, we have:
[Equation]
Review 1: Authors should present the motivation to choose
WideResNet over ResNet in their experiments.
WideResNet is overparameterized and it is of more interest to compress models such as ResNet.
For such models, analyzing the stability of certified robustness at different levels of pruning budget would add value to this work.
Further, results are demonstrated for a larger model of relatively small-scale datasets – TinyImagenet.
Our Revision: We illustrate the rationale of selecting
WideResNet as the network for evaluation on Tiny-ImageNet in the model architecture paragraph of Section V-A. Concretely, we evaluate on WideResNet 28-10 network for Tiny-ImageNet mainly because the SOTA verification LiRPA achieves the highest certified robustness with the type of network (Table 2 in [73]), and thus we use the same architecture for fair comparison purposes.
Review 2:
In the generic pruning literature, there already exist methods that could outperform Shapley in terms of network pruning.
It is of interest to investigate the certified robustness of pruned models obtained with this method.
I would like the authors to comment on why Shapely is needed for certified robustness and not one of the SOTA pruning methods (BAR, Chipnet or CHIP) be tested for it?
Our Revision: Besides the comparisons with the SOTA unstructured pruning method Flying Bird, we additionally compare with the SOTA pruning method BAR and provide the results in Table Table XIII.
The results suggest that FaShapley outperforms BAR in terms of both certified robustness and clean accuracy under different pruning ratios.
We also add more analysis about the advantage of FaShapley over SOTA pruning methods in that paragraph.
We deem that the reason why FaShapley as a score-based criterion can outperform
SOTA pruning methods is that our FaShapley-based neuron importance metric is stable and indicative of the location of good subnetworks; whereas the training of learnable mask in most SOTA methods suffers from bad initialization, which is also demonstrated by HYDRA (Figure 2 in [e]).
Review 3: For the Tiny ImageNet, the authors do not report the HYDRA results and only compare their method with different pruning criterions.
More experiments should be conducted to show the author’s method is better than HYDRA.
Our revision: We add comparisons with HYDRA on TinyImageNet and provide the results in Tab VI.
The results show that FaShapley outperforms HYDRA to achieve high certified robustness on pruned networks on Tiny-ImageNet via autoLiRPA verification.
Note that Tiny-ImageNet with WideResNet 28-10 is the large-scale evaluation we can achieve with deterministic verification, which requires the bound propagation in neurons and is extremely time-consuming.
Therefore, the additional evaluation further demonstrates the effectiveness of
FaShapley.
We include the illustration with more details in
Section V-B.
Review 4: I suggest adding the explanation for the selection of pruning ratio in the experiments.
Our Revision: We add a separate paragraph in Section V-A to illustrate how we select the pruning ratios in the experiments.
Concretely, we follow the criterion that the comparison is done under an arithmetic sequence of pruning ratios such that
FaShapley achieves comparable certified accuracy and clean accuracy as the pre-trained model for the smallest pruning ratio and the baselines perform poorly for the largest pruning ratio.
Therefore, different methods can be compared in a wide and meaningful range of pruning ratios.
Review 5 : Is the proposed gradient-based estimation for
Shapley value better than other estimation methods, such as the sampling-based estimation used in ”Few-shot Backdoor
Defense Using Shapley Estimation”?
Please provide comparison results on computational efficiency, certified robustness, and clean accuracy.
Our Revision: We add comparisons with the SV-based neuron importance evaluation method ShapPrune and show the results in Table XIV.
The results suggest that 1) for unstructured pruning, ShapPrune cannot scale up to large architectures such as CNN-7, while FaShapley requires significantly less runtime and shows the effectiveness of high certified robustness of pruned networks, and 2) for structured pruning, FaShapley not only demonstrates significant performance improvements but also requires less runtime.
All the analysis is added in
Section V-B.