Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models
Authors: Jianwei Li
Qi Lei
Wei Cheng
Dongkuan Xu
Venue: EMNLP 2023 Main (2023)
Keywords: Robust Pruning
Language Models
TL;DR: This paper introduces a novel pruning strategy for language models that conserves more pre-trained knowledge, resulting in improved balance between model accuracy, sparsity, and robustness against adversarial attacks, while reducing pruning cost.

Abstract: The pruning objective has recently extended beyond accuracy and sparsity to robustness in language models. Despite this, existing methods struggle to enhance robustness against adversarial attacks when continually increasing model sparsity and require a retraining process. As humans step into the era of large language models, these issues become increasingly prominent. This paper proposes that the robustness of language models is proportional to the extent of pre-trained knowledge they encompass. Accordingly, we introduce a post-training pruning strategy designed to faithfully replicate the embedding space and feature space of dense language models, aiming to conserve more pre-trained knowledge during the pruning process. In this setup, each layer's reconstruction error not only originates from itself but also includes cumulative error from preceding layers, followed by an adaptive rectification. Compared to other state-of-art baselines, our approach demonstrates a superior balance between accuracy, sparsity, robustness, and pruning cost with BERT on datasets SST2, IMDB, and AGNews, marking a significant stride towards robust pruning in language models.

Towards Robust Pruning: An Adaptive Knowledge- Pruning Strategy for Language Models
Jianwei Li! Qi
Wei Cheng' Dongkuan Xu
North Carolina State University, (jli265, dxu27) @ncsu.edu
"New York University, q1518@ nyu.edu NECC- weicheng @ nec-labs.com
Abstract
remain resilient against adversarial attacks, even in challenging real-world circumstances (Tran et al., 2022; Wang et al., 2023). Therefore, exploring ro- bust pruning strategies against adversarial attacks in language models could potentially yield a sub- stantial impact (Xu et al., 2021; Du et al., 2023).
The pruning objective has recently extended he- yond accuracy and sparsity to robustuess in lan- guage models. Despite this, existing methods struggle to enhance robustness against adversar- ial attacks when continually increasing model sparsity and require retraining process. As humans step into the era of large language mod- els, these issues become increasingly promi- nent. This paper proposes that the robustness of language models proportional to the ex- tent of pre-trained knowledge they encompass. Accordingly, we introduce a post-training prun- ing strategy designed faithfully replicate the embedding space and feature space of dense language models, aiming to conserve more pre- trained knowledge during the pruning process. In this setup, each layer's reconstruction error not only originates from itself but also includes cumulative еттоr from preceding layers, fol- lowed by an adaptive rectification. Compared other state-of-ar art baselines, our approach demonstrates superior balance between ac- curacy, sparsity, robustness, and pruning cost with BERT on datasets SST2, IMDB, and AG- News, marking significant stride towards ro- bust pruning in language models.
Recent research has extended the pruning of lan- guage models beyond accuracy and sparsity, with an emphasis on the trade-off' between accuracy, sparsity, robustness and cost (Du et al., 2023; Xu et al., 2021; Liang et al., 2021: Xi et al., 2022). Zheng et al. (2022) propose a joint optimization ob- jective to guide the pruning and adversarial training simultaneously. Their approach views the identi- fied subnetworks as robust tickets, which can be trained normal and offer enhanced robustness. Despite achieving state-of-the-arr results on target datasets, these methods still display vulnerabilities, as evidenced by a significant gap between met- rics of clean accuracy and accuracy under attack. Moreover, the performance also rapidly declines when sparsity exceeds moderate level. Expanding on their work, Xi et al. (2022) propose using robust early-bird tickets to reduce the computational cost from adversarial training. However, they face simi- lar challenges regarding the trade-off between bustness and sparsity. summary, existing robust pruning works often demonstrate limited sparsity, insufficient robustness, and expensive cost, indicat- ing the ongoing challenge of the balance between accuracy and the other three aspects.
1 Introduction
Pruning is widely recognized compression method employed decrease the model size and accelerate model inference (Frankle and Carbin, 2018; Chen et al., 2020; Prasanna et al., 2020; Chen et al., 2021). In the age of large language mod- els (Andrew and Gao, 2007; Brown et al., 2020; Chowdhery et al., 2022; OpenAI, 2023; Touvron et al., 2023; Ouyang et al., 2022; Smith et al., 2022), the necessity of pruning has increased because it greatly reduces deployment costs (Frantar and Al- istarh, 2023). In addition to the significant com- putation cost, the robustness of language models has emerged as a crucial factor that demands at- tention. This is primarily because models need to
To address this challenge, this paper investigates why language models susceptible to adversarial attacks. (Wang et al., 2021; Garg and Ramakrish- nan, 2020; Jin al., 2020). Previous studies have indicated that language models frequently capital- ize on biases and artifacts inherent in datasets as predictive shortcuts, which impedes reasoning abil- ity and skills to develop advaneed semantic com- prehension. (Du et al., 2021; Niven and Kao, 2019;
accuracy without adversarial attacks

---

McCoy et al., 2020; Du et al., 2023). This reliance leads more severe loss of pre-trained knowl- edge during the pruning process. Furthermore, the adversarial samples in Natural Language Process- ing (NLP) are crafted by replacing components of sentences with semantically similar counterparts, thereby retaining high semantic similarity in the entire sentence (Li et al., 2020a; Ren et al., 2019; Jin al., 2020). In this way, language models that depend on spurious features from particular words can not defend against adversarial attacks constructed by replacing those words with seman- tically similar alternatives. To put it more plainly, this primarily stems from the fact that, without pre-trained knowledge, the sparse language model treats the substitute word simply an integer iden- tifier. Based on the above observation, explore the following questions in this paper:
the pruning-dependedd information in accordance with the sparse output generated by previous layers. Concurrently, there's ' an ongoing effort to correct these errors collectively. Moreover, our method, being a post-training approach, is cost-effective for current language models, as it circumvents rigor- ous retraining processes. Extensive experiments show that our approach achieves better trade-off between accuracy, sparsity, robustness, and prun- ing cost in SST2, AGNews, and IMDB compared with other state-of-an methods.
Related Work
Textual Adversarial Attacks and Defense. Tex- tual adversarial attacks pose significant challenge to the robustness of language models. These at- tacks, formulated by carefully altering certain seg- ments of sentences with semantically similar coun- terparts, aim to fool language models (Jin et al., 2020; Li et al., 2020a). To enhance the robustness of language models and defend against adversarial attacks, a range of potent defensive strategies, such as adversarial training, has been proposed. (Madry et al., 2017; Zhu et al., 2019; Li and Qiu, 2021). Different from their research, which focuses on dense models, we explore the robustness in the context of language model pruning.
What is the core to defend against adversarial attacks for sparse language models?
Question 1.
This paper proposes that the robustness of sparse language models is directly proportional amount of pre-trained knowledge retained after pruning. Intuitively, the robustness of a sparse lan- guage model is fundamentally tied to its capability to distill advanced semantic features from input sentences. This capability is largely established during the pre-training phase of dense language models, emphasizing the pivotal role of acquired semantic knowledge. The extensive experiments well support our statement.
the
Robust Model Pruning. Prior studies indicate that sparse models tend to underperform in Com- pression Identified Examples (CIE), suggesting that the pruning process exacerbates the inherent algo- rithmic biases hidden within the datasets (Hooker et al., 2020). In Computer Vision (CV), simultanc- ous optimization of model pruning and adversarial training has been advocated as an effective solution to this issue (Gui et al., 2019; Ye et al., 2019; Se- hwag et al., 2020; Vemparala et al., 2021). In NLP, Du et al. (2023) propose prevent model overfit- ting on easy samples by leveraging sample diffi- culty in the context of pruning. Concurrently, Xu et al. (2021) suggest the generation of robust sub- networks through Knowledge Distillation and Post- training Quantization. Taking a different approach, Liang et al. (2021) strive to enhance model gen- eralizability by extracting the super tickets, while Zheng et al. (2022) and Xi et al. (2022) seek to iden- tify robust tickets. Despite recent advancements, achieving enhanced robustness alongside increased sparsity remains a challenge. This paper signifi- cantly promotes a better trade-off among accuracy. robustness, sparsity, and pruning cost.
Question 2.
How can we efficiently prevent the loss of pre-trained knowledge pruning to pre- serve or even enhance robustness?
Previous research has demonstrated that pruning exacerbates the model's dependency on spurious features (Xu et al.. 2021; Du et al., 2023). We fur- ther confirm that traditional pruning methods lead a considerable loss of pre-trained knowledge and poor robustness. To prevent the above things, we propose a pruning approach that minimizes dam- age to the embedding space and feature space of dense language models, striving to replicate the features in each layer completely. Specifically, for each layer, we iteratively eliminate single weight at a time and counterbalance the loss by updating the remaining weights based on the Hessian Ma- trix. In this setup, the reconstruction error at each layer arises not only from its own layer but also incorporates the accumulated error from preceding layers. This is achieved by adaptively updating

---

3 Preliminary
be found in the Appendix.
3.1 Shortcut Learning and Mitigation
4 Methodology
Recent studies provide evidence that language mod- els are inclined to capitalize on inherent biases and spurious features present in datasets, using these convenient predictive shorteuts (Niven and Kao, 2019; Du al., 2021; McCoy et al., 2020). This tendency impedes the development of more advanced semantic understanding and reasoning capacity necessary for NLU tasks. Various pre- liminary studies have begun to address this bias issuc, such as adversarial training and posterior reg- ularization (Stacey et al., 2020; Chen et al., 2021). From unique perspective, we let language models against adversarial attacks by mitigating this short- cut issue through weight averaging. This method will be elaborated further in Section 4.2.
This section proposes a pruning method for lan- guage models that can better balance accuracy, spar- sity, robustness, and pruning cost. Figure 1 depicts the architecture of this method.
4.1 Rethink Robust Model Pruning
Given that the predominant challenge in robust pruning primarily centers on robustness and prun- ing cost, we mainly focus on these two aspects in this paper. To enhance the robustness, we ex- plore the root cause of the poor performance of sparse language models under adversarial attacks. We note that adversarial samples are often crafted by replacing certain words in the sentence with semantically similar substitutes. Thus it is essen- tial to ensure that the representation of the original words and their substitutes remain similar in the embedding space and feature space even after prun- ing. Based the above observation, propose to maintain highly close alignment between the sparse and dense language models. In other words, robust pruning is supposed to seek sparse parame- ters Wi that minimize the discrepancy between the outputs of dense and sparse layers. The problem can be formally expressed as follows:
3.2 Pruning with Hessian Matrix
Drawing inspiration from (LeCun et al., 1989; Has- sibi et al., 1993), previous study has provided math- ematical formulations for effectively eliminating a single weight from a layer and updating the remain- ing weights to correct the resulting error according to the information from Hessian Matrix (Frantar and Alistarh, 2022). The equations presented below:
Up argmin
Wpp Wp
H
argmin Ex, Llf((,, (o), s.t. 11 I
H
Jpp
where H is the Hessian Matrix, up represents the single weight that will be pruned, while uy denotes the remaining weights that will be updated. The notation [pp refers to the Pt diagonal entry of the inverse Hessian Matrix, and H.P represents its Ptù column. However, the inversion of the Hessian Matrix requires updates at each weight removal, which is exceedingly costly. Frantar and Alistarh (2022) observes that Hessian values across different weight matrix rows are independent, as single weight removal only impacts its respective row output. Accordingly, they simplify the calculation of Hessian Matrix H and leverage the Gaussian elimination technique to accelerate the update of H-1, described mathematically below:
Here, each layer of language models is represented by mathematical function ([WiX))), and X denotes inputs. k designates the total number of weights that remain non-zero after the pruning process. Predominantly, the Mean Squared Error (MSE) is usually employed to measure the pruning error of each layer. Therefore, the preceding prob- lem can be further reformulated using the MSE, as expressed in the subsequent equation:
argminw, || W.xill s.t. [[aloo &
(4)
To reduce the pruning cost, we adopt post- training setting in strategy. Specifically, we only utilize small subset of data to calibrate the weights and generate sparse substitutes to replace them. In summary, our pruning method does not need a rigorous retraining process.
X
H =1= (H
H.
4.2
Weight Averaging for Robust Dense Model
Here, -p denotes the removal action of single weight at index p. A more detailed explanation can
We also realize that language models may rely on surface-level or spurious features in the data

---

11, 57
by.
Ng $3
h, 5
Pruning
M-syey'
Adkartie
M
M2
M,
M,
Poaiin
WF
Auaiee tra dioro slavu
Wg
m,
m
m
Adlaptian
Adaptive Pruning Hessien
M
M
Average
X
y0
31
Desconding
Apgressively
Robust denae model
fondor
ingredients
Weight Averaging for Robust Dense Model
B: Adaptive Pruning Robust Sparse Model
Figure 1: Architecture of Main Strategy. A: First, generate robust and dense language model in steps: we fine-tune the pre-trained weight with various hyperparameters and settings, resulting in multiple models with different knowledge; we then employ greedy algorithm only average the weights models that contribute the final performance. B: Second,
apply adaptive pruning method to generate robust and sparse language models layer-wird wise setting. Specifically, we optimize the ! original independent pruning process of each layer an adaptive way. This requires subsequent layers update the Hessian Matrix and the optimal dense weight according to the sparse outputs of preceding layers, thereby inheriting and correcting the accumulated error together.
rather than capturing sophisticated semantic fea- tures. Thus, when sparse language models fail to defend against adversarial attacks, becomes challenging to determine whether the failure stems from the pruning methods or inherent issues within the dense model. We circumvents this risk by con- structing robust and dense model before pruning.
details can be found in Algorithm 3.
4.3
Ada-Pruning for Robust Sparse Model Notation
4.3.1
To accurately replicate the dense model's behavior regarding embedding space and feature space of each layer, we use the method described in Sec- tion 3.2 as the backbone. However, its layer-wise setting, which treats each layer as an independent pruning problem, introduces limitations in realiz- ing globally optimal solution. To elaborate, let 's consider single layer as example in the follow- ing sections. We'll use X,, Wi, and Yj to represent the input, weight, and output of the layer, respec- tively, with the subscript / indicating La layer. The use of a hat, as seen in X₁, W,, or Y,, represents the input, weight, or output within a sparse context.
Inspired by Croce et al. (2023) and Wortsman et al. (2022), we generate a robust language model via weight averaging. The key idea is to train mul- tiple models with different hyperparameters and settings, allowing each model to capture distinct nuances of the data and generalize in diverse ways. By averaging their weights, we can create a ro- bust model that benefits from collective knowledge. Specifically, we order these models in descending order based on the accuracy under attack. Then, we selectively average the weights that contribute to the final robustness. Finally, obtain robust and dense model the foundation of subsequent op- erations. This approach ensures that any detected vulnerabilities in sparse language models result from the pruning process, climinating the possibil- ity of them arising from spurious features. More
4.3.2 Adaptive Hessian Matrix
After completing the pruning of the leb layer, certain amount of error stemming from the sparse matrix operation inevitably arises. No matter how minor this error might be, it's important to realize that the output of this layer, denoted as Y, influ-

---

Algorithm/ 1 Prune linear layers (41-1n) ) of BERT with target sparsity 8 and calibration data X
output Yi more closely. Here, Wi denotes the up- dated dense weight, and we design the following equations to derive Wi:
Require: Collect original X, W, Y for 1: procedure LAYERWISE (RUNING(H41
2: for i - 110 do
Wi (x/X))
(5)
4: X..
3:
where T represents the transpose operation, and -1 1 denotes the inverse operation. To ensure that x X is invertible, we also introduce a regulariza- tion term, such as le 4, to the diagonal entries of the matrix. Finally, we can compute the pruning loss more accurately with the updated weight Wi.
WAdaptive update 6: H," < (X.X,)-'
5:
7:
if * then
8:
W; + H X end if
9:
10:
11:
# Praning with Hessian Matrix
12:
dan t- imput dimension k int (din
We also calibrate the optimal weights for non- pruned layers (such as the pooler layer and classi- fication layer in BERT) with Equation 5, aligning the dense layers' output with the altered input. Al- gorithm I provides detailed steps for the code im- plementation, offering a comprehensive overview of our methodology. We also provide comprehen- sive analysis of the computational complexity of our method in the Appendix.
13:
14:
for 1 to do
parallel in
15:
p argminnped..
[W]]
12 lpp
16:
W; <- W:-
H.
[Wilp
m Top
17:
tmp + [H.]..p [H:]
18:
H, H,
[R; "pp tmp
19:
W; + W. remove [Wi]p
20:
end for
21:
22:
# Adaptive update
23:
Y₁ < W,X,
Experiments
24:
X1+1 + post-process(Y))
25:
end for
We first compare our method against several base- line methods, assessing accuracy, robustness, spar- sity, and cost. Then, an ablation study is performed to elucidate the contributions of each part in our method. Finally, we augment our core findings with additional experiments and analyses to further illuminate our method.
26:
return )...
27:
end procedure
ences the input of the subsequent layer, denoted as X As result, the initial Hessian Matrix for the (1+11) 1)th layer, defined as H(+1 X1+1 becomes outdated. Thus it's crucial to recalculate the Hessian Matrix to obtain more precise pruning- dependent information. We suggest adaptively up- dating the Hessian Matrix for the subsequent layer after pruning the preceding layers.
5.1 Baselines and Datasets
Consistent with the previous works (Devlin et al., 2018; 8; Du et al., 2023; Xu et al., 2021; Zheng et al., 2022; Xi et al., 2022), BERT base serves as the foundational model for all our experiments. We compare our approach with various baselines in- Bluding:Robusti (Zheng et al., 2022), which opti- mizes the pruning mask and input perturbation si- multaneously for robust tickets; Bag-of-Ticks (Xu et al., 2021), which improves sparse model robust- ness via Knowledge Distillation and Post- fost-Training Quantization; RMC (Du et al., 2023), a technique preventing sparse language models from overfit- ting on easy samples using sample difficulty; Su- perTicket (Liang et al., 2021), which identifies super mask during pruning to reduce variance while preserving bias. Our evaluation primarily involves three text classification datasets: Internet Movie Database (IMDB, Maas et al. 2011), AG News Corpus (AGNEWS, Zhang et al. 2016), and Stanford Sentiment Treebank for binary classifica- tion (SST-2, Socher et al. 2013).
4.3.3 Adaptive Dense Weight
We also note that the loss generated by removing a single weight depends on the current weight Wi from corresponding layer, as derived from Equa- tion 1. However, an inevitable fact is that the origi- nal dense weight Wi is not optimal for the expected dense output Y, after pruning the preceding layers (0.A... (I ).) Given that the input X₁ has been altered to X, due to the accumulated error, it would be suboptimal to continue using the original weight Wi to calculate the pruning loss for the current layer. To be more clear, the result of X,W2 could substantially deviate from the original output Y.- This is incompatible with our goal of producing an output Yi identical to the original Yi in the pruning process. Thus it's essential to update the dense weight so that Wi can approximates the original

---

SST2
AGNEWS
IMDB
Methods
Pararm
Re-T
Acc
Aua
Asr
Acc
Aua
Asr
Acc
Aua
Asr
Fine-tune
85M
Y
92.3
12.7
86.2
94.7
19.1
80.0
95.1
7.4
92.2
FreeI.B
85M
91.5
28.3
69.1
94.8
37.8
60.1
94.3
36.2
61.6
Weight Average
85M
30.4
66.75
94.4
48.5
95.2
53.4
sparsity < 30%
SuperTicket
72M
Y
93.2
14.3
84.7
94.8
9.7
89.8
95.0
17.3
81.8
Bag-of- Tricks
60M
86.3
25.7
70.3
87.3
31.8
63.6
85.4
24.6
71.2
RMC
60M
94.2
21.4
93.9
76.3
RobusT
60M
Y
90.8
28.9
68.2
94.9
33.4
64.8
92.1
55.7
39.5
Ours
60M
N
90.2
42.3
53.1
93.8
48.6
48.2
94.6
57.3
39.4
sparsity 50%
Bag-of of Tricks
43M
87.2
21.6
75.2
90.6
33.5
63.0
91.3
76.8
RMC
43M
Y
90.8
9.7
89.3
94.1
21.2
77.5
94.1
14.7
84.4
RobusT
43M
Y
90.5
24.8
73.9
94.8
28.8
69.7
93.2
31.5
66.2
Ours
43M
N
88.31
43.1
51.2
93.4
48.5
48.1
94.2
53.2
43.6
sparsity 87.5%
Bfg-of-Tricks
11M
N
85.9
17.8
85.7
89.4
11.3
87.4
87.7
8.9
89.9
RMC
11M
Y
86.3
3.6
95.8
92.1
4.5
95.5
91.3
11.2
87.7
Robus'T
I1M
85.2
7.8
90.8
91.8
8.3
91.0
89.2
6.5
92.7
Ours
11M
37.6
92.4
41.3
55.3
35.6
61.1
Table 1: Summary of Adversarial Robustness Assessment on BERT The entry highlighted with orange background denotes our robust and dense model, which serves as the initialization range of robust pruning methods except RobustT (RobustT is generated from the pre-traine preitrained weight). Obviously, our method consistently outperforms all baselines in terms of the Aua % and Asr% metrics. Regarding Ace% there is a minor decrease in our method's performance lower sparsity levels, yet it regains superiority at higher sparsity levels. The highest performance is highlighted in bold. The column Re-T indicates whether the method necessitates model retraining. Consistent with previous research, we exclude embedding matrices from the calculation parameter count.
5.2 Robustness Evaluation
seeds. Furthermore, we assess the performance under three different levels of sparsity: 30%, 50%, and 87.5%. Additional implementation details be found in Appendix.
We assess model's effectiveness against adver- sarial attacks using the TextFooler, which substi- tutes crucial words in sentences with semantically similar synonyms (Jin et al., 2020). Following pre- vious works (Zheng et al., 2022; Xi et al., 2022), our evaluations utilize key metrics like Clean Accu- racy Ace% % (accuracy on clean test data), Accuracy Under Attack Aua% (accuracy when subjected to adversarial attacks), and Attack Success Rate A (ratio of successful text perturbations to to- tal attempts). robust method is expected show higher clean accuracy and accuracy under attack coupled with a lower attack success rate. We also evaluate more attack methods in the Appendix.
5.4 Main Result on Robustness Evaluation
Table provides a comprehensive comparison of various robust pruning methods, evaluated across three distinct datasets: SST2, AGNEWS, and IMDB, and under varying degrees of model spar- sity. Key observations be made follows: 1) Our strategy even enhances the robustness of lan- guage models after pruning. We believe this en- hancement stems from the regularization effect of sparse architecture. 2) Our strategy distinguishes itself by consistently surpassing other methods in the Aua% % and Asr%s, regardless of the dataset or the level of sparsity. These results imply that our strategy effectively maintains robustness during the pruning of language models. 3) Impressively, our method achieves higher robustness even with fewer parameters compared several other approaches, which further underscores the effectiveness of our robust pruning method. 4) Although the Ace%
5.3 Implementation Details
To begin with, employ the technique mentioned in Section 4.2 generate a robust language model use our method
for each dataset. Subsequently, to prune these robust language models with a small calibration dataset. All experimental results are the average of five trials, each initiated with different

---

035
634
635
031
030
013
00
045
(17))
044
exe
1531
100
Figure 2: Attention Score Visualisation in BERT base- We have selected adversarial sample ("it's 's hewitching and oflen repercussions journey. ") from SST2 and visualized the attention scores in the robust and dense model (2b, 2e), the sparse language model generated with IMP+FreeLB (2a, 2d), and the sparse language model created using method (2c, 2f). Here, Figures 2a, 2b, and 2c depict the attention scores from the first transformer block of BERT Base, while Figures 2d, 2e,and 2f show scores from the last transformer block. Evidently, the attention scores produced by our method align more closely with those from the robust and dense model.
SST2
AGNEWS
IMDB
Methods
#Param
ReT
Acc
Aua
Asr
Acc
Aua
Ast
Acc
Aua
Asr
Fine-tune
85M
Y
92.3
12.7
86.2
94.7
19.1
80.0
95.1
7.4
92.2
Weight Average
85M
91.4
30.4
66.75
94.4
48.5
48.6
95.2
44.4
53.4
IMP
43M
92.6
94.8
94.9
7.1
92.5
94.1
7.7
91.8
IMP FreeLB
43M
92.4
91.5
94.3
9.2
90.2
93.8
14.3
84.8
LTH
43M
Y
91.6
2.8
96.9
93.5
10.1
89.2
93.2
4.6
95.1
ITH + Fr
43M
Y
91.7
9.8
89.3
93.2
12.3
86.8
93.1
9.5
89.8
Ours
43M
88.31
43.1
51.2
93.4
48.5
94.2
43.6
Table 2: Ablation Study with Pruning Methods Replacement. We replace pruning method with most famous others (IMP and LTH) supplemented with adversarial training (FreeLB). Similarly, the orange entry is used for model initialization. Once again, method outperforms others in preserving enhancing robustness.
our method is generally lower than other baselines at lower sparsity levels, the improvement of robust- ) far outweighs the degree of accuracy degradation. 5) At higher method outperforms other
method when compared to the baselines. Moreover, we explore the effectiveness our methods within structured pruning context, and once again, our approach outperforms the state-of-the-arr method: EarlyRobur (Xi et al., 2022). More details can be found in Appendix.
ness (reflected in Aua % and Asr
levels of sparsity, baselines across all metrics. 6) Our method does not require model retraining, confirming that our approach offers better trade-off between accuracy, robustness, sparsity, and pruning cost.
5.5 Ablation Study
To elucidate the contributions of each part of ap- proach, we conduct an ablation study with the fol- lowing settings: We replace our pruning technique with methods known as LTH and IMP (Frankle et al., 2020; Frankle and Carbin, 2018), 8)), and supple-
Beyond Bertsase- our methodology was also ex- tended to Berttarges model encompassing 330M parameters. The resulting performance, as pre- sented in Table 3, reaffirms the superiority of our

---

SST2
AGNEWS
IMDB
Methods
fPaara
Re-T
Acc
Aua
Asr
Acc
Aua
Asr
Acc
Aua
Asr
Weight Average
309M
Y
93.5
36.4
61.1
96.2
56.5
41.3
95.9
48.
49.6
Bag-of- Tricks
Bag-of-Tricks
155M
90.3
27.6
69.4
93.1
35.5
61.9
93.4
29.3
68.6
RMC
155M
14.7
84.1
95.4
19.2
79.9
95.8
16.7
82.6
RobusT
155M
Y
92.1
29.8
67.7
95.1
32.8
65.6
95.2
31.9
66.5
Ours
155M
N
91.7
47.1
48.6
95.5
53.5
44,0
95.3
55.8
41.4
Table Summary of Adversarial Robustness Assessment on BeRge Similarly, the entry highlighted with an orange background is used for model initialization. Once again, our method consistently outperforms all baselines in terms of the Aua % and Sue% metrics.
ment them with the additional adversarial training method FreeLB (Zhu al., 2019). The results are presented in Table 2. From the results, make the following key observations: 1) Sparse language models generated by traditional pruning methods performs even worse than the vanilla fine- tuned dense model. This highlights the challenges associated with robust pruning. 2) Our approach consistently generates more robust sparse language models than conventional pruning methods, even supplemented with adversarial training methods. 3) We conjecture that the limited effect of adversar- ial training here stems from the discrete nature of word tokens and the substantial loss of pre-trained knowledge during pruning.
Table Quantitative Analysis of Distance from Sen- tence Embeddings. We compare the distances between sentence embeddings derived from various layers dense and sparse language models. Our findings reveal that our method aligms better with the dense model, re- gardless of whether we use the original or adversarial sentence. Refer Figure 5 for visualization of these sentence embeddings.
Distance with dense
Layer
Data
ADT
0L0086
0.0000
Oni
0.0141
0.0015
0.0142
0.0015
Adv
0.0288
0.0012
Adv
0.0407
0.0017
Adl
0.0324
0.0867
Osi
0,1319
qur165
0.0255
Oni
5.6 Discussion
0.097
0.0253
0.1299
0.0496
In this section, we design additional experiments to illustrate our robust pruning method further.
0.1478
0.0501
Adv
0,2547
0.1078
Adv
0.1680
5.6.1 Pretrained Knowledge Detection
0.2767
0.0749
Ade
0.3804
0.1254
Ori
10
To demonstrate the effectiveness of our robust prun- ing mechanism preserving pre-trained knowl- edge, we've chosen adversarial samples that are effectively defended by our method but not by oth- ers. We then visualize the attention scores of them in Figure 2. Our method demonstrates superior performance, as evidenced by more reasonable at- tention scores that align more closely with those from the robust and dense model. addition, we visualize the distance of sentence representation from sparse language models and their dense coun- terparts in the feature space. As depicted in Ta- ble 4 and Figure 5, our method results in smaller distances between the dense and sparse representa- tions. These findings indicate the superior ability of our robust pruning method preserve seman- tie knowledge and maintain cognizance. In other words, our method outperforms others in maintain- ing robustness during pruning.
0.4932
0.2312
Oci
0.317
0.0625
0.6892
0.2231
0.6903
0.1344
Adv
5.6.2 Impact of Calibration Data
The calibration data crucial for our methodology because it directly affects the computation of the Hessian Matrix. As outlined in Algorithm the Hessian Matrix can be derived from H = =''X. To further explore the impact of the number of data points, we designed experiments that gradu- ally increased the number of data points used in strategy. The results of these experiments are detailed in Figure 3. Our observations indicate that as the number of used data points increases. the robustness and accuracy of the sparse language modes increase, but only up to a certain threshold. We hypothesize that the model can initially retain

---

more general knowledge data points increase. However, once threshold is crossed where the new data cannot provide additional information for general features, adding more data points from a similar distribution no longer contributes to model robustness and accuracy.
80
Danse
RabusE
Dente
-a- Aua ISC-FrceLB
Ace EarlyRobust
Performance
60
Acc Ours
Acc RobusiT
Aus Earlyfobuet
40
5.6.3 Impact Sparsity
20
As illustrated in Figure 4, we explore the robustness and accuracy of our sparse language models across a range of sparsity levels. In a departure from previous studies Zheng et al. (2022), our observa- tions indicate that sparsity increases, robustness decreases with similar pace like accuracy. This trend suggests that the impact of increasing sparsity on model robustness might be less severe than pre- viously assumed. This disparate pattern may stem from the post-training nature of our method. Fur- thermore, our observations regarding the trend in robustness align with the findings of previous stud- ies by Zheng et al. (2022) and Liang et al. (2021), We note that the robustness of our sparse language models initially improves as sparsity escalates up certain threshold. After crossing this thresh- old, the robustness begins to decline. However, sustains a level of robustness that is higher than the peak value observed in other models and does not collapse even with 10x compression. This finding further highlights the outstanding performance of our method in robust pruning.
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Sparsity
Figure 4: Impact of Sparsity Levels on SST2
of experiments conducted across various tasks.
Limitations
This work introduces a post-training method that can robustly prune the language models without model retraining. Despite bypassing the rigorous retraining process, the computational cost of our method remains significant due to the calculation of the Hessian Matrix and its inverse. Consequently, this approach may not be feasible for language models comprised of billions of parameters. As a next step, aim to refine technique devise more efficient strategy replicate the feature space and embedding space of language models
Acknowledgements
45.0
The authors wish to thank the anonymous reviewers for their helpful comments.
90
Accuracy Under Attacks
42.5
40.0
Ethics Statement
37.5
35.0
This work complies with the ACL Ethics Policy and we have carried out our research following the highest ethical standards. In our work develop- ing pruning strategy enhance robustness in language models, we carefully considered the broader implications and ethical dimensions of this innovation.
32.5
84
30.0
82
27.5
80
500
1000
1500
2000
Number
Figure 3: Impact of # of Calibration Data from SST2.
While our research primarily concerns the im- provement of model accuracy, sparsity, and robust- ness, we acknowledge that the use of these en- hanced models can potentially be dual-use, which means they can be applicd in both beneficial and harmful ways. An improved model contribute positively by enhancing various NLP applications such as text summarization, machine translation, and sentiment analysis, potentially increasing ef- ficiency and the overall quality of output. Fur-
6 Conclusion
In this paper, we investigate the application of ro- bust pruning methods for language models. We propose adaptive pruning method and place special emphasis on replicating the embedding and feature space of dense models preserve as much pre-trained knowledge as possible. The effective- ness of this approach confirmed through series

---

thermore, these advancements could contribute to reducing the computational resources required for training and using large language models, which aligns with efforts to reduce the environmental im- pact of machine learning.
BERT: Efficient BERT training via early-bird lottery tickets. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the I 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 2195-2207, Online. Association for Computa- tional Linguistics.
However, the increased robustness of models against adversarial attacks could also be used mali- ciously if the technology falls into the wrong hands. Bad actors could potentially exploit robust models for the generation of disinformation or manipula- tion of public sentiment, for instance. Furthermore, although our technique aims to laithfully replicate the feature space of dense models, bias present in the original training data could be preserved in the pruned models. Consequently, decisions made based on the output of these models could perpetu- ate these biases.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez. Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev. Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hycontaek Lim, Barret Zoph. Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang. Brennan Sacta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Tekk, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- cling with pathways.
We encourage the use of our findings and meth- ods for applications that promote the public good and contribute to human welfare. Further, we rec- ommend that researchers and practitioners using this technique take into account potential biases in their training data and consider strategies for minimizing their impact. In the future, we hope to conduct more research mitigating bias and other ethical issues associated with our pruning strategy. It is our belief that technology should be developed and used in a way that is transparent, fair, and beneficial to all.
Francesco Croce. Sylvestre-Alvis Rebuffi, Evan Shel- hamer, and Sven Gowal. 2023. Seasoning model soups for robustness to adversarial and natural dis- tribution shifts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 12313-12323,
References
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing, arXiv preprint arXiv:1810.04805.
Galen Andrew and Jianfeng Gao. 2007. Scalable train- ing of Li Li-regularized log-linear models. In Proceed- ings the 24th International Conference on Machine Learning, pages 33-40.
Mengnan Du Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncour, Jiuxiang Gu, Tong Sun, and Xia Hu. 2021. Towards interpreting and mitigating shortcut learning behavior of niu models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 915-929.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sustry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
Mengnan Du Subhabrata Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, and Ahmed Hassan Awadallah. 2023. Robustness challenges in model distillation and pruning for natural language understanding. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pages 1766-1778, Dubrovník, Croatia, As- sociation for Computational Linguistics.
Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, and Michael Carbin. 2020. The lottery ticket hypothesis for pre- trained bert networks. Advances in neural informa- tion processing systems, 33:15834-158.. 15846.
Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arxiv: 1803,03633
Xiaohan Chen, Yu Cheng, Shuohang Wang. Zhe Gan, Zhangyang Wang, and Jingjing Liu. 2021. Early-

---

Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. 2020. Linear mode con- nectivity and the lottery ticket hypothesis. In Inter- national Conference on Machine Learning, pages 3259-3269. PMLR.
6202, Online. Association for Computational Linguistics.
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020b. Bert-attack: Adversar- ial attack against bert using bert. arXiv preprint arXiv:2004, 099844.
Elias Frantar and Dan Alistarh. 2022. Optimal brain compression: A framework for accurate post- training quantization and pruning. arXiv preprint
Linyang Li and Xipeng Qiu. 2021. Token-aware vir- tual adversarial training in natural language under- standing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8410-8418.
arXiv:2208. 1/580.
Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Mas- sive language models can be accurately pruned in one-shot.
Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, and Weizhu Chen. 2021. Super tickets in pre-trained language models: From model compression to im- proving generalization. In Proceedings of the 59th Annual Meeting of the Association for Compulational Linguisticx and the 1 11th International Joint Confer- ence on Natural Language Processing ( (Volume 1: Long Papers), pages 6524 6538, Online. Association for Computational Linguistics.
Siddhant Garg and Goutham Ramakrishnan. 2020. BAE: BERT-based adversarial examples for text clas- sification. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 6174-6181, Online. Association for Computational Linguistics.
Mitchell Gordon, Kevin Duh, and Nicholas Andrews. 2020. Compressing BERT: Studying the effects of weight pruning on transfer learning. In Proceedings of the 5th Workshop on Representation Learning for NLP. pages 143-155, Online. Association for Com- putational Linguistics.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Lin- guistics.
Shupeng Gui, Haotao Wang, Haichuan Yang. Chen Yu, Zhangyang Wang, and Ji Liu. 2019. Model compres- sion with adversarial robustness: unified optimiza- tion framework. Advances in Neural Information Processing Systems, 32.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint 17X6vv1706.0660
B. Hassibi, D.G. Stork, and G.J. Wolff. 1993. Optimal brain surgeon and general network pruning. In IEEE International Conference on Neural Networks, pages 293-299 v.l.1.
R Thomas McCoy, Junghyun Min, and Tal Linzen. 2020. Berts of a feather do not generalize together: Large variability in generalization across models with simi- lar test set performance. In Proceedings of the Third BlackboxNI.P Workshop Analyzing and Interpret- ing Neural Networks for NLP,, pages 217-227.
Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio, and Emily Denton. 2020. Characteris- ing bias in compressed models. arXiv preprint arXiv:2010.03058.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? a strong base- line for natural language attack on text classification and entailment. In Proceedings of the AAAI con- ference on artificial intelligence, volume 34, pages 8018-8025.
John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. 2020. Textattack: A frame- work for adversarial attacks, data augmentation, and adversarial training in nlp. In Proceedings of the 2020 Conference on Empirical Methods in Natu- ral Language Processing: System Demonstrations, pages 119-126.
Yann LeCun, John Denker, and Sara Solla. 1989. Opti- mal brain damage. Advances in neural information processing systems, 2.
Timothy Niven and Hung- Yu Kao. 2019, Probing neu- ral network comprehension of natural language argu- ments. In Proceedings of the 57th Annual Meeting of the A Association for Computational Linguistics, pages 4658 4664, Florence, Italy. Association for Compu- tational Linguistics.
Jinfeng Li. Shouling Ji. Tianyu Du, Bo Li, and Ting Wang. 2018. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:/812.05277
Linyang Li. Ruotian Ma. Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020a. BERT-ATTACK: Adversar- ial attack against BERT using BERT. In Proceed- ings of the 2020 Conference on Empiricul Methods in Natural Language Processing (EMNLP), pages
OpenAl. 2023. Gpt-4 technical report.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.

---

2022. Training language models follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744,
Dustin Tran, Jeremiah Zhe Liu, Michael W Dusen- berry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda E Mariet, Huiyi Hu, Neil Band, Tim G. J. Rudner. Zachary Nado, Joost van Amersfoort, Andreas Kirsch, Rodolphe Jenatton, Nithum Thain, E. Kelly Buchanan, Kevin Patrick Murphy, D. Scul- ley, Yarin Gal, Zoubin Ghahramani, Jasper Snoek, and Balaji Lakshminarayanan. 2022. Plex: Towards reliability using pretrained large model extensions. In First Workshop on Pre-training: Perspectives, Pit- falls, and Paths Forward at ICML 2022.
Sai Prasanna, Anna Rogers, and Anna Rumshisky. 2020. When BERT Plays the Lottery, All Tickets Are Win- ning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3208 3208-3229, Online. Association for Computational Linguistics.
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial exam- ples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 1085- 1097, Florence, Italy. Association for Computational Linguistics.
Manoj- Rohit Vemparala, Nael Fasfous, Alexander Frick- enstein, Sreetama Sarkar, Qi Zhao, Sabine Kuhn, Lukas Frickenstein, Anmol Singh, Christian Unger, Naveen-Shankar Nagaraja, et al. 2021. Adversarial robust model compression using in-train pruning. In Proceedings of the IEEE/CVF Conference on Com- puler Vision and Pattern Recognition, pages 66-75.
Victor Sanh, Thomas Wolf, and Alexander Rush. 2020. Movement pruning: Adaptive spansity by fine-tuning. Advances in Neural Information Processing Systems, 33:20378-2008 20389.
Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Hao- jun Huang, Wei Ye, Xiubo Geng, et al. 2023, On the robustness of chatgpt: An adversarial and -fist--distributiin perspective. arXiv preprint arXiv:2302. 20000550...
Vikash Schwag. Shiqi Wang. Prateck Mittal, and Suman Jana. 2020. Hydra: Pruning adversarially robust neural networks. Advances in Neural Information Processing Systems, 19:19655-19666
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng. Zexiong Pang. Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan. Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu. Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, and Xuanjing Huang. 2021. TextFlint: Unified multilingual robustness evaluation toolkit for natural language processing. In Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages -47-355, Online. Association for Computational Linguistics.
Shaden Smith, Mostofa Patwary. Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zhang, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro. 2022. Using deepspeed and megatron to train megatron- nlg 530b, a large-scale generative language model.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 Conference on Empiri- cal Methods in Natural Language Processing, pages 1-642, Seattle, Washington, USA. Association for Computational Linguistics.
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lppes Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Komblith, and Ludwig Schmidt. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increas- ing inference time.
Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Se- bastian Riedel, and Tim Rocktäschel. 2020. Avoiding the Hypothesis-Only Bias in Natural Language Infer- ence via Ensemble Adversarial Training. In Proceed- ings of the 2020 Conference on Empiricul Methods in Natural Language Processing (EMNLP), pages 8281-8291, Online. Association for Computational Linguistics.
Zhiheng Xi, Rui Zheng, Tao Gui, Qi Zhang, and Xu- anjing Huang. 2022. Efficient adversarial training with robust early-bird tickets. In Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 8318-8331, I,, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet. Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Fric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models.
Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, and Furu Wei. 2021. Beyond preserved accuracy: Evaluating loyalty and robustness of BERT compression. In Proceedings of the 2021 Conference

---

on Empirical Methods in Natural Language Process- ing, pages 10653-10659. Online and Punta Cana, Dominican Republic. Association for Computational Linguisties.
Shaokai Yc, Kaidi Xu, Sijia Liu, Hao Cheng, Jan-Henrik Lambrechts, Huan Zhang, Aojun Zhou, Kaisheng Ma, Yanzhi Wang, and Xue Lin. 2019. Adversarial robustness vs. model compression, or both? Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 111-120.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2016. Character-level convolutional networks for text clas-
sification.
Rui Zheng, Bao Rong, Yuhao Zhou, Di Liang, Sirui Wang, Wei Wu, Tao Gui, Qi Zhang, and Xuanjing Huang. 2022. Robust lottery tickets for pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume Long Papers), pages 2211-22 1-2244, Dublin, Ireland. Association for Computational Lin- guistics.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Gold- stein, and Jingjing Liu. 2019. Freelb: Enhanced ad- versarial training for natural language understanding. arXiv preprint arXiv:1909.1 11764.

---

A Appendix-A
B Appendix-B
A.1 Pruning with Hessian Matrix
B.1 Efficiency Analysis of Hessian Matrix
We recognize the importance of addressing the efficiency concern related to Hessian Matrix cal- culation. However, grasping the intricate bal- ance between computational complexities and their broader implications is crucial. To provide clar- ity, we offer an in-depth analysis of computational complexities from both micro and macro view- points, contrasting it with approaches that necessi- tate model retraining.
As described in Section 3.2, we prune each layer of language models in a luyer-wise setting. It in- volves iterative step that removes a single weight for each step and updates the remaining weights until the desired sparsity level is attained. While this approach yields a locally optimal solution, involves a computationally expensive step: calcu- lating the Hessian matrix at each iteration. It is important to note that storing the information for Hessian Matrix, denoted as H. requires d x d mem- ory, and updating it has computational complexity of (dd), where drow
B.2 Micro Perspective
When considering models like Bertiase and Bertiarge- the computational requirements for the Hessian Matrix of one layer do not exceed that of model retraining in most cases. To clarify it, we analyze the complexity of our method step by step based on the Algorithm 2.
deol-
A.2 Accelerated Pruning with Hessian Matrix
Previous research highlights that the Hessian val- ues across different rows of the weight matrix are independent. This is because the removal of single weight in each row of the matrix only affects its corresponding row value. Conse- quently, wc can simplify the objective function with Eeen [W,. 21 and a separate Hessian Matrix of appropriate size (deat x doal) for each row is sufficient to locate the optimal weight for removal. Additionally, since the output Y = W of the dense layer remains fixed, and the objective function for each row takes the standard form of least squares, its Hessian Matrix can be calculated by H 2X (Frantar and Alistarh, 2022).
Algorithm 2 Prune linear layer of BERT with target sparsity s and calibration data X
Input: Collect original X, W. Y for l. procedure PRUNING()
Set W, X, <
Adaptive Update 1: Calculate H 1- (XX')
5: Set W + ---IxTy
Pruning with Hessian Matrix:
6:
Set din + input dimension.
7:
Set k < int(din s).
As the Hessian Matrix H is longer dependent the weight, only need to compute H once. After each pruning step, the Hessian Matrix HM (M means the operation of removing or masking one single weight) can be obtained by masking the value at the corresponding location. However, when it comes to 11-1 the aforementioned trick cannot be applied as (H (H~)) * (HM)-², mak- ing the computation still expensive. Frantar and Alistarh (2022) uses the Gaussian elimination tech- nique for more efficient update of H H-1, 1. A math- ematical exposition of this technique is provided below:
for j = 1 to k (parallel in rows) do
Set p argmin
[W]p
pp
10:
Set W W
H
[W]p.
Pp
11:
Set A +- 1
12:
Set B <- [H]
p.:
13:
Set H +- H-1
AB
H 1
1pp
14:
Remove [W]p from W
15:
end for Adaptive Update 2: Set + WX.
16:
17:
Update X of next layer with post- process(Y)
18: end procedure
H = (H
H. HH1H 'ppp
--p
(6)
H
Notations: To facilitate the understanding. we first introduce the notations essential for the complexity analysis. The sparsity ratio, a value lying between and 1, is denoted by s. The input dimension of the linear layer represented by din, and the output
where -p meas remove single weight at index p. For more comprehensive details, please refer to the work of Frantar and Alistarh (2022).

---

SST2
AGNEWS
IMDB
Methods
FParam 85M
ROT
Fine-tune
92.3
12.7
86.2
94.7
19.1
80.0
95.1
7.4
92.2
FreeLB
85M
Y
91.5
28.3
69.1
94.8
37.8
60.1
94.3
36.2
61.6
Weight Average
85M
Y
91.4
30.4
66.75
94.4
48.5
48.6
95.2
44.4
53.4
sparsily= 50%
EarlyRobust (Stru)
43M
Y
91.2
15.6
82.9
94.1
28.4
69.8
90.7
33.2
63.3
Ours (w/o Stru)
43M
N
88.31
43.1
51.2
93.4
48.5
48.
94.2
53.2
43.6
Ours (Stru 32:64)
43M
N
88.42
44.3
49.9
93.2
49.1
47.3
94.8
53.4
43.7
Table 5: Summary of Adversarial Robustness Assessment on BERT base in Structured Pruning. "Stru 32:64" refers to a pruning strategy where, for every 64 continuous weights (a bank) in a weight matrix. 32 of them are retained.
dimension, aligning with the weight matrix's other dimension, is symbolized by deut. We use den dout illustrate the comprehensive size of the weight matrix. The batch size and the sequence length are, respectively, given by n and seq.
substantially.
B.3 Comparison with Re-Training Method
In contrast, when training a single layer using SGD, the complexity is approximately O(n x seq x din x dont). This complexity scales linearly with the batch size n, which can increase markedly with large datasets and the number of training epochs. Although the complexity of the pruning operation remains consistent regardless of n, the training complexity escalates, posing computational chal- lenges for extensive datasets, prolonged sequences, and increased training epochs. We also dive deeper into the comparative insights.
Adaptive Update (1): In this phase, the matrix multiplication X plays a pivotal role. Given the dimensions of X n seq, din and that of X din seq X n, the resulting matrix has a shape of di din- This multiplication alone possesses a complexity of O(n x seq x d Additionally, ma- trix inversion is another vital step with a complexity of (dd,,). The computation of H X Y; further contributes to the complexity, having a magnitude of (n seq x din dout).
Pruning with the Hessian Matrix: this con- text, the outer loop spans dout iterations. Within each row of W, an inner loop determined by k = int( (din x s) is executed. This loop com- prises various operations with O(d Summing up, the inner loop complexity is O(k ) Conse- quently, the combined complexity for the pruning phase is (din x di, dout). simplifying to Od X 8 x dout).
Our pruming method capitalizes calibration data, thus constricting to moderate values, notably between 128 to 1024. This sharply diverges from the conventional training paradigm where n can inflate significantly due to extensive datasets and number of training epochs, thereby magnifying its computational requisites.
Batch Size:
Dimensionality Dependency: The intrinsic com- plexity of our pruning algorithm reveals cubic dependency on dnn This can render computation- ally onerous, especially for layers endowed with an extensive din- Conversely, traditional training exhibits linear correlation with both din and dout-
Adaptive Update (2): The matrix multiplication Y = WX X dominates with complexity of On x seq X din dows). Summing complexities for single layer yields O(2n x seq x din x dout n x seqx di + +2d3, +dB, xsx x with the dominant terms being (d)), x dows). Thus, pruning a layer has a complexity of Od x doat), which is also proved by Frantar and Alistarh (2022).
In summary, the computational demands of our pruning method, particularly for layers with a large din, are unquestionably stringent. However, it's im- portant to recognize the significant computational burden introduced by traditional training, mainly because of its responsiveness to large dataset sizes. Understanding this balance and trade-off is crucial when comparing the effectiveness and suitability of our pruning approach to traditional retraining.
Key observations: A pivotal observation is that this complexity remains uninftuenced by the batch size n because calibration data keeps n restricted to constant fall in [128, 1024]. The cubic rela- tionship with dm is the primary driver behind the complexity, and for larger d this can escalate

---

Method
Dataset
Attack
Sparsity
Accuracy
Accuracy under attack
Ours
SST2
TextBugger
2x
88.31%
50.34%
RobustT
SST2
TextBugger
2x
90.5%
35.6%
EarlyRobust
SST2
TextBugger
2x
91.2%
36.7%
Ours
SST2
TextBugger
4x
86.93%
49.08%
Ours
SST2
TexiBugger
85.6% 8x
48.85%
Ours
SST2
BERT-Attack
2x
88.31%
51.95%
RobustT
SST2
BERT-Attack
2x
90.5%
28.3%
EarlyRobust
SST2
BERT-Attack
2x
91.2%
30.2%
Ours
SST2
BERT-Attack
4x
86.93%
50.57%
Ours
SST2
BERT-Attac
8x
85.6%
49.32%
Ours
IMDB
TextBugger
2x
94.2%
58.2%
RobustT
MMDB
TextBugger
2x
93.2%
46.
EarlyRobust
IMDB
TextBugger
90.7% 2x
48.7%
Ours
IMDB
BERT-Attack
94.2% 2x
52,
RobustT
MMBB
BERT-Atlack
2x
93.2%
43.1%
FarlyRobust
IMDB
BERT-Attack
2x
90.7%
43.5%
Ours
AGNews
TextBugger
2x
93.2%
62.0%
RobustT
AGNews
TextBugger
2x
94.8%
44.1%
FarlyRobust
AGNews
TextBugger
2x
94,1
46.2%
Ours
AGNews
BERT-Attack
2x
93.2%
70.8%
RobustT
AGNews
BERT-Attack
2x
94.8%
36.8%
EarlyRobust
AGNews
BERT-Attack
2x
94.1%
39.3%
Table 6: Evaluation of various methods and datasets against different adversarial attacks.
B.4 Macro Perspective
processes.
Computational Commitment: While it's knowledged that pruning with the Hessian Ma- trix does possess computational time costs, it's paramount understand our larger vision. The ultimate objective isn' merely to save time but to sculpt a model characterized by three pillars: spar- sity, robustness, and high performance. Such a model offers considerable advantages in real-world scenarios. Thus, the computational expenses en- countered in the training phase be viewed less as costs and more strategic investments.
Predicable Processing Time and Promised Out- put: Notably, from a broader view, while our ap- proach introduces a dependency for each layer and potentially increases processing times, the number of layers in common language models is limited. accurately predict the
This suggests that time needed to complete the pruning process, and expect positive results in return.
Layer-by-Layer Computation for Resource Effi- ciency: While the sum of Hessian Matrix computa- tions of the entire language model is time-intensive our approach uniquely addresses this by employing a layer-by-layer resolution strategy. This methodol- ogy means there's no necessity to simultaneously load the entire model into the memory of compu- tational resources. Consequently, from memory allocation standpoint, our pruning with the Hes- sian Matrix be viewed resource-saving measure.
Appendix-C
C.1 More Adversarial Attacks
To demonstrate the superiority of our method, we have incorporated further experiments targeting two more recognized adversarial attacks: BERT- Attack and TextBugger (Li et al., 2020b, 2018). BERT-Attack, powered by BERT, guarantees flu- ency and retains semantics in its adversarial outputs. Conversely, TextBugger integrates both character and word-level perturbations to yield adversarial instances, thereby introducing new set of chal- lenges for our defense mechanism. We use state- of-the-art art methods (RobustT and EarlyRobust) as baselines and describe the results in Table 6 (Zheng
Efficient Post-training Pruning: A post-training pruning strategy at the heart of our methodol- ogy. Unlike many other approaches that might require recurrent training sessions exhaustive reitcrations, ours stands out its ability to save significant resources by strategically avoiding these

---

al., 2022; Xi et al., 2022). Our approach consis- tently demonstrated superiority in the robustness of sparse language models across various sparsity levels and datasets.
The number of calibration data in our main ex- periments ranges from 256 1024 sentences. Dur- ing pruning, we conduct our experiments on a server with single NVIDIA 3090 GPU. Due to the layer-wise setting, we do not need to occupy substantial GPU memory, and our adaptive rule en- ables to obtain end-to-end rectification effect similar to SGD optimization.
C.2 More Pruning Baseline
As recommended by the reviewer, we have in- cluded Movement Pruning (Sanh al., 2020) as an additional baseline in our experiments. Our original selection baselines was grounded on their capacity to simultaneously address accuracy, sparsity, robustness, and pruning cost. It should be noted that Movement Pruming predominantly emphasizes accuracy and sparsity.
D.2 Impact of Structured Pruning
Drawing inspiration from the work by Xi et al. (2022), we also investigate the impact of structured pruning in our strategy. In particular, we evaluate our method's performance under N:M structured patterns and summarize the results in Table 4. We made several key observations from these experi- ments: 1) our method consistently produces better robust pruning results than other robust pruning methods the context of structured pruning. 2) As proven by Xi et al. (2022), structured pruning enhances the robustness of subnetworks in com- parison unstructured pruning. Our experiments confirm the positive impact of structured patterns in pruning, solidifying the effectiveness of our robust pruning method.
Nevertheless, to offer complete perspective. we have included Movement Pruning in our exper- imental evaluation. The comparative results are presented in Table 7. It is evident that, while our method may trail slightly in terms of clean accu- racy, it significantly outperforms Movement Prun- ing under ad adversarial conditions, highlighting the robustness of our approach.
Appendix-D
D.1 More Implementation Details
We utilize various hyperparameters and settings to fine-tune multiple downstream models for each dataset. The hyperparameters and settings em- ployed are presented in Table 8. Subsequently, we apply the technique of weight average in a greedy manner derive robust and dense models. The detailed procedure is outlined step-by-step Algo- rithm
E
Appendix-E.
E.1 Model Pruning
Pruning aims to climinate redundant elements in neural networks, traditionally targeting elements of the smallest magnitude, which includes weights, output sensitivity, gradients, and Hessian matrices of training loss, among others. Pruming pre- pre-trained language models like BERT has been an active field of research. Prasanna et al. (2020) demonstrated that unstructured pruning yields more sparse and accurate models. Pruning the pre-training stage has been favored by researchers like Gordon et al. (2020) and Chen al. (2021), due to its efficiency and effective knowledge transfer to downstream tasks. Sanh et al. (2020) adds penalty terms to the loss function to eliminate redundant weights. Frantar and Alistarh (2022) introduce an effective post-training pruning method, which is the first ap- proach that prunes language model in a one-shot manner without significant degradation in accuracy. However, these studies neglect robustness, focus- ing mainly on the accuracy-sparsic trade-off. Re- cent work has begun to note the issue of robustness for sparse language models, but the challenge enhancing robustness with increased sparsity per-
Algorithm Greedy Weight Averaging
procedure GREEDYWA((N)
kx)) ha}
2:
(01....,0z) 000 (hi,...
3:
(mi..... m& (0...... 02)
4:
Sont(01..... 3) with {mi, m......... me]
ingredients for to do
if (rvalllavergeeeeeeedeenn (0,1)) >
Eval(average (mgredent())))eeeentss then ingredients ingredients (0.}
10:
end if end
11:
12:
return average(ingredients)
13:
end procedure
We adopt Textatrack (Morris al., 2020) im- plement the method of adversarial attacks. More- over, Aua% and Sue% are evaluated on all 872 test examples for SST-2, 500 randomly selected test samples for IMDB and AG NEWS.

---

Table 7: Comparison between our method and Movement Pruning under various attacks and sparsity levels.
Method
Dataset
Attack
Sparsity
Accuracy
Accuracy under attack
Ours
SST2
TextFooler
2x
88.31%
43.12%
Movement Pruning
SST2
TextFooter
90.6%
14.85%
Ours
SST2
TextFooler
4x
86.93%
40.15%
Movement Pruning
SST2
TextFooler
4x
90.5%
8.27%
Ours
SST2
TexeFooler
8x 85.6%
37.63%
Movement Pruning
SST2
TextFooler
90.0%
9.14%
Ours
SST2
TextBugger
2x
88.31% %
50.34%
Movement Pruning
SST2
TextBugger
2x 90.6%
24.8 %
SST2
TexiBugger
86.93%
49.08%
Movement Pruning
SST2
TextBugger
4x
90.5%
21.35%
Ours
SST2
TextBugger
8x 85.6%
48.85 %
Movement Pruning
SST2
TextBugger
90.0%
15.13%
ids
Ir
opt
soed
epoc
wd
adt
E.3 Layer-wise Pruning
2e-5
Adam
42
10
le-2
Layerwise Pruming is an important approach to optimizing language models, offering distinct methodology compared end-to-end pruning. Un- like end-to-end pruning. which simultancously evaluates and prunes the entire model whole, layerwise pruning tackles each layer the neural network individually. This means pruning deci- sions are based on a layer-specific analysis, often using metric like the magnitude of the weights to determine which parameters within that layer are least significant and can be removed without substantially impacting the layer's output. By selec- tively reducing the number of parameters each layer, layerwise pruning can effectively decrease the computational requirements and memory foot- print of language models while maintaining their accuracy. The layerwise approach offers an ad- vantage in that it provides more granular level of control over the pruning process, which can be beneficial in preserving model performance while achieving efficiency gains.
#2
3e-5
Adam W
426
20
le-2
#3
5c-5
SGD
Random
30
lo-2
#4
2e-5
Adam W
302
10
le-3
#5
4e-2
Adam W
Random
30
le-2
#6
5c-5
SGD
42
3
lo-2
N
#7
le-5
Adam W
107
20
le-3
#8
3e-5
Adam
Random
5
le-2
N
#9
2e-5
Adam
302
30
le-3
#10
2e-5
SGD
Random
15
lc-2
Table 8: A Range Hyperparameters and Settings for Weight Averaging
sists (Zheng et al., 2022: Du et al., 2023; Xu et al.. 2021; Liang et al., 2021; Xi et al., 2022), and the underlying causes of low robustness in language models remain elusive.
E.2 Post-Training Pruning
Pruning methods can be categorized into Post- Training Pruning and In-Training Pruning accord- ing to the pruning methods need extra model retraining. In the former, we are given a trained but uncompressed model, together with small amount of calibration data. we must produce accurate compressed model in one shot, i.e., single com- pression step, without retraining and with limited computational costs. This is motivated by practical scenarios such as the large language models, which are hard to train or even finetune because of the complicated training process. In this paper, our method is a Post- Training pruning method.

---

layar 12
ADT Ourr (2s)
Toyer
layer
layer
lever
layer 10
layer?
layer
leyer
layer
layar 8yer 9
layer
layer
layer
layer 10
layer
layer 2
12
la seayott
layer
layer
layer 32
layer 11
layer
loyer
lever
(a)
lever 4
por cure 1701
leyer
layer
layer
layer
layer
layer
layer 3
leyer
layer
layer
layer 6
layer
layer
layer
layer
layer
layer 10
layer
layer's layer
layer
layer
layer
laver Rayer
layer-10
layer 9
-00
layer 11
laygr
layar
layer 12
Iaygr
(b)
Figure 5: Visualization Sentence Embeddings. We've compared the distance of sentence embeddings between the robust and dense model (red), the sparse language models generated with IMP+FreeL.B (green), and the sparse language models created using our method (blue). Figure 5a displays the two-dimensional representation of the embeddings from different layers of various models for sentence (" ("allows us hope that nolan is prepped to embark major career commercial yet shrewd scriptwriter"). Similarly. Figure 5b showcases the two-dimensional representation of the embeddings from different layers of various models for sentence ii ("allows to hope that nolan is poised to embark major career commercial yet inventive filmmaker"). Note that sentence originates from SST2 dataset, and all three models accurately predict its label. On the other hand, sentence ii, an adversarial sample generated from sentence i, only correctly predicted by robust and dense model and our sparse language model. We the embedding of the first token ([CLS)) the representation of sentences, as the model uses this for the final classification. Clearly, our method can generate embeddings and features that align closely with the robust and dense model under adversarial attacks.


Reviews:
[{"review_id": "aQPoMkMm9K", "reviewer": "Reviewer_q9t9", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Question: Hessian Matrix operation part is too inefficient\n\nWe are grateful for the reviewer's keen observation concerning the efficiency of the Hessian Matrix operation in our study. Please see our responses below to clarify the concern. Also, if you feel that your original concern has been resolved, we would appreciate it if you would update your evaluation to reflect this. Thank you!\n\nWe recognize the importance of addressing this highlighted concern. However, Grasping the intricate balance between computational and memory complexities and their broader implications is crucial. **To provide clarity, we offer an in-depth analysis of computational complexities from both `micro` and `macro` viewpoints, contrasting it with approaches that necessitate model retraining.** Furthermore, we elucidate other pivotal design elements in our method that bolster the overall efficiency.\n\n### 1. **Analysis of Complexity**:\n  - **`Micro Perspective - Computation`**: When considering models like Bert_base and Bert_large, the computational requirements for the Hessian Matrix of one layer do not exceed that of model retraining in most cases. To clarify it, we analyze the complexity of our method step by step based on one linear layer.\n\n    **Algorithm**: Prune a linear layer $l$ of BERT with target sparsity `s` and calibration data `X`\n\n    1. **Input:** \n        - Collect original $X$, $W$, $Y$ for $l$.\n\n    2. **Procedure:** Pruning $l$\n\n        1. Set $W$, $X$, $Y$ ← $l$\n\n        ---\n\n        - _Adaptive Update_:\n\n            1. Calculate $H^{-1}$ ← $(XX^{T})^{-1}$\n            2. Set $W$ ← $H^{-1}X^{T}Y$\n\n        ---\n\n        - _Pruning with Hessian Matrix_:\n\n            1. Set $d_{in}$ ← input dimension.\n            2. Set $k$ ← int($d_{in} \\cdot s$).\n            3. **for** $j$ from 1 to $k$ (parallel in rows):\n                1. Calculate $p$ ← $argmin_{p\\in{d_{in}}}\\frac{1}{[H^{-1}]{pp}} \\cdot [W]_{p}^{2}$.\n                2. Set $W$ ← $W-[H]_{:,p}^{-1}\\frac{1}{[H^{-1}]{pp}}\\cdot [W]{p}$.\n                3. Set $A$ ← $[H]^{-1}_{:,p}$\n                4. Set $B$ ← $[H]^{-1}_{p,:}$\n                5. Set $H^{-1}$ ← $H^{-1}-\\frac{1}{[H^{-1}]{pp}}AB$\n                6. Remove $[W]_{p}$ from $W$\n\n        ---\n\n        - _Adaptive Update_:\n\n            1. Set $Y$ ← $WX$.\n            2. Update $X$ with post-process($Y$) for next layer\n\n    **To clarify, we present more notations used to calculate complexity below**\n    \n    - $\\( s \\)$ as the sparsity ratio (a value between 0 and 1).\n    - $\\( d_{in} \\)$ as the input dimension of the linear layer.\n    - $\\( d_{out} \\)$ as the output dimension (which will align with the weight matrix's other dimension).\n    - $\\( d \\)$ as $\\( d_{in} \\times d_{out} \\)$, representing the overall size of the weight matrix.\n    - $\\( n \\)$ as the batch size.\n    - $\\( seq \\)$ as the sequence length.\n\n    **Let's break down the computational complexity of each step again:**\n     1. **Adaptive Update**:\n        - Matrix multiplication $(X_{i}X_{i}^{T})$: Given $(X_{i})$ is of shape $(n \\times seq, d_{in})$, and $(X_{i}^{T})$ will be $(d_{in}, seq \\times n)$, resulting in a matrix of shape $(d_{in} \\times d_{in})$. The complexity is $(O(n \\times seq \\times d_{in}^2))$.\n        - Inverting the matrix: $(O(d_{in}^3))$.\n        - $(H_{i}^{-1}X_{i}^{T}Y_{i})$: $(O(n \\times seq \\times d_{in} \\times d_{out}))$.\n\n    2. **Pruning with Hessian Matrix**:\n        - The outer loop runs for $(d_{in})$ iterations. For each row of $(W_{i})$, the inner loop, determined by $( k = int(d_{in} \\times s) )$, runs.\n            - Finding the argmin: $(O(d_{in}))$.\n            - Updating $(W_{i})$: $(O(d_{in} \\times d_{out}))$.\n            - Computing tmp: $(O(d_{in} \\times d_{out}))$.\n            - Updating $(H_{i}^{-1}): (O(d_{in} \\times d_{out}))$.\n            - Removing row from $(W_{i}): (O(d_{in} \\times d_{out}))$.\n\n        The complexity for the inner loop is $(O(k \\times d_{in} \\times d_{out}))$. The combined complexity for the entire pruning phase becomes $(O(d_{in}^2 \\times s \\times d_{in} \\times d_{out}) = O(d_{in}^3 \\times s \\times d_{out}))$.\n\n    3. **Adaptive Update (2)**:\n        - $\\(Y_{i} = W_{i}X_{i}\\)$: This is matrix multiplication with complexity $\\(O(n \\times seq \\times d_{in} \\times d_{out})\\)$.\n\n    4. **Combining all calculations for one layer:**\n          - $\\[O(2n \\times seq \\times d_{in} \\times d_{out} + n \\times seq \\times d_{in}^2 + 2d_{in}^3 + d_{in}^3 \\times s \\times d_{out})\\]$\n\n    5. **Dominant terms:**\n          - $\\[O(d_{in}^3 \\times d_{out})\\]$\n\n    **Based on the above analysis, the complexity of pruning one layer is:** $[O(d_{in}^3 \\times d_{out})]$\n    \n    > **NOTE**: The complexity of the algorithm is also substantiated by the paper `(Frantar and Alistarh; 2022)` available at: https://arxiv.org/pdf/2208.11580.pdf**)\n\n    **Key observations:**\n    - **This complexity is independent of batch size** $(n)$, because we use calibration data, which restricts (n) to small values like 128 ~ 1024.\n    - **The major factor driving the complexity is the cubic relationship with** $(d_{in})$, which for larger values of $(d_{in})$ can become significant.\n\n    **Let's compare the computational complexity with traditional layer retraining:**\n\n    For training one layer, the complexity is approximately: $[O(n \\times seq \\times d_{in} \\times d_{out})]$\n\n    **Key observations:**\n     - **Here, the complexity is directly proportional to batch size** $(n)$, which becomes massive due to large datasets during training. This means that as the dataset size grows, the training complexity scales accordingly.\n     - While the pruning method's complexity is invariant with respect to (n), training complexity grows, making it computationally demanding for larger datasets or longer sequences.\n\n    **Comparative Insights:**\n\n    - **Batch Size \\(n\\)**:  Our pruning method leverages calibration data, limiting $(n)$ to smaller values (128-1024). This starkly contrasts traditional training, where $(n)$ can be considerably larger due to extensive datasets, amplifying its computational demands.\n\n    - **Dimensionality Dependency**: The pruning algorithm has a cubic dependency on $(d_{in})$, which can make it computationally intensive, especially for layers with a large $(d_{in})$. On the other hand, traditional training has a linear relationship with both $(d_{in})$ and $(d_{out})$.\n\n    - **Aggregate Computational Load**: Considering the entire training process involving multiple passes (epochs) over the data, the cumulative computational requirement can be vast. Yet, it's essential to emphasize that despite being executed once, our pruning method has a hefty computational load, especially for specific layers.\n\n    **In conclusion**, while our pruning method's computational demands are undeniably hefty, particularly for layers with a large $(d_{in})$, it's imperative to recognize the considerable computational burden posed by traditional training due to its scaling with the extensive dataset size. We believe that understanding this trade-off is crucial when considering the suitability and efficiency of our pruning approach vs. traditional retraining.\n\n  - **`Macro Perspective - Time`**: While our method does exhibit a **layerwise dependency**, which could potentially increase processing time, it's important to note that the number of layers in language models is limited. This means the time to complete the pruning process can be easily predicted with promised results as a reward.\n\n### 3. **Layer-by-Layer Computation for Resource Efficiency**: \n\nWhile the sum of Hessian Matrix computations of the entire language model is time-intensive, our approach uniquely addresses this by employing a layer-by-layer resolution strategy. This methodology means there's no necessity to simultaneously load the entire model into the memory of computational resources. Consequently, from a memory allocation standpoint, our pruning with the Hessian Matrix can be viewed as a resource-saving measure.\n\n### 3. **Efficient Post-training Pruning**: \n\nA post-training pruning strategy is at the heart of our methodology. Unlike many other approaches that might require recurrent training sessions or exhaustive reiterations, ours stands out in its ability to save significant resources by strategically avoiding these processes.\n\n### 4. **Justifying Computational Commitment**: \n\nWhile it's acknowledged that pruning with the Hessian Matrix does possess computational time costs, it's paramount to understand our larger vision. The ultimate objective isn't merely to save time but to sculpt a model characterized by three pillars: sparsity, robustness, and high performance. Such a model offers considerable advantages in real-world scenarios, particularly operational ones. Thus, the computational expenses encountered in the training phase can be viewed less as costs and more as strategic investments. Investments we're confident will provide tangible returns when our model is deployed in actual applications.\n\n### 5. **Objective and Potential Method:**\n\nDrawing from established pruning methods utilizing the Hessian Matrix, we have pioneered an adaptive approach that stands out from the rest. Instead of merely focusing on the traditional sparsity-accuracy trade-off, our method delves deep into a holistic consideration of four pivotal dimensions: sparsity, accuracy, robustness, and pruning cost. **While we acknowledge the inherent time-consuming nature of Hessian Matrix computation, it's essential to note that our approach does not introduce this challenge.** Nonetheless, we are actively exploring advanced or approximate methods to expedite this process further. As a case in point, we are considering the utilization of an approximation function to concurrently estimate the inverse of the Hessian Matrix for all rows of weight, a technique underscored in the recent paper by `Elias Frantar (2023)` [https://arxiv.org/pdf/2301.00774.pdf].\n\n### 6. **Conclusion**:\n\nIn response to the reviewer's initial concerns, we emphasize a nuanced understanding of algorithm complexities, memory efficiency, resource-saving, trade-offs, and possible solutions. While inherent costs are tied to our approach, we believe they are offset by its many benefits and the broader objective it supports. Viewed holistically, our method offers a balanced interplay between computational demands and the primary goals of model robustness, efficiency, and real-world relevance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Strong performance of the adversarial robustness assessment compared to other methods.\n2. Clear writing, pseudo-code, and demonstration.\n3. Investigated how sparsity impact the robustness is less than assumed.", "reasons_to_reject": "1. The Hessian matrix operation part is too inefficient ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper shows that positive correlation between robustness of language model and its knowledge coverage. It also proposes a novel robust pruning techniques tat are adaptive to knowledge retention. It trains multiple knowledge models and only average the contributing model weights, and then applies adaptive pruning method layer-wise to get the robust and sparse model.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "818l8aqPnm", "reviewer": "Reviewer_MLxF", "summary": "", "questions": "1. In Line 240 - 246, the authors observes that the cause of adversarial attacks in NLP is that the semantically substitution words have very distinctive embedding information with the original words. First, does this argument hold for most of the adversarial attacks? Second, even if this argument holds, why this leads to the \"highly close alignment between the sparse and dense language models\". I would suggest the author to provide more intuitions or supporting experimental results for this.\n\n2. From my perspective, security-related papers would normally evaluate their defense methods against multiple attack methods. Why do you only choose TextFooler? Could you provide more experiments against other popular adversarial attacks in text classification task?\n\n3. Regarding the setting for experiments in Table 1, could you explain why you use the weights from Weight Average to initialize most of the baseline methods?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their insightful comments and questions, which have offered us a clear perspective on areas for improvement. Please see our responses below to clarify the main concerns. Also, if you feel that your original concerns have been resolved, we would appreciate it if you would update your evaluation to reflect this. Thank you!\n\n## Novelty Concern\n\nNovelty Issue. I have concerns about the novelty of the paper since it seemingly just combines several interesting methods to solve the problem of \"robustness under model pruning.\n\n---\n\nThank you for raising the question about the novelty of our work. Our paper goes beyond just a simple amalgamation of existing methods; it introduces a distinctive adaptive pruning mechanism that addresses the core challenges persistent in the domain. Here's how our approach distinguishes itself:\n\n1. **Deeper Insight into Adversarial Attacks**: Our paper critically examines the root cause behind the susceptibility of sparse or pruned language models to adversarial attacks. Through our observations, we argue that the robustness of a sparse language model is directly proportional to its retained pre-trained knowledge. This insight is pivotal as it shifts the paradigm from merely pruning to maintaining critical semantic understanding, which is instrumental in countering adversarial threats.\n\n2. **Adaptive Pruning Mechanism**: The main innovation lies in our Ada-Pruning approach. While traditional pruning mechanisms often introduce significant reconstruction errors, our method emphasizes minimizing disruptions to dense language models' embedding and feature space. The novelty is evident in our iterative elimination of weights and the subsequent recalibration based on the Hessian Matrix. This iterative, layer-wise consideration ensures minimal loss of pre-trained knowledge.\n\n3. **Holistic Consideration of Model Layers**: Our approach is not siloed to individual layers. Recognizing that the error in one layer can propagate and impact subsequent layers, we introduce the adaptive updating of the Hessian Matrix. This adaptive mechanism accounts for the cumulative effect of errors, ensuring that subsequent layers are pruned with the most updated information possible.\n\n4. **Optimal Dense Weight Calculation**: Another unique aspect is our attention to the dense weight after pruning preceding layers. We identified that using the original weight matrix in conjunction with inputs that have been altered due to accumulated pruning errors can introduce significant discrepancies. To our knowledge, our solution to recalibrate dense weights in line with updated inputs is a novel contribution that better aligns the pruned model with its dense counterpart.\n\n5. **Cost-Effectiveness**: Our methodology is uniquely post-training, eliminating the need for rigorous retraining processes. This improves efficiency and makes our method more practical for deployment.\n\n6. **Demonstrable Superiority**: We've backed our method with extensive experiments, which, as the results indicate, offers a compelling trade-off between accuracy, sparsity, robustness, and cost, outperforming other state-of-the-art methods.\n\n7. **In conclusion**, while our work draws from the broader framework of pruning and robustness literature, it incorporates novel mechanisms, insights, and solutions that significantly differentiate it from existing methods. We sincerely believe that our adaptive pruning strategy provides a fresh, efficient, and more holistic solution to the age-old challenge of robustness under model pruning.\n\n## Question 1:  \n\nIn Line 240 - 246, the authors observes that the cause of adversarial attacks in NLP is that the semantically substitution words have very distinctive embedding information with the original words. First, does this argument hold for most of the adversarial attacks? Second, even if this argument holds, why this leads to the \"highly close alignment between the sparse and dense language models\". I would suggest the author to provide more intuitions or supporting experimental results for this.\n\n---\n\nWe appreciate the opportunity to clarify the points you've highlighted, especially concerning Lines 240-246 and the alignment between sparse and dense language models in the face of adversarial attacks.\n\n1. **Misinterpretation of the Claim**: Firstly, we'd like to address a potential misinterpretation. Our main concern is not outlining the general cause of adversarial attacks in NLP models. Our claim zeroes in on the susceptibility of sparse language models against adversarial attacks when contrasted with their dense counterparts. Specifically, we suggest that while a dense model might be robust, traditional pruning often results in a sparse model that loses this robustness.\n\n2. **Why Close Alignment?**: Given this context, the goal of maintaining a \"highly close alignment between the sparse and dense language models\" is a strategy to preserve as much of the inherent robustness of the dense model in the pruned, sparse version. Through our observations, we've identified that sparse models, post-pruning, deviate considerably in their word embeddings and intermediate features from their robust, dense counterparts. Our intuition is that by closely aligning the sparse model with the robust dense models in both embedding and feature space, we can potentially retain the original model's robustness after pruning.\n\n3. **Adversarial Samples and Embeddings**: Your question on the universality of our observation in relation to most adversarial attacks is duly noted. Our findings primarily draw from a commonly employed technique in NLP where adversarial samples are generated by substituting semantically similar words. We designed our solution with this form of adversarial attack in mind. Given your feedback, we aim to test our method against various adversarial attacks. We will update the results accordingly based on this extension of our tests described in Question 2.\n\n4. **Summary**: We are committed to revising the manuscript to offer clearer explanations and insights on these aspects. We believe that with these adjustments, the manuscript's content and its significance will be more comprehensible. Your feedback has been invaluable in pointing us in the right direction to refine our work.\n\n## Question 2:\n\nFrom my perspective, security-related papers would normally evaluate their defense methods against multiple attack methods. Why do you only choose TextFooler? Could you provide more experiments against other popular adversarial attacks in text classification tasks?\n\n---\n\nThank you for your feedback, especially concerning the range of adversarial attack methods we've tested our defense against. We acknowledge that comprehensive testing against various adversarial attacks is pivotal for a security-related paper.\n\n1. **Selection of TextFooler**: Our initial choice of TextFooler was driven by its recent popularity and effectiveness in the NLP community. However, we recognize the importance of evaluating our defense method against a diverse range of adversarial attack techniques to provide a more comprehensive validation of our approach.\n\n2. **Addition of New Experiments**: In response to your feedback, we have expanded our experiments to encompass two more popular adversarial attacks: `BERT-Attack (Li et al., 2020)` and `TextBugger (Li et al., 2018)`. By leveraging BERT, BERT-Attack ensures fluency and semantic preservation in the adversarial text it generates. Meanwhile, TextBugger employs character-level and word-level perturbations to create adversarial examples, providing another dimension of challenge for our defense method. \n\n    **We compared our method with the state-of-the-art (SOTA) methods, namely `RobustT` ([source](https://aclanthology.org/2022.acl-long.157.pdf)) and `EarlyRobust` ([source](https://aclanthology.org/2022.emnlp-main.569.pdf)). Our approach consistently demonstrated superiority in the robustness of sparse language models across various sparsity levels.**\n\n    | Method | Dataset   | Attack | Sparsity | Accracy | Accracy under attack |\n    |----------|-----------|---------------|:----------------:|:----------:|:------------:|\n    | Ours | SST2      | TextBugger   | 2x      |   88.31%   |  **50.34%**   |\n    | Ours | SST2      | TextBugger   | 4x       |  86.93% |  **49.08%**  |\n    | Ours | SST2      | TextBugger   | 8x       | 85.6% |  **48.85%**   |\n    | RobustT | SST2      | TextBugger   | 2x      |   90.5%  |  35.6%   | \n    | EarlyRobust | SST2      | TextBugger   | 2x       |  91.2  |  36..7% |\n    |----------|-----------|---------------|----------------|----------|------------|\n    | Ours | SST2      | BERT-Attack  |  2x     |  88.31% |  **51.95%**   | \n    | Ours | SST2      | BERT-Attack    |  4x       | 86.93% | **50.57%**   |\n    | Ours | SST2      | BERT-Attack   |  8x       | 85.6% |  **49.32%**  | \n    | RobustT | SST2      | BERT-Attack    | 2x       |  90.5% |  28.3%   | \n    | EarlyRobust | SST2      | BERT-Attack    | 2x     |  91.2% |  30.2% | \n    |----------|-----------|---------------|----------------|----------|------------|\n    | Ours | IMDB      | TextBugger   | 2x   | 94.2% | **58.2%**   |\n    | RobustT | IMDB      | TextBugger   | 2x     |  93.2% | 46.1%   | \n    | EarlyRobust | IMDB      | TextBugger   | 2x   | 90.7% | 48.7%   |\n    |----------|-----------|---------------|----------------|----------|------------|\n    | Ours | IMDB      | BERT-Attack    | 2x      | 94.2% | **52.1%**   |\n    | RobustT | IMDB      | BERT-Attack    | 2x    |  93.2%   |  43.1%  |\n    | EarlyRobust | IMDB      | BERT-Attack    | 2x     | 90.7% | 43.5%  |\n    |----------|-----------|---------------|----------------|----------|------------|\n    | Ours | AGNews    | TextBugger   | 2x      | 93.2% | **62.0%**  | \n    | RobustT | AGNews    | TextBugger   | 2x      | 94.8% | 44.1%   | \n    | EarlyRobust | AGNews    | TextBugger   | 2x      | 94.1% | 46.2%   |\n    |----------|-----------|---------------|----------------|----------|------------|\n    | Ours | AGNews    | BERT-Attack   | 2x     | 93.2% | **70.8%**  |\n    | RobustT | AGNews    | BERT-Attack    | 2x    | 94.8% |  36.8%   | \n    | EarlyRobust | AGNews    | BERT-Attack    |  2x    | 94.1% | 39.3%   | \n\n    > **Note**: The hyperparameter max_candidates of Bert-Attack is set to 8.\n\nThrough these expanded experiments, we hope to thoroughly examine our defense method's efficacy against various adversarial attacks across different datasets and sparsity levels.\n\n## Question 3:\nRegarding the setting for experiments in Table 1, could you explain why you use the weights from Weight Average to initialize most baseline methods?\n\n---\n\nThank you for your insightful observation regarding initializing our baseline methods using weights from Weight Averaging. Your inquiry allows us to elaborate on a pivotal aspect of our methodology.\n\n1. **Challenge with Surface-Level Features**: A central realization we embraced in our research is the propensity of language models to lean on surface-level or spurious features in data. While these features may benefit certain tasks, they become a liability when defending against adversarial attacks. An inevitable question arises: If a sparse language model struggles against adversarial threats, is the failure attributed to the inherent shortcomings of the dense model's reliance on these superficial features, or is it a by-product of the pruning process itself?\n\n2. **Our Rationale for Using Weight Averaging**: To dissociate the effects of the pruning methodology from any intrinsic vulnerabilities of the dense model, it's crucial to begin with a robust foundation. Weight Averaging serves this purpose in our research. As detailed in our manuscript `[Sec 4.2, Weight Averaging for Robust Dense Model]`, this technique synergizes the strengths of different models, reducing their individual susceptibilities to misleading surface features. By combining models trained under diverse hyperparameters and settings (as highlighted in Table 4), we aim to craft a model less influenced by surface patterns in the data. Consequently, any pruned version of this model ideally retains this inherent robustness.\n\n3. **Why Not Other Methods?**: While alternative methods, such as adversarial training, are available, our experiments revealed that relying solely on adversarial training doesn't consistently produce a robust model. Our decision to utilize weight averaging allows us to achieve a robust, dense model with enhanced stability, ensuring a dependable foundation resistant to adversarial challenges.\n\n4. **Delving Deeper**: Our Weight Averaging technique is methodically detailed in Algorithm 2 within the Appendix for a more detailed understanding. This algorithm ensures an optimal blend of model weights, contributing to our goal of adversarial resistance.\n\n5. **Summary**: Our choice of Weight Averaging as an initializer is driven by our commitment to ensure our pruning methods are assessed on a consistently robust foundation. This approach assures that any observed adversarial vulnerabilities reflect the pruning process and are not remnants of latent frailties in the dense model.\n\n## Reference Concern 1\nIn Line 111, the authors mention, \"The extensive experiments well support our statement.\". Could you add some references or explanations here?\n\n---\n\n\nThank you for drawing attention to Line 111. We understand your request for clearer explanations or references supporting our statement. Here's a breakdown of our rationale:\n\n- **Meaning of \"Extensive Experiments\":**\n  - Our use of the term \"extensive experiments\" refers to a comprehensive series of tests and validations conducted to fortify our claim.\n  - These experiments are pivotal to the proof and are intricately tied to the hypotheses derived from the content presented prior to Line 111.\n\n- **Direct Correlation with Hypotheses:**\n  - We have discussed the susceptibility of sparse or pruned language models to adversarial attacks due to a lack of pretrained knowledge\n  - Our experiments are tailored to:\n    - Measure the degree of this susceptibility.\n    - Validate our assertion that without pre-trained knowledge, sparse language models treat substitute words merely as integer identifiers.\n\n- **Breadth of Experiments:**\n  - Addressing the multifaceted challenge of balancing sparsity, accuracy, robustness, and cost necessitated a wide-ranging experimental design.\n  - Our tests included:\n    - Different adversarial attack strategies.\n    - Varying degrees of pruning.\n\nIn light of the above points, we hope our response clarifies the foundational role of our extensive experiments in validating our claims and hypotheses. We appreciate your feedback and will ensure a clearer representation in the revised manuscript.\n\n## Reference Concern 2\n\nIn Line 202, could you add some references to the \"previous studies\"?\n\n---\n\nThe study referenced in Line 202 is taken from the paper by `Frantar and Alistarh (2022)`. The details of this paper can be found at https://arxiv.org/pdf/2208.11580.pdf.\n\n## Presentation Concern\nFor the metric, I would suggest that the author use popular abbreviations. E.g., ASR for Attack Success Rate.\n\n---\nThank you for your insightful feedback on our paper presentation. Your suggestion to use \"ASR\" for \"Attack Success Rate\" is straightforward and will enhance the paper's readability. Given its benefits, we will integrate this amendment. \n\n**We genuinely value your feedback, as it allows us to articulate and refine our work more clearly.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Generally the author provide very rich evaluation results on the proposed method and a very broad analysis on the method.\n2. The writing is easy to follow. For example, the authors have provided intuitions on the proposed method.\n3. The presentation is generally good.", "reasons_to_reject": "1. Some arguments are not supported well. See {Questions For The Authors} for details.\n2. The evaluation part is not thorough enough. See {Questions For The Authors} for details.\n3. Novelty Issue. I have concerns on the novelty of the paper since it seemingly just combines several interesting methods to solve the problem of \"robustness under model pruning\".", "questions_to_authors": "1. In Line 240 - 246, the authors observes that the cause of adversarial attacks in NLP is that the semantically substitution words have very distinctive embedding information with the original words. First, does this argument hold for most of the adversarial attacks? Second, even if this argument holds, why this leads to the \"highly close alignment between the sparse and dense language models\". I would suggest the author to provide more intuitions or supporting experimental results for this.\n\n2. From my perspective, security-related papers would normally evaluate their defense methods against multiple attack methods. Why do you only choose TextFooler? Could you provide more experiments against other popular adversarial attacks in text classification task?\n\n3. Regarding the setting for experiments in Table 1, could you explain why you use the weights from Weight Average to initialize most of the baseline methods?", "paper_topic_and_main_contributions": "Pruning is a popular method in compressing machine learning models, as it decreases the models' size and increase the inference efficiency. Currently, the objective of pruning has been extended from \"accuracy\", \"sparsity\" to \"robustness\", which focuses on the models' capability in defending against the adversarial attacks. In the paper, the author proposes a method that could better balance the three objective. Specifically, it first delves into the root cause of adversarial attacks and proposes the corresponding methods for increasing the robustness of the models. then, the authors uses the weight average method to further alleviate the adversarial attacks. Various experiments across different datasets demonstrate the efficacy of the proposed method.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. For the metric, I would suggest author to use popular abbreviations. E.g., ASR for Attack Success Rate."}, "extra_text": {"Missing_References": "1. In Line 111, the authors mention that \"The extensive experiments well support our statement.\". Could you add some references or explanations here?\n2. In Line 202, could you add some references on the \"previous studies\"?"}}, {"review_id": "6AvsLntSaQ", "reviewer": "Reviewer_VnKg", "summary": "", "questions": "How does Figure 5 show that your method generates embeddings that align more closely?  Do you have quantitative results?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for their insightful comments and questions, which have offered us a clear perspective on areas for improvement. We have carefully gone through the feedback and summarized the major points raised. Below, we have listed the primary concerns and questions from the review to address them systematically in our rebuttal.\n\n---\n\n**Questions for the Authors:**\n- How does Figure 5 illustrate that your method generates embeddings that align more closely with the robust and dense model? Do you have any quantitative results to support this claim? (**We quantify the distance with the Equation 3**)\n- What does K represent in Equation 3?\n- Compared to retraining, how computationally intensive is your method?\n- Could you clarify the use of data points in the \"Impact of Calibration Data\" section?\n\n**Feedback and Suggestions:**\n- Reframing the introduction to reflect the paper's content more accurately is recommended.\n- Including a few adversarial examples at the beginning of the paper would provide a clearer context for readers.\n- Some sections, like those discussing the Hessian and Algorithm 1, would benefit from further clarification.\n- The placement and reference to Figure 5 need to be consistent.\n\nThank you for the review and for bringing up these questions again. Please see our responses below to clarify the main concerns. Also, if you feel that your original concerns have been resolved, we would appreciate it if you would update your evaluation to reflect this. Thank you!\n\n## Questions for the Authors\n---\n\n### Question 1:\nHow does Figure 5 illustrate that your method generates embeddings that align more closely with the robust and dense model? Do you have any quantitative results to support this claim? (**We quantify the distance with the Equation 3**)\n\n---\n\n**Acknowledgment:** We understand the confusion stemming from the qualitative nature of Figure 5.  \n\n**Resolution:** In response to this concern, we undertook a quantitative analysis to gauge the distance between the embeddings produced by our approach and those from the dense model. Across the board, our technique outperformed other pruning methods regarding similarity scores. This distance measurement is detailed in Equation 3.\n\n**Result Table:**  \n\n**`Distance for datapoint from original SST2 dataset`**\n| Layer | Distance with IMP + ADT (2x) | Distance with Ours (2x) |\n|-------|-------------------------|--------------------|\n| 1 | 0.0086 | 0.0000 |\n| 2 | 0.0144 | 0.0015 |\n| 3 | 0.0156 | 0.0014 |\n| 4 | 0.0193 | 0.0017 |\n| 5 | 0.0324 | 0.0067 |\n| 6 | 0.0763 | 0.0255 |\n| 7 | 0.1299 | 0.0496 |\n| 8 | 0.2530 | 0.1308 |\n| 9 | 0.1880 | 0.0958 |\n| 10 | 0.2804 | 0.1254 |\n| 11 | 0.4932 | 0.2322 |\n| 12 | 0.6872 | 0.2231 |\n\n**`Distance for adversarial sample generated from above datapoint`**\n| Layer | Distance with IMP + ADT (2x) | Distance with Ours (2x) |\n|-------|-------------------------|--------------------|\n| 1 | 0.0086 | 0.0000 |\n| 2 | 0.0142 | 0.0015 |\n| 3 | 0.0258 | 0.0012 |\n| 4 | 0.0407 | 0.0017 |\n| 5 | 0.1319 | 0.0069 |\n| 6 | 0.0967 | 0.0253 |\n| 7 | 0.1478 | 0.0501 |\n| 8 | 0.2547 | 0.1078 |\n| 9 | 0.2767 | 0.0749 |\n| 10 | 0.3909 | 0.1049 |\n| 11 | 0.7317 | 0.0625 |\n| 12 | 0.6903 | 0.0349 |  \n\n**Results indicate that our method consistently achieves lower distance scores across all layers for original SST2 data points and adversarial samples. This phenomenon becomes even more pronounced under adversarial attack scenarios.** Specifically, our technique's embeddings are closer to the dense model than those generated using the IMP + ADT  method.\n\n## Question 2: \nWhat does K represent in Equation 3?\n\n---\n\nThank you for pointing out the ambiguity regarding the representation of \\(K\\) in Equations 3 and 4. We sincerely apologize for the oversight.\n\nTo address your question: in the context of the paper, the goal is to find sparse parameters \\(\\hat{W}_l\\) that align closely with the dense model while ensuring robustness against adversarial attacks. This alignment is quantified by minimizing the discrepancy between the dense and sparse layer outputs. The term \\(k\\) in the provided equations sets a constraint on the sparsity of the pruned model. Specifically, \\(k\\) dictates the maximum number of non-zero weights (or parameters) the pruned model is allowed to retain.\n\n**In simple terms, \\(K\\) in Equations 3 and 4 designates the total number of weights that remain non-zero after the pruning process, thus serving as a threshold to control the model's sparsity.**\n\nWe realize the importance of clarifying this in our paper, and we will ensure that this point is revised and made more explicit in the subsequent version. Again, we appreciate your valuable feedback, and we apologize for any confusion caused.\n\n### Question 3: \nCompared to retraining, how computationally intensive is your method?\n\n---\n\nOur approach encompasses two main components: the **Average Weighting for Robust Dense Model** and the **Adaptive Pruning for Robust Sparse Model**. For clarity and depth, we will individually discuss each component in comparison to the retraining method. Specifically, our examination will be bifurcated into time (macro-level) and computational workload (micro-level).\n\n1. **Average Weighting for Robust Dense Model**:\n\n    - **Purpose and Flexibility**: The weight averaging process for achieving a robust dense model is not a mandatory component for our robust pruning approach. Any effective and swift method to generate a robust dense model is compatible with our core methodology. The choice of average weighting has been made based on its relative stability compared to the more erratic behavior of traditional adversarial training in the realm of NLP. In essence, average weights offer a greedy yet stable means to consistently produce a robust dense model.\n\n    - **Macro Perspective - Time**: The models used for weight averaging operate independently, without inter-dependencies. As such, there's no cumulative time lag due to the sequencing of model execution.\n\n    - **Micro Perspective - Computation**: The computational demand for average weights primarily depends on the number of hyperparameter sets. Notwithstanding, it’s pivotal to understand that while average weights provide stable robust dense models, they can be readily substituted by any other efficient method that swiftly produces a robust dense model.\n\n2. **Adaptive Pruning for Robust Sparse Model**:\n\n    - **`Micro Perspective - Computation`**: When considering models like Bert_base and Bert_large, the computational requirements for the Hessian Matrix of one layer do not exceed that of model retraining in most cases. To clarify it, we analyze the complexity of our method step by step based on one linear layer.\n\n      **Algorithm**: Prune a linear layer $l$ of BERT with target sparsity `s` and calibration data `X`\n\n      1. **Input:** \n          - Collect original $X$, $W$, $Y$ for $l$.\n\n      2. **Procedure:** Pruning $l$\n\n          1. Set $W$, $X$, $Y$ ← $l$\n\n          ---\n\n          - _Adaptive Update_:\n\n              1. Calculate $H^{-1}$ ← $(XX^{T})^{-1}$\n              2. Set $W$ ← $H^{-1}X^{T}Y$\n\n          ---\n\n          - _Pruning with Hessian Matrix_:\n\n              1. Set $d_{in}$ ← input dimension.\n              2. Set $k$ ← int($d_{in} \\cdot s$).\n              3. **for** $j$ from 1 to $k$ (parallel in rows):\n                  1. Calculate $p$ ← $argmin_{p\\in{d_{in}}}\\frac{1}{[H^{-1}]{pp}} \\cdot [W]_{p}^{2}$.\n                  2. Set $W$ ← $W-[H]_{:,p}^{-1}\\frac{1}{[H^{-1}]{pp}}\\cdot [W]{p}$.\n                  3. Set $A$ ← $[H]^{-1}_{:,p}$\n                  4. Set $B$ ← $[H]^{-1}_{p,:}$\n                  5. Set $H^{-1}$ ← $H^{-1}-\\frac{1}{[H^{-1}]{pp}}AB$\n                  6. Remove $[W]_{p}$ from $W$\n\n          ---\n\n          - _Adaptive Update_:\n\n              1. Set $Y$ ← $WX$.\n              2. Update $X$ with post-process($Y$) for next layer\n\n      **To clarify, we present more notations used to calculate complexity below**\n    \n      - $\\( s \\)$ as the sparsity ratio (a value between 0 and 1).\n      - $\\( d_{in} \\)$ as the input dimension of the linear layer.\n      - $\\( d_{out} \\)$ as the output dimension (which will align with the weight matrix's other dimension).\n      - $\\( d \\)$ as $\\( d_{in} \\times d_{out} \\)$, representing the overall size of the weight matrix.\n      - $\\( n \\)$ as the batch size.\n      - $\\( seq \\)$ as the sequence length.\n\n      **Let's break down the computational complexity of each step again:**\n       1. **Adaptive Update**:\n          - Matrix multiplication $(X_{i}X_{i}^{T})$: Given $(X_{i})$ is of shape $(n \\times seq, d_{in})$, and $(X_{i}^{T})$ will be $(d_{in}, seq \\times n)$, resulting in a matrix of shape $(d_{in} \\times d_{in})$. The complexity is $(O(n \\times seq \\times d_{in}^2))$.\n          - Inverting the matrix: $(O(d_{in}^3))$.\n          - $(H_{i}^{-1}X_{i}^{T}Y_{i})$: $(O(n \\times seq \\times d_{in} \\times d_{out}))$.\n\n      2. **Pruning with Hessian Matrix**:\n          - The outer loop runs for $(d_{in})$ iterations. For each row of $(W_{i})$, the inner loop, determined by $( k = int(d_{in} \\times s) )$, runs.\n              - Finding the argmin: $(O(d_{in}))$.\n              - Updating $(W_{i})$: $(O(d_{in} \\times d_{out}))$.\n              - Computing tmp: $(O(d_{in} \\times d_{out}))$.\n              - Updating $(H_{i}^{-1}): (O(d_{in} \\times d_{out}))$.\n              - Removing row from $(W_{i}): (O(d_{in} \\times d_{out}))$.\n\n          The complexity for the inner loop is $(O(k \\times d_{in} \\times d_{out}))$. The combined complexity for the entire pruning phase becomes $(O(d_{in}^2 \\times s \\times d_{in} \\times d_{out}) = O(d_{in}^3 \\times s \\times d_{out}))$.\n\n      3. **Adaptive Update (2)**:\n          - $\\(Y_{i} = W_{i}X_{i}\\)$: This is matrix multiplication with complexity $\\(O(n \\times seq \\times d_{in} \\times d_{out})\\)$.\n\n      4. **Combining all calculations for one layer:**\n            - $\\[O(2n \\times seq \\times d_{in} \\times d_{out} + n \\times seq \\times d_{in}^2 + 2d_{in}^3 + d_{in}^3 \\times s \\times d_{out})\\]$\n\n      5. **Dominant terms:**\n            - $\\[O(d_{in}^3 \\times d_{out})\\]$\n\n      **Based on the above analysis, the complexity of pruning one layer is:** $[O(d_{in}^3 \\times d_{out})]$\n    \n      > **NOTE**: The complexity of the algorithm is also substantiated by the paper `(Frantar and Alistarh; 2022)` available at: https://arxiv.org/pdf/2208.11580.pdf**)\n\n      **Key observations:**\n      - **This complexity is independent of batch size** $(n)$, because we use calibration data, which restricts (n) to small values like 128 ~ 1024.\n      - **The major factor driving the complexity is the cubic relationship with** $(d_{in})$, which for larger values of $(d_{in})$ can become significant.\n\n      **Let's compare the computational complexity with traditional layer retraining:**\n\n      For training one layer, the complexity is approximately: $[O(n \\times seq \\times d_{in} \\times d_{out})]$\n\n      **Key observations:**\n      - **Here, the complexity is directly proportional to batch size** $(n)$, which becomes massive due to large datasets during training. This means that as the dataset size grows, the training complexity scales accordingly.\n       - While the pruning method's complexity is invariant with respect to (n), training complexity grows, making it computationally demanding for larger datasets or longer sequences.\n\n      **Comparative Insights:**\n\n      - **Batch Size \\(n\\)**:  Our pruning method leverages calibration data, limiting $(n)$ to smaller values (128-1024). This starkly contrasts traditional training, where $(n)$ can be considerably larger due to extensive datasets, amplifying its computational demands.\n\n      - **Dimensionality Dependency**: The pruning algorithm has a cubic dependency on $(d_{in})$, which can make it computationally intensive, especially for layers with a large $(d_{in})$. On the other hand, traditional training has a linear relationship with both $(d_{in})$ and $(d_{out})$.\n\n      - **Aggregate Computational Load**: Considering the entire training process involving multiple passes (epochs) over the data, the cumulative computational requirement can be vast. Yet, it's essential to emphasize that despite being executed once, our pruning method has a hefty computational load, especially for specific layers.\n\n      **In conclusion**, while our pruning method's computational demands are undeniably hefty, particularly for layers with a large $(d_{in})$, it's imperative to recognize the considerable computational burden posed by traditional training due to its scaling with the extensive dataset size. We believe that understanding this trade-off is crucial when considering the suitability and efficiency of our pruning approach vs. traditional retraining.\n\n    - **`Macro Perspective - Time`**: While our method does exhibit a **layerwise dependency**, which could potentially increase processing time, it's important to note that the number of layers in language models is limited. This means the time to complete the pruning process can be easily predicted with promised results as a reward.\n\n## Question 4: \nCould you clarify the use of data points in the \"Impact of Calibration Data\" section?\n\n---\n\nThank you for inquiring about the use of data points in the \"Impact of Calibration Data\" section. I'd like to present a more structured explanation to address your question:\n\n1. **Role of Calibration Data, $\\(X\\)$**:\n   \n   The calibration data $\\(X\\)$ is pivotal for our methodology, as it directly influences the computation of the Hessian Matrix. As described in our work, the Hessian $\\(H\\)$ can be derived from $\\(X\\)$ as $\\(H = X^TX\\)$. For the context of language models, $\\(X\\)$ is structured with dimensions $\\( (N, S, D) \\)$. When computing \\(H\\), we reshape this data to a dimension of $\\( (NS, D) \\)$, ensuring it is well-suited for our mathematical operations.\n\n>   An integral part of our methodology is derived from the insights of **\\citet{frantar2022optimal}**. The research underscores that Hessian values across different rows of a weight matrix are independent. This independence facilitates our optimization, where the removal of one data point predominantly affects its respective row value. This crucial property allows us to simplify our Hessian matrix computation, obtaining it directly as $\\(H = X^TX\\)$. Thus, the calibration data $\\(X\\)$ isn't just a dataset; it directly shapes the Hessian matrix, which in turn governs our pruning process.\n\n2. **Significance in Pruning Algorithm**:\n    \n   Referring to the main algorithm provided, the calibration data $\\(X\\)$ underpins the adaptive update step. Within this step, the inverse Hessian $\\(H^{-1}\\)$ is computed as $\\(H^{-1} \\gets (X_iX_i^T)^{-1}\\)$. This relationship is fundamental because the precise calculation of the Hessian matrix ensures the effectiveness of our pruning process.\n\n3. **Exploration of Calibration Data Size**:\n\n   In the \"Impact of Calibration Data\" section, our intention was to scrutinize how different sizes (i.e., varying $\\(N\\)$) of the calibration dataset influence our pruning strategy. Our experiments, as illustrated in Figure 3, conclusively showed an increased robustness and accuracy of sparse language models with a rising number of data points. However, there's a saturation point — a threshold beyond which adding more data ceases to enhance the model's performance.\n\nTo conclude, the calibration data isn't just a variable in our approach — it's foundational. By understanding its impact and optimizing its use, we ensure the effectiveness and efficiency of our pruning process. We hope this comprehensive explanation offers a clear understanding of our method's rationale and its dependence on calibration data. We'll enhance this section with a more in-depth description and practical examples.\n\n## Feedback and Suggestions:\n---\n\n**Reframing the introduction:** We concur that the introduction could benefit from clearer alignment with the paper's content. We have revised it to more accurately capture our focus on proposing a robust pruning method without digressing into the broader adversarial attack landscape.\n\n**Adversarial examples:** Incorporating your suggestion, we've added a section early in the paper showcasing adversarial examples. This will give readers an immediate grasp of the challenges we're addressing.\n\n**Clarification on the Hessian and Algorithm 1:** Your feedback is valuable; we've expanded sections in appendix A. The revised manuscript will provide a clearer explanation of the Hessian's relevance and will break down Algorithm 1 more intuitively.\n\n**Consistency with Figure 5:** We've rectified this inconsistency by moving Figure 5 to the main body of the paper, ensuring its placement matches the reference.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The results are strong against competitors, particularly on accuracy under attack and minimizing the success ratio of adversarial tasks. The method does not require retraining (though they admit in Limitations that their method is also computationally expensive because of \"calculation of the Hessian and its inverse\".  They compare to multiple models from the literature, and have improved performance.  They assess their method at multiple levels of sparsity.", "reasons_to_reject": "The motivation of the paper seems to be to keep weights after pruning similar to in the dense model, however I do not see quantitative results on how similar the resulting model is. Seems like you could calculate this based on equation 4, and I'd like to see that result. Figure 2 of attention score visualization and Figure 5 of a sentence embedding are used as evidence, but these are only on 1-2 examples.   Figure 5 in particular says \"Clearly, our method can generate embeddings and features that align more closely with the robust and dense model under adversarial attacks\", but I do not see how one can conclude this from the image. \n\nI think a bit of a reframing of the intro would be helpful.  The intro states \"this paper investigates why language models are susceptible to adversarial attacks\".  I wouldn't say that the paper addresses this question - rather it proposes one robust pruning method.  Question 1, \"What is the core to defend against adversarial attacks for sparse language models\" also goes unanswered. This issues can be corrected by reframing the paper. \n", "questions_to_authors": "How does Figure 5 show that your method generates embeddings that align more closely?  Do you have quantitative results?", "paper_topic_and_main_contributions": "The authors propose a new robust pruning method for LLMs that also is meant to improve the model's ability  to defend itself against adversarial attack.  They authors report improve results on adversarial robustness assessments vs. multiple existing models. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Figure 5 needs axis labels.\n\nI recommend adding a few adversarial examples early on in the paper so the reader has a nice example of the task you're approaching.\n\nI recommend a little more clarification on the math.  For instance, remind the reader what the Hessian is, and why we're using it. \n\nIn Equation 3, what is K? \n\nCan you make Algorithm 1 - particularly the \"Pruning with Hessian Matrix\" section, more intuitive?\n\nYou mentioned that your method also requires a lot of computation.  How much, compared to retraining?\n\nFigure 5 is mentioned in the main body of the paper, but is in fact in the appendix so should be relabeled, or moved. \n\nSectoin 5.6.2 \"Impact of Calibration Data\", could use more explanation. How are these data points used?"}}, {"review_id": "KFpL0C9A4R", "reviewer": "Reviewer_YNs7", "summary": "", "questions": "1. Does your method need additional tuning to recover the performance loss after pruning?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the insightful comments and questions provided by the reviewer. Your feedback is invaluable for enhancing the quality and clarity of our work. To ensure we address each concern thoroughly, we've summarized the primary questions raised:\n\n1. **Methodology and Additional Tuning**: Does our Hessian-matrix-based method require a post-pruning tuning step to recover performance?\n2. **Choice of Baseline**: Why wasn't the Movement Pruning approach, which retains just 3% of parameters with minimal accuracy loss, chosen as a baseline?\n3. **Origins of Performance Loss and Its Correlation to Pruning**: Could we further elucidate our claim regarding the performance loss post-pruning being attributed to the loss of pretrained knowledge? In light of recent research, notably, Jain and Wallace (2019), is using attention scores to represent pretrained knowledge justified?\n5. **Writing and Stylistic Concerns**: Are there plans to address the stylistic issues, especially in Eqs. (3) and (4), where 'argmin' is in italics and \";\" should be replaced with \"s.t.\"?\n\nThank you for the review and for bringing up these questions again. Please see our responses below to clarify the main concerns. Also, if you feel that your original concerns have been resolved, we would appreciate it if you would update your evaluation to reflect this. Thank you!\n\n ## Question 1 (Methodology and Additional Tuning)\n---\nWe appreciate the reviewer's inquiry about post-pruning tuning. First and foremost, it is essential to emphasize that our method unequivocally does not require additional tuning after pruning to recover performance. This is a testament to our pruning approach's efficacy and stability. \n\nBeyond this point, our experiments did reveal an intriguing observation. Upon obtaining the pruning results, we found that minor fine-tuning could sometimes enhance outcomes regarding clean accuracy and accuracy under adversarial attacks. Yet, this improvement isn't universally consistent. However, prior work `(Xu et al., 2021, https://aclanthology.org/2021.emnlp-main.832.pdf)` claims that such fine-tuning could potentially compromise robustness. While our approach remains solid without needing post-pruning adjustments, we value the reviewer's insight, as it provides thoughtful considerations for future explorations.\n\n## Question 2 (Choice of Baseline):\n---\n\nWe acknowledge the reviewer's suggestion of Movement Pruning (Sanh et al., 2020) as a potential competitive baseline. We initially did not prioritize this method because it primarily emphasizes accuracy and sparsity. Our selection of baselines was driven by works that holistically address accuracy, sparsity, robustness, and pruning cost simultaneously.\n\nHowever, to address the reviewer's concern and provide a comprehensive perspective, we revisited our experiments and included Movement Pruning in our evaluations. The results are showcased in the subsequent table. While our method might slightly lag in clean accuracy compared to this baseline, it's noteworthy that we outperform it in terms of accuracy under attack. This lead in robustness is significant and underscores the value and distinctiveness of our approach.\n\n| Method | Dataset   | Attack | Sparsity | Accracy | Accracy under attack |\n|----------|-----------|---------------|:----------------:|:----------:|:------------:|\n| Ours | SST2      | TextFooler   | 2x      |   88.31%   |  **43.12%**   |\n| Ours | SST2      | TextFooler   | 4x       |  86.93% |  **40.15%**  |\n| Ours | SST2      | TextFooler   | 8x       | 85.6% |  **37.63%**   |\n| Movement Pruning | SST2      | TextFooler   | 2x       | 90.6% |  14.85%   |\n| Movement Pruning | SST2      | TextFooler   | 4x       | 90.5% |  8.27%   |\n| Movement Pruning | SST2      | TextFooler   | 8x       | 90.0% |  9.14%   |\n| Ours | SST2      | TextBugger   | 2x      |   88.31%   |  **50.34%**   |\n| Ours | SST2      | TextBugger   | 4x       |  86.93% |  **49.08%**  |\n| Ours | SST2      | TextBugger   | 8x       | 85.6% |  **48.85%**   |\n| Movement Pruning | SST2      | TextBugger   | 2x       | 90.6% |  24.85%   |\n| Movement Pruning | SST2      | TextBugger   | 4x       | 90.5% |  21.35%   |\n| Movement Pruning | SST2      | TextBugger   | 8x       | 90.0% |  15.13%   |\n\n## Question 3 (Origins of Performance Loss and Its Correlation to Pruning)\n---\n\nThank you for pinpointing a critical aspect of our work and challenging our choice of using attention scores to indicate pre-trained knowledge. Based on your concern and the cited work of Jain and Wallace (2019), we understand the importance of a more definitive measure. Here’s our revised approach:\n\n1. **Acknowledgment of the Concern**:\n   - We acknowledge that solely relying on attention scores may not sufficiently represent the richness and breadth of pre-trained knowledge, and we appreciate the reference to the findings of Jain and Wallace (2019), which challenges its use as a singular metric.\n\n2. **Delineation of Pre-trained Knowledge**:\n   - While we initially proposed attention scores as one metric, we understand that a more comprehensive definition is essential. Pre-trained knowledge encompasses not just attention patterns but semantic understanding, feature embeddings, and the model's ability to generalize across various inputs.\n\n3. **Updated Experimental Approach**:\n   - **Experiment A - Feature Embedding Analysis**: We compare the embeddings produced by pruned and dense models for various inputs. The similarity in embeddings will indicate the retention of pre-trained knowledge. Specifically, we undertook a quantitative analysis to gauge the distance between the embeddings produced by our approach and those from the dense model. Across the board, our technique outperformed other pruning methods regarding feature distance. This distance measurement is detailed in Equation 3.\n\n       **Result Table:**  \n\n       **`Distance for datapoint from original SST2 dataset`**\n       | Layer | Distance with IMP + ADT (2x) | Distance with Ours (2x) |\n       |-------|-------------------------|--------------------|\n       | 1 | 0.0086 | 0.0000 |\n       | 2 | 0.0144 | 0.0015 |\n       | 3 | 0.0156 | 0.0014 |\n       | 4 | 0.0193 | 0.0017 |\n       | 5 | 0.0324 | 0.0067 |\n       | 6 | 0.0763 | 0.0255 |\n       | 7 | 0.1299 | 0.0496 |\n       | 8 | 0.2530 | 0.1308 |\n       | 9 | 0.1880 | 0.0958 |\n       | 10 | 0.2804 | 0.1254 |\n       | 11 | 0.4932 | 0.2322 |\n       | 12 | 0.6872 | 0.2231 |\n\n       **`Distance for adversarial sample generated from above datapoint`**\n       | Layer | Distance with IMP + ADT (2x) | Distance with Ours (2x) |\n       |-------|-------------------------|--------------------|\n       | 1 | 0.0086 | 0.0000 |\n       | 2 | 0.0142 | 0.0015 |\n       | 3 | 0.0258 | 0.0012 |\n       | 4 | 0.0407 | 0.0017 |\n       | 5 | 0.1319 | 0.0069 |\n       | 6 | 0.0967 | 0.0253 |\n       | 7 | 0.1478 | 0.0501 |\n       | 8 | 0.2547 | 0.1078 |\n       | 9 | 0.2767 | 0.0749 |\n       | 10 | 0.3909 | 0.1049 |\n       | 11 | 0.7317 | 0.0625 |\n       | 12 | 0.6903 | 0.0349 |  \n\n       **Our methodology consistently showcases superior performance, achieving minimized distance scores throughout all layers for both the original SST2 data points and the adversarial instances. This superiority is even more evident in scenarios involving adversarial attacks**, where our technique's embeddings are distinctly closer to the dense model in comparison to the IMP + ADT method.\n\n   - **Experiment B - Adversarial Robustness Comparison**: By challenging pruned models against adversarial attacks, we can better understand their semantic retention and robustness. This is certified by our main experiment,  and the pivotal results are presented in Table 1 in our paper.\n\n4. **Clarification & Way Forward**:\n   - While our earlier approach provided a preliminary view, we aim to refine and expand our methodologies to validate our hypothesis more accurately. In addition to the aforementioned experiments, we are also open to other ways to holistically assess pre-trained knowledge's preservation.\n\nIn summary, we sincerely appreciate the feedback. We are committed to refining our research methodologies to offer a clearer and more comprehensive insight into the effects of pruning on the retention of pre-trained knowledge in language models.\n\n## Question 4 (Writing and Stylistic Concerns)\n---\nWe sincerely appreciate the reviewer's feedback on our paper's writing and presentation aspects. We acknowledge the points raised, including the improper italicization of \"argmin\" in Eqs. (3) and (4) and the suggested replacement of \";\" with \"s.t.\". We are committed to enhancing the clarity and precision of our paper, and we will incorporate these suggested changes in our revision to ensure the manuscript meets high presentation standards.\n\nWe are committed to comprehensively addressing these questions in our revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The authors propose an interesting Hessian-matrix-based pruning method.\n2. The experiments show that the proposed method preserves performance well and is robust to adversarial attacks.", "reasons_to_reject": "1. The writing could be improved and some descriptions are not clear.\n2. The chosen baseline seems not competitive.", "questions_to_authors": "1. Does your method need additional tuning to recover the performance loss after pruning?", "paper_topic_and_main_contributions": "In this paper, the authors propose an adaptive tuning based on optimal brain damage, which utilizes the Hessian matrix to guide the pruning process. The proposed method looks interesting because the Hessian matrix can well characterize the curvature of the loss surface to best preserve the performance of pruned models, but it becomes obsolete given the parameter scale of modern neural networks. It is good to see the authors bring this method back based on some recent advances in efficiently estimating the Hessian matrix. However, I have some questions regarding the main content: After the pruning, does your method still need an additional tuning step to recover the performance loss? If so, I believe the authors should choose a more competitive baseline like Movement Pruning (Sanh et al., 2020), which retains only 3% parameters with minimal accuracy loss. The authors also claim the performance loss of pruning originates from the loss of pretrained knowledge. In Section 5.6.1, the authors use attention scores as a proxy of pretrained knowledge. Given the observations from recent research (Jain and Wallace, 2019), it may not be a reasonable choice and the authors should give a clearer definition. Last but not least, the writing of this paper could be improved, e.g., argmin in Eqs. (3) and (4) should not be in italics and \";\" should be replaced by \"s.t.\".", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "argmin in Eqs. (3) and (4) should not be in italics and \";\" should be replaced by \"s.t.\"."}, "extra_text": {"Missing_References": "Sanh, Victor, Thomas Wolf, and Alexander Rush. \"Movement pruning: Adaptive sparsity by fine-tuning.\" Advances in Neural Information Processing Systems 33 (2020): 20378-20389.\n\nJain, Sarthak, and Byron C. Wallace. \"Attention is not explanation.\" arXiv preprint arXiv:1902.10186 (2019)."}}, {"review_id": "V03EGPFZ6s", "reviewer": "Reviewer_1hiE", "summary": "", "questions": "1. Could you conclude the limitations in existing methods? Why is the proposed model able to outperform existing models?\n\n2. One of the proposed techniques, weight averaging, requires obtaining several different versions of a model. Will it increase the computation cost?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's time and effort in evaluating our paper. Your comments and questions provide valuable insights that will aid in enhancing the quality and clarity of our work. Specifically, you've raised the following questions:\n\n- **Question 1**: Could you conclude the limitations in existing methods? Why is the proposed model able to outperform existing models?\n- **Question 2**: One of the proposed techniques, weight averaging, requires obtaining several different versions of a model. Will it increase the computation cost?\n\nThank you for the review and for bringing up these questions again. Please see our responses below to clarify the main concerns. Also, if you feel that your original concerns have been resolved, we would appreciate it if you would update your evaluation to reflect this. Thank you!\n\n## Question 1 （Limitation Related)\n---\n\nThank you for bringing up the clarifications regarding the limitations of existing methods and the distinctiveness of our proposed model.\n\n**Limitations in Existing Methods**:\n\n1. **`[Line 058-063, 068-072]` Robustness-Accuracy Trade-off**: Many current methods, including those by \\citet{zheng-etal-2022-robust} and \\citet{xi-etal-2022-efficient}, struggle to maintain a balanced relationship between robustness (accuracy under adversarial conditions) and standard accuracy. Although some results on particular datasets may suggest optimal performance, a discernible vulnerability emerges in the disparity between standard and adversarially tested accuracy metrics.\n\n2. **`[Line 068-072]` Limited Sparsity**: It's pertinent to mention that numerous prevailing robust pruning techniques achieve only modest sparsity, often less than a 2x compression rate. As sparsity exceeds 2x, these methods often witness a stark reduction in robustness and can even compromise clean accuracy.\n\n3. **`[Line 234-237, 262-267]` Retraining Overheads**: A predominant limitation in many existing pruning strategies is the unavoidable retraining phase, which increases computational demands and extends the time to deployment for pruned models.\n\n**Why Our Method Outperforms Existing Works**\n\n1. **Preservation of Pre-trained Knowledge**: We are anchored in the conviction that the robustness of a pruned model correlates directly with the extent of pre-trained knowledge it retains. Our methodology emphasizes replicating the embedding and feature spaces of the original full models, ensuring that pruned versions maintain substantial semantic insights. This meticulous preservation inherently enhances the model's robustness, facilitating adept handling of sophisticated semantic intricacies even under adversarial onslaughts.\n\n2. **Strategic Layer-by-Layer Pruning**: Our pruning methodology is sequential and tailored to individual layers. This granular strategy minimizes the forfeiture of critical semantic features, ensuring the pruned model's functional integrity.\n\n3. **Adaptive Error Rectification**: Recognizing that pruning one layer might inadvertently cascade errors, potentially amplifying pre-existing ones, our approach incorporates an adaptive error correction mechanism. This recursive refinement process ensures that the resulting pruned model harmoniously melds sparsity, robustness, and accuracy — and remarkably, sidesteps the need for exhaustive retraining.\n\n4. **Post-training Pruning**: A hallmark of our model is its capability to prune post-training. This approach obviates the necessity for computationally intensive retraining, thereby economizing computational resources.\n\nIn light of these innovations, our empirical evaluations persistently underscore our method's superiority over conventional benchmarks, predominantly in accuracy, sparsity, robustness, and pruning efficiency domains. By proactively addressing the intrinsic challenges of extant pruning strategies, our method paves the way for a more holistic, adept, and efficient paradigm in the realm of robust pruning for language models.\n\n## Question 2 (Computation Cost of Weight Averaging):\n\nThank you for your insightful inquiry regarding the computation cost introduced by our weight averaging technique. Let's delve deeper into this methodology's intricacies and address the associated computational implications:\n\n1. **Acknowledging Increased Computation**:\n   Initially, we acknowledge that the weight averaging technique incurs an additional computational overhead. This method involves training multiple models and then averaging them to achieve a robust and dense model. This approach is designed to ensure a precise analytical foundation. By doing so, we can eliminate factors that may distort our experimental results interpretation.\n\n2. **Purpose of Weight Averaging**:\n   Despite language models exhibiting commendable performance on specific datasets, they frequently rely on surface-level or spurious features rather than delving deep into sophisticated semantic nuances. This presents a dilemma when determining if the vulnerability of a pruned model stems from the inherent challenges of the dense model or the pruning techniques adopted. Our weight averaging strategy proactively addresses this issue by building a robust and dense foundation before any pruning.\n\n3. **Inspiration & Approach**:\n   Our technique, inspired by \\citet{croce2023seasoning} and \\citet{wortsman2022model}, harnesses the power of weight averaging to forge robust language models. The core principle is to train multiple models with varied hyperparameters or settings. This variety enables each model to understand and capture unique data nuances. By strategically averaging their weights, we amalgamate this dispersed knowledge, leading to a stronger model.\n\n4. **Greedy Weight Averaging**:\n   Conventional adversarial training tends to show unpredictable performance on language models. Our experiments have revealed unsatisfactory outcomes with these methods. In contrast, our greedy weight averaging approach consistently produces a stable, robust, and dense model. This strategy meticulously arranges models based on their performance under adversarial attacks and averages the weights that bolster the overall robustness, as elucidated in Algorithm~\\ref{algo:2}.\n\n5. **Efficient Pruning Strategy**:\n   Once we achieve a robust and dense language model, our main pruning strategy functions as a post-training method. This approach strategically eliminates the necessity for extensive training phases, thus efficiently conserving computational resources.\n\n6. **Eyes on the Prize**: \n   The endgame is to craft a model that's sparse, robust, and primed for high performance, making it ideal for real-world deployments. Given this context, the clear benefits such a model presents in operational environments undoubtedly justify the computational expenses borne during the training phase. This up-front computational commitment is a strategic investment, poised to yield substantial returns in real-world applications.\n\nTo sum it up, the computational increase due to weight averaging is a deliberate and tactical move, aimed at birthing a model that seamlessly merges robustness with practical efficiency.\n\nWe will address these concerns comprehensively in the following sections.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The research topic is important and meaningful.\n\n2. According to the experimental results, the proposed method is effective.\n", "reasons_to_reject": "1. The limitations in existing methods are not introduced clearly. It is not easy for people who are not working on pruning to understand the paper.\n\n2. The method needs to obtain several different models for weight averaging. It will increase the computation cost.\n", "questions_to_authors": "1. Could you conclude the limitations in existing methods? Why is the proposed model able to outperform existing models?\n\n2. One of the proposed techniques, weight averaging, requires obtaining several different versions of a model. Will it increase the computation cost?", "paper_topic_and_main_contributions": "This paper introduces a pruning strategy, which can have a balance between accuracy, sparsity, robustness, and pruning cost. More specifically, they firstly use weight averaging to obtain a robust dense model. Then, an adaptive pruning method is adopted to obtain sparse models. The experimental results demonstrate the effectiveness of their proposed method.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The limitations of exiting methods should be introduced in a more direct way."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Overall, The reviewers are very positive about the soundness and excitement of this work. Reviewers highlight the strong experimental results in particular regarding adversarial attacks, the good presentation, and the extensive experimental setting. As such, I recommend acceptance the main conference.
Decision: Accept-Main