A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models
Authors: Yuanxin Liu
Fandong Meng
Zheng Lin
Jiangnan Li
Peng Fu
Yanan Cao
Weiping Wang
Jie Zhou
Venue: NeurIPS 2022 Accept (2022)
Keywords: BERT compression
sparse subnetwork
dataset bias
OOD generalization

Abstract: Despite the remarkable success of pre-trained language models (PLMs), they still face two challenges: First, large-scale PLMs are inefficient in terms of memory footprint and computation. Second, on the downstream tasks, PLMs tend to rely on the dataset bias and struggle to generalize to out-of-distribution (OOD) data. In response to the efficiency problem, recent studies show that dense PLMs can be replaced with sparse subnetworks without hurting the performance. Such subnetworks can be found in three scenarios: 1) the fine-tuned PLMs, 2) the raw PLMs and then fine-tuned in isolation, and even inside 3) PLMs without any parameter fine-tuning. However, these results are only obtained in the in-distribution (ID) setting. In this paper, we extend the study on PLMs subnetworks to the OOD setting, investigating whether sparsity and robustness to dataset bias can be achieved simultaneously. To this end, we conduct extensive experiments with the pre-trained BERT model on three natural language understanding (NLU) tasks. Our results demonstrate that \textbf{sparse and robust subnetworks (SRNets) can consistently be found in BERT}, across the aforementioned three scenarios, using different training and compression methods. Furthermore, we explore the upper bound of SRNets using the OOD information and show that \textbf{there exist sparse and almost unbiased BERT subnetworks}. Finally, we present 1) an analytical study that provides insights on how to promote the efficiency of SRNets searching process and 2) a solution to improve subnetworks' performance at high sparsity. The code is available at \url{https://github.com/llyx97/sparse-and-robust-PLM}.

A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models
Yuanxin Fandong Meng', Zheng Lin 1 Jiangnan Li Peng Fu', Yanan Cao
Weiping Wang', Zhou
Institute of Information Engineering, Chinese Academy of Sciences
MOE Key Laboratory of Computational Linguistics, Peking University School of Computer Science, Peking University
SScooo of Cyber Security, University of Chinese Academy of Sciences
PPtttern Recognition Center, WeChat AI, Tencent Inc, China liuyuanxin@stu .ku. edu. en, {fandongmeng, twwttttooooooteeeeeeeeeeeeeeeeee com (linzheng, 1i jijiangnan, fupeng, caoyanan, vangweipingg ac.
Abstract
Despite the remarkable success of pre-trained language models (PLMs). they still face two challenges: First, large-scale PLMs are inefficient in terms of memory footprint and computation. Second, on the downstream tasks, PLMs tend rely on the dataset bias and struggle to generalize to out-of -istirtiinttion (OOD) data. In response the efficiency problem, recent studies show that dense PLMs can be replaced with sparse subnetworks without hurting the performance. Such subnetworks can be found in three scenarios: 1) the fine-tuned PLMs, 2) the raw PLMs and then fine-tuned in isolation, and even inside 3) PLMs without any parameter fine-tuning fine-tuning. However, these results are only obtained in the in- distribution (ID) setting. In this paper, we extend the study on PLMs subnetworks to the OOD setting, investigating whether sparsity and robustness dataset bias can be achieved simultaneously. To this end, we conduct extensive experiments with the pre-trained BERT model on three natural language understanding (NLU) tasks. Our results demonstrate that sparse and robust subnetworks (SRNets) consistently be found in BERT, across the aforementioned three scenarios, using different training and compression methods. Furthermore, bound of SRNets using the OOD information and show that there exist sparse and almost unbiased BERT subnetworks. Finally, present 1) analytical study that provides insights on how to promote the efficiency of SRNets searching process and 2) solution improve subnetworks' performance at high sparsity. The code is available at hhttps::///. // //g thub. com/11yx97/ /ttr///aaareeeeeee and robust--LM
explore the upper
1 Introduction
Pre-trained language models (PLMs) have enjoyed impressive success in natural language processing (NLP) tasks. However, they still face two major problems. On the one hand, the prohibitive model size of PLMs leads to poor efficiency in terms of memory footprint and computational cost [11 37 On the other hand, despite being pre-trained on large-scale corpus, PLMs still tend to rely on dataset
bias 16 30 50 35). i.e., the spurious features of input examples that strongly correlate with the
Work was done when Yuanxin Liu was graduate student of IIE, CAS. 1 'Corresoondigg author: Zheng Lin. Joint work with Pattern Recognition Center, WeChat A1, Tencent Inc, China.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

PLM
PLM
PLM
fine-tuning
A
A pruning
pruning
Fine-tuned
SubNet
SubNet
pruning
fine-tuning
SubNet
Fine- tuned SubNet
(a)
(b)
(c)
Three kinds of PLM subnetworks obtained from different pruning and fine-tuning paradigms. (a) Pruning a fine-tuned PLM. (b) Pruning the PLM and then fine-tuning the subnetwork. (c) Pruning the PLM without fine-tuning model parameters. The obtained subnetworks are used for testing.
Figure 1:
label, during downstream fine-tuning. These two problems pose great challenge to the real-worl world deployment of PLMs, and they have triggered two separate lines works.
In terms of the efficiency problem, some recent studies resort to sparse subnetworks as alternatives to
the dense PLMs. [23 BT 25 compress the fine-tuned PLMs in a post-hoc fashion. B 32 27 24
extend the Lottery Ticket Hypothesis (LTH) 8 to search PLMs subnetworks that can be fine-tuned
in isolation. Taking one step further, [51] propose to learn task-specific subnetwork structures via
mask training 19 28), without fine-tuning any pre-trained parameter. Fig. | Tillustrates these three
paradigms. Encouragingly, the empirical evidences suggest that PLMs can indeed be replaced with sparse subnetworks without compromising the in-distribution (ID) performance.
To address the dataset bias problem, numerous debiasing methods have been proposed. A prevailing 18
category of debiasing methods 4 41 21
12 42 adjust the importance of training examples, terms of training loss, according their bias degree, so as to reduce the impact of biased examples (examples that can be correctly classified based on the spurious features). As a result, the model is forced to rely less on the dataset bias during training and generalizes better to OOD situations.
35
Although progress has been made in both directions, most existing work tackle the two problems independently. To facilitate real-world application PLMs, the problems of robustness and efficiency should be addressed simultancously. Motivated by this, we extend the study on PLM subnetwork to the OOD scenario, investigating whether there exist PLM subnefworks that are both sparse and robust against dataset bias? To answer this question, we conduct large-scale experiments with the pre-trdi BERT model (5) on three natural language understanding (NLU) tasks that are widely-studied in the question of dataset bias. We consider variety of setups including the three pruning and fine-tuning fine-tuning paradigms, standard and debiasing training objectives, different model pruning methods, and different variants of PLMSs from the BERT family. Our results show that BERT does contain sparse and robust subnetworks (SRNets) within certain sparsity constraint (e.g., less than 70%). giving affirmative answer to the above question. Compared with standard fine-tuned BERT, SRNets exhibit comparable ID performance and remarkable OOD improvement. When comes to BERT model fine-tuned with debiasing method, SRNets can preserve the full model's 's ID and OOD performance with much fewer parameters, On this basis, we further explore the upper bound of SRNets by making use of the OOD information, which reveals that there exist sparse and almost unbiased subnetworks, even in a standard fine-tuned BERT that is biased.
Regardless of the intriguing properties of SRNets, we find that the subnetwork searching process still have room for improvement, based on some observations from the above experiments. First, we study the timing to start searching SRNets during full BERT fine-tuning, and find that the entire training and searching cost can be reduced from this perspective. Second, we refine the mask training method with gradual sparsity increase, which is quite effective in identifying SRNets at high sparsity.

---

Our main contributions are summarized as follows:
We extend the study on PLMs subnetworks to the OOD scenario. To our knowledge, this paper presents the first systematic study on sparsity and dataset bias robustness for PLMs.
We conduct extensive experiments to demonstrate the existence sparse and robust BERT subnetworks, across different pruning and fine-tuning setups. By using the OOD information, we further reveal that there exist sparse and almost unbiased BERT subenetworks.
We present analytical studies and solutions that can help further refine the SRNets searching process in terms of efficiency and the performance of subnetworks at high sparsity.
2 Related Work
2.1
BERT Compression
Studies on BERT compression can be divided into two classes. The first one focuses the design of
model compression techniques, which include pruning [0] knowledge distillation [34 38 20
261. parameter sharing [22], quantization [47 49 and combining multiple techniques [39 29 25
The second one, which is based on the lottery ticket hypothesis [8), investigates the compressibility of BERT different phases of the pre- vre-training and fine-tuning paradigm. It has been shown that
BERT can be pruned to sparse subnetwork after [TO] and before fine-tuning [3] 32 27 [3] without hurting the accuracy. Moreover, (51) show that directly learning subnetwork structures the pre-trained weights can match fine-tuning the full BERT. this paper, we follow the second branch of works, and extend the evaluation of BERT subnetworks the OOD scenario.
2.2 Dataset Bias in NLP Tasks
To facilitate the development NL.P systems that truly learn the intended task solution, instead of relying on dataset bias, many efforts have been made recently. On the one hand, challenging OOD test sets are constructed [16 30 50 BS I by eliminating the spurious correlations in the training sets, in order to establish more strict evaluation. On the other hand, numerous debiasing methods 21
4. [1 21 18 35 12 42 are proposed to discourage the model from learning dataset bias during training. However, few attention has been paid to the influence of pruning the OOD generalization ability of PLMs. This work presents a systematic study on this question.
2.3 Model Compression and Robustness
Some pioneer attempts have also been made to obtain models that are both compact and robust to
adversarial attacks [14 46 36 9 45) and spurious correlations [48 7 Specially, [45] 7 study the compression and robustness question
PLM. Different from 45 which is based adversarial robustness, we focus on the spurious correlations, which more common than the worst-case adversarial attack. Compared with 77). which focus post-hoc pruning of the standard fine- tuned BERT, we thoroughly investigate different fine-tuning methods (standard and debiasing) and subnetworks obtained from the three pruning and fine-tuning paradigms. A more detailed discussion of the relation and difference between our work and previous studies on model compression and robustness is provided in Appendix D.
3 Preliminaries
3.1 BERT Architecture and Subnetworks
BERT is composed of an embedding layer, a stack of Transformer layers 43 and task-specific classifier. Fach Transformer layer has a multi-head self-attention (MHAtt) module and feed-forward network (FFN). MHAtt has four kinds of weight matrices, i.e., the query, key and value matrices WQ..V E 1 X dassd and the output matrix WAO AO E dndea X Xomaaa FFN consits of two linear layers Win E Rdmsta Wont E I den dasad where dFFx is the hidden dimension of FFN.
To obtain the subnetwork of model (() parameterized by 0, we apply binary pruning mask m {0, 1)10 to its weight matrices, which produces f(m c 0), where @ is the Hadamard product.
3

---

For BERT, we focus on the L Transformer layers and the classifier. The parameters to be pruned are
pr (Wa)] U ( W W.. W Ao: W W 1
1=1 where War cts is the classifier weights.
3.2 Pruning Methods
3.2.1 Magnitude-based Pruning
Magnitude-based pruning [7 zeros-out parameters with low absolute values. It usually realized in iterative manner, namely, iterative magnitude pruning (IMP). IMP alternates between pruning and training and gradually increases the sparsity of subnetworks. Specifically, a typical IMP algorithm consists of four steps: (i) Training the full model to convergence. (ii) Pruning fraction of parameters with the smallest magnitude. (iii) Re-training the pruned subnetwork. (iv) Repeat (ii)-(iii) until reaching the target sparsity. To obtain subnetworks from the pre- pre-trained BERT, i.e., (b) and (c) in Fig. T the subnetwork parameters are rewound to the pre-trained values after (ii),, and (i) can be abandoned. More details about IMP implementations can be found in Appendix A.I.1.
3.2.2 Mask Training
Mask training treats the pruning mask m as trainable parameters. Following 28 51 33
27,. we
achieve this through binarization in forward pass and gradient estimation in backward pass.
Each weight matrix W E IRd1 Xd2, which is frozen during mask training, associated with a bianry mask {0, 13 dd2,d,, and real-valued mask r xd2, In the forward pass, W is replaced with m C W, where m is derived from n through binarization:
{1
if m. 2 otherwise
m
where e is the threshold. the backward pass, since the binarization operation is not differentiable, use the traighhtthrough estimator [2] to compute the gradients for m using the gradients of m, OC
ac where is the learning rate. i.e, 3m where L is the loss. Then, m is updated as m +- m 1) 7m
initialize the real-valued masks according to the magnitude of the original weights. The complete mask training algorithm is summarized in Appendix A.1.2.
Following 33 27
3.3 Debiasing Methods
As described in the Introduction, the debiasing methods measure the bias degree of training examples. This is achieved by training bias model. The inputs to the bias model are hand-crafted spurious features based on our prior knowledge of the dataset bias (Section 4.1.3 describes the details). In this way, the bias model mainly relies on the spurious features to make predictions, which can then serve as measurement of the bias degree. Specifically, given the bias model prediction ps (......, .P()) over the K classes, the bias degree 3 = P i.e., the the probability of the ground-truth class
Then, 3 can be used to adjust the training loss in several ways, including product-of-experts (PoE) 18 21], example reweighting (35 12] and confidence regularization [41]. Here we describe the standard cross-entropy and PoE, and the other two methods are introduced in Appendix A.2.
Standard Cross-Entronn Eussopy computes the cross-entropy between the predicted distribution pm and the ground-truth one-hot distribution y as Lad -y loog Pm-
Product-of-Exyee Effexxsrts combines the predictions of main model and bias model, i.e., Ps and Pmss and then computes the training loss as Lpoe -y log softmax (log pm + log Ps).
3.4 Notations
Here we define some notations, which will be used in the following sections.
At((((0)): Training f()) with loss L for steps, where can be omitted for simplicity.
PB))(0)): Pruning f(0) using pruning method p and training loss L.
M(f(m))) Extracting the pruning mask of fm00) i..., M(f.M) m.

---

L E {Lud. Lpoe. Freweihtt L and {imp, imp-rw, mask), J. where "imp" and "imp- denote the standard IMP and IMP with weight rewinding, as described in Section 3.2.1 "mask" stands for mask training.
(O)) Evaluating (() on the test data with distribution d & (ID,
4 Sparse and Robust BERT Subnetworks
4.1
Experimental Setups
4.1.1 Datasets and Evaluation
Natural Language Inference We use MNLI 44 as the ID dataset for NLL MNLI is comprised of premise-hyee -yyoois-yypo pairs, whose relationship may entailment, contradiction, or neutral. In MNLI the word overlap between premise and hypothesis is strongly correlated with the entailment class. To solve this problem, the OOD HANS dataset 30 is built so that such correlation does not hold.
Paraphrase Identification The ID dataset for paraphrase identification is QQP which contains question pairs that are labelled as either duplicate or non-duplicate. In QQP, high lexical overlap also strongly associated with the duplicate class. The OOD datasets PAWS-qqP and PAWS-wiki [50) are built from sentences in Quora and Wikipedia respectively. PAWS sentence pairs with high word overlap have balanced distribution over duplicate and non-duplicate.
adopted as the dataset of fact verification, where the task to assess whether a given evidence supports refutes the claim, or whether there is not-enough-injs to reach a conclusion. The OOD dataset Pever-Symment (v1 and v2) [35] is proposed to evaluate the influence of the claim-only bias (the label can be predicted correctly without the evidence).
Fact Verification FEVER 40
For NLI and fact verification, we use Accuracy the evaluation metric. For paraphrase identification, we evaluate using the Fl score. More details of datasets and evaluation are shown in Appendix B.1.
4.1.2 PLM Backbone
We mainly experiment with the SerTT-bae-uncaan model [5) has roughly 110M parameters in total, and 84M parameters in the Transformer layers. As described in Section [.1 we derive the subnetworks from the Transformer layers and report sparsity levels relative to the 84M parameters. To generalize our conclusions to other PL.Ss, we also consider two variants of the BERT family, namely RoBERTa-base and BERT-large, the results of which can be found in Appendix C.5.
4.1.3 Training Details
Following
we use simple linear classifier as the bias model. For HANS and PAWS, the spurious features are based on the the word overlapping information between the two input text sequences. For FreverSyymmerri,, the spurious features are max-pooled word embeddings of the claim sentence. More details about the bias model and the spurious features are presented in Appendix B.
4
Mask training and IMP basically use the same hyper-parameters (adopting from 42)) as full BERT. exception is longer training, because we find that good subnetworks high sparsity levels require more training to be found. Unless otherwise specified, we select the best checkpoints based on the ID dev performance, without using OOD information. All the reported results are averaged over 4 runs. We defer training details about each dataset, and each training and pruning setup. to Appendix B.3.
4.2
Subnetworks from Fine-tuned BERT
4.2.1
Problem Formulation and Experimental Setups
Given the fine-tuned full BERT f() Ac, (f(O)))) where Opt and f are the pre-trained and fine-tuned parameters respectively, the goal is to find subnetwork f(m @ 0).)) ) = Ph, (0))))) that
"See the licence information at http:///////eeee//ao/oooaaaoooea
5

---

0.34
640
0.50
E26
0/6
HANL 000 A3E
6/60
PAINS Mel F1
0.74
0.72
0.3
02
844
0.8
0.6
Sparaity
0.22
Symmma ACC
14055
0.33
0.75
2:10
0.6
015
0.2
DX
Sparstyy
Figure 2: Results of subnetworks pruned from the CE fine-tuned BERT. "std" means standard, and the shadowed areas denote standard deviations, which also apply to the other figures of this paper.
1.22
PAWSS MAM 53
0/40
bet mest 20n0
2.82
6.16
sart
6.78
5.14
bet boc
2:18
030
033
6.0
C2
Co
6o
Sparsiy
Sparsity
Sparshy
0.71
0.55
8
0.54
0.55
Samenz MCC
3.50
5:15
3.3
50
Sparsty
1.6
0.48
0.2
Sparsty
66
04
0.30
Sparsity
C4
Sparshy
Figure 3: Results of subnetworks pruned from the PoE fine-tuned BERT. Results the "mask train
(poe)" subnetworks from Fig. 2 (the orange line) are also reported for reference.
satisfies a target sparsity level and maximize the ID and OOD performance.
(EID (f(m (m)0)))) + EOOD ()mm))))) m.0,
o),)))
m
max
s.t.
(1 s)
(2)
0
where Illlo the Lo norm and 10pr| is the total number of parameters to be pruned. In practice, the above optimization problem is achieved via PE, (), which minimizes the loss L2 on the ID training set. When the pruning method IMP, the subnetwork parameters will be further fine-tuned and fy * ftt For mask training, only the subnetwork structure is updated and 0x = 0
We consider two kinds of fine-tuned full BERT, which utilize the standard CE loss and PoE loss respectively (i.e., Li E (Cus. Cyee ). IMP and mask training are used as the pruning methods (i.e.. p E {imp, mask)). For the standard fine-tuned BERT, both Lut and Cpee are examined in the pruning process. For the PoE fine-tuned BERT, we only use Lyee during pruning. Note that in this work, we mainly experiment with Land and Lpe.. freweight and Lconfreg are also examined for subnetworks from fine-tuned BERT, the results of which can be found Appendix C.I.
4.2.2 Results
Subnetworks from Standard Fine-tuned BERT The results are shown in Fig. (In this paper, we present most results in figures for clear comparisons. Actual values of the results can found in the code link.). We discuss them from three perspectives. For the full BERT, we can see that standard

---

8.0
0.50
0.42
447
PAinS MM
C.0
0.3
0.22
2.25
6.18
sur VRT
0.0
0.2
0330
6.66
59% bed
0.44
0.3
0.2
0.6
0.8
18
02
0.6
1.
02
07
0.3
0x
Spardhy
Sparsity
Sparsky
0.3
6.00
2 1082
0.00
Symmna ACC
0.000
Symmil
0312
0.50
6:75
0.30
2,35
6/6
0.00
5.0
Spariity
1.6
0.5
0.2
8-4 Sparsity
02
Sparsity
Sparsity
Figure 4: Results of BERT subnetworks fine-tuned in isolation. "ft" is short for fine-tuning
CE fine-tuning, which achieves good results on the ID dev sets, performs significantly worse on the OOD test sets. This demonstrates that the ID performance of BERT depends, to large extent, on memorizing the dataset bias.
In terms of the subnetworks, we can derive the following observations: (1) Using any of the four pruning methods, we can compress a large proportion of the BERT parameters (up to 70% sparsity) and still preserve 95% of the full model's ID performance. (2) With standard pruning, i.e., "mask train (std)" or "imp (std)", we can observe small but perceivable improvement over the full BERT on the HANS and PAWS datasets. This suggests that pruning may remove some parameters related to the bias features. (3) The OOD performance of "mask train (poe)" and "imp (poe)" subnetworks is even better, and the ID performance degrades slightly but is still above 95% of the full BERT. This shows that introducing the debiasing objective in the pruning process is beneficial. Specially, as mask training does not change the model parameters, the results of "mask train (poe)" implicates that the biased "full bert (std)" contains sparse and robust subnetworks (SRNets) that already encode a less biased solution to the task. (4) SRNets can be identified across a wide range of sparsity levels (from 20% ~ 70%). However at higher sparsity of 90%. the performance of the subnetworks is not desirable. (5) We also find that there is an abnormal increase the PAWS Fl score at 70% 90% sparsity for some pruning methods, when the corresponding ID performance drops sharply. This is because the class distribution of PAWS is imbalanced (see Appendix B.1), and thus even a naive random-gues guessing model can outperform the biased full model on PAWS. Therefore, the OOD improvement should only acceptable when there no large ID performance decline.
Comparing IMP and mask training, the latter performs better in general, except for "mask train (poe)" 90% sparsity QQP and FEVER. This suggests that directly optimizing the subnetwork structure is a better choice than using the magnitude heuristic as the pruning metric.
Subnetworks from PoE Fine-tuned BERT Fig. presents the results. We can find that: (1) For the full BERT, the OOD performance is obviously promoted with the PoE: debiasing method, while the ID performance is sacrificed slightly. (2) Unlike the subnetworks from the standard fine-tuned BERT, the subnetworks of PoE fine-tuned BERT (the green and blue lines) cannot outperform the full model. However, these subnetworks maintain comparable performance at up to 70% sparsity, on both the ID and OOD settings, making them desirable alternatives to the full model in essource-constriig scenarios. Moreover, this phenomenon suggests that there a great redundancy BERT parameters, even when OOD generalization is taken into account. (3) With PoE- based pruning, subnetworks from the standard fine-tuned BERT (the orange line) is comparable with subnetworks from the PoE fine-tuned BERT (the blue line). This means we do not have to fine-tune a debiased BERT before searching for the SRNets. (4) IMP, again, slightly underperforms mask training at moderate sparsity levels, while it is better at 90% sparsity on the fact verification task.
7

---

4.3 BERT Subnetworks Fine-tuned in Isolation
4.3.1
Problem Formulation and Experimental Setups
subnetwork f (m @ ptt) is obtained before downstream fine- Given the pre-trained BERT f()pt),
tuning. The goal is to maximize the performance of the fine-tuned subnetwork Ac, (f(m @ ppt)):
m lo 00pll
max (EID (Ac, (f 0pt))) EOOD (Ac, (f ppt)))) m Following the LTH 8 we solve this problem using the train-prune- rrin--ruune-eewidd pipeline. For IMP, the procedure is described in Section 3.2.1 and m MMPPi (f(pp)))) For mask training. the subnet- work structure is learned from f(0 (same the previous section) and = MMPPE (f(O
s.t.
s1
(3)
We employ CE and PoE loss for model fine-tuning (i.e., Li E {Lua pre }). Since we have shown that using the debiasing loss pruning conducive, the CE loss not considered (i.e., L2 Lpae).
4.3.2 Results
The results of subnetworks fine-tuned in isolation are presented in Fig. It can be found that: (1) For standard CE fine-tuning, the "mask train (poe)" subnetworks are superior "full bert (std)" on the OOD test data, i.c.. the subnetworks are less susceptible to the dataset bias during training. (2) In terms of the PoE-based fine-tuning, the "imp (poe)" and "mask train (poe)" subnetworks are generally comparable to "full bert (poe)". (3) For most of the subnetworks, "poe ft" clearly outperforms "std ft" in the OOD setting, which suggests that is important to use the debiasing method fine-tuning, even if the BERT subnetwork structure has already encoded some unbiased information.
Moreover, based (1) and (2), we can extend the LTH on BERT 3 32 24 27
The pre-trained BERT contains SRNets that can be fine-tuned in isolation, using either standard or debiasing method, and match even outperform the full model in both the ID and 0OD evaluations.
4.4
BERT Subnetworks Without Fine-tuning
4.4.1
Problem Formulation and Experimental Setups
This setup aims finding subnetwork f'm @ ppt) inside the pre-trained BERT, which can be directly employed to task. The problem is formulated as:
||m (1 - s)
max (EnD (f @ 0pt)) EOOD (f(m C 0p)))), s.t.
(4)
0prl
Following (51) we fix the pre-trained parameters Ont and optimize the mask variables m. This process can be represented as Pmnask (J((p))) where L {Load, Lroe).
4.4.2 Results
As we can see in Fig. 5 (1) With CE-based mask training, the identified subnetworks (under 30% sparsity) in pre-trained BERT are competitive with the CE fine-tuned full BERT. (2) Similarly, using PoE-based mask training, the subnetworks under 50% sparsity are comparable to the PoE fine-tuned full BERT, which demonstrates that SRNets for particular downstream task already exist in the pre-trained BERT. (3) "mask train (poe)" subnetworks in pre-trained BERT can even match the subnetworks found in the fine-tuned BERT (the orange lines) in some cases (e.g., on PAWS and FEVER under 50% sparsity). Nonetheless, the latter exhibits a better overall performance.
4.5 Sparse and Unbiased BERT Subnetworks
4.5.1 Problem Formulation and Experimental Setups
To explore the upper bound of BERT subnetworks in terms of OOD generalization, we include the OOD training data in mask training, and use OOD test sets for evaluation. Like the previous sections, we investigate three pruning and fine-tuning paradigms, as formulated by Eq. 2 3 and respectively. We only consider the standard CE for subnetwork and full BERT fine-tuning, which is more vulnerable the dataset bias. Appendix B.3.3 summarizes the detailed experimental setups.

---

0.94
076
NNU 00 606
0.02
5.44
PAWS winl FL.
6.4
220
0.70
046
022
15% 2urt
0.2
8-4
02
Sparaty
0.00
6.05
6.72
0.00
14055
Symmna
6-10
8.40
2443
04
0.2
02
Spandty
Figure 5: Results of BERT subnetworks without fine-tuning. Results of the "mask train (poe)" subnetworks from Fig. (the orange line) are also reported for reference.
0.5
043
MRL dr ACC
0.5
1315
HANO
078
085
355
Sparsty
Sparsty
Traring Steps
Traring Steps
Figure NIJ results of BERT subnetworks found using the OOD information. Results of the other two tasks can be found in Appendix C.2.
Figure 7: NLI mask training curves (70% sparse), starting from BERT fine-tuned for varied steps. Appendix C.3 shows results of the other two tasks.
4.5.2 Results
From Fig. [6] we can observe that: (1) The subnetworks from fine-tuned BERT ("bert-ft subnet") at 20% 70% sparsity achieve nearly 100% accuracy on HANS, and their ID performance is also close to the full BERT. (2) The subnetworks in the pre-trained BERT ("bert-pt subnet") also have very high OOD accuracy, while they perform worse than "bert-ft subnet" in the ID setting. (3) "bert-pt subnet + ft" subnetworks, which are fine-tuned in isolation with CE loss, exhibits the best ID performance, and the poorest OOD performance. However, compared to the full BERT, these subnetworks still rely much less the dataset bias, reaching nearly 90% HANS accuracy at 50% sparsity. Jointly, these results show that there consistently exist BERT subnetworks that are almost unbiased towards the MNLI training set bias, under the three kinds of pruning and fine-tuning paradigms.
Refining the SRNets Searching Process
In this section, study how further improve the SRNets searching process based mask training, which generally performs better than IMP, as shown in Section 4.2 and Section
5.1 The Timing to Start Searching SRNets
Compared with searching subnetworks from the fine-tuned BERT, directly searching from the pre- trained BERT is more efficient in that it dispenses with fine-tuning the full model. However, the former has a better overall performance, as we have shown in Section 4.4 This induces question: At which point of the BERT fine-tuning process, can we find subnetworks comparable to those found after the end of fine-tuning using mask training? To answer this question, we perform mask training on the model checkpoints f(0,) = =A (f p())) from different steps i. of BERT fine-tuning.

---

MNLI ALC
DOP
PAWSS-up
PMWS will
ROWNY
epch-
Parpphrese Identification
gndial
HANS ACC
FEVER ALC
Symm1 Art
Symm2 Acc
G6b NEJ
(c) Fort Verification
Figure 8: Comparison between fixed sparsity and gradual sparsity increase for mask training with the standard fine-tuned full BERT. The subnetworks 90% sparsity.
Fig. shows the mask training curves, which start from different (.e)) We can see that "ft step=0" converges slower and a worse final accuracy, compared with "ft end", especially on the HANS dataset. However, with 20,000 steps of full BERT fine-tuning, which is roughly 55% of the "ft to end", the mask training performance is very competitive. This suggests that the total training cost of SRNet searching can be reduced, by large amount, in the full model training stage.
To actually reduce the training cost, we need to predict the exact timing start mask training. This is intractable without information of all the training curves in Fig. 7 A feasible solution is adopting the idea of early-stopping (see Appendix E.1 for detailed discussions). However, accurately predicting the optimal timing (with the least amount fine-tuning and comparable subnetwork performance to fully fine-tuningl indeed difficult and we invite follow-up studies to investigate this question.
5.2 SRNets at High Sparsity
As the results of Section 4 demonstrate, there is a sharp decline of the subnetworks' performance from 70% ~ 90% sparsity. We conjecture that this is because directly initializing mask training to 90% reduces the model's capacity too drastically, and thus causes some difficulties in optimization. Therefore, we gradually increase the sparsity from 70% ~ 90% during mask training, using the cubic sparsity schedule [52] (see Appendix C.4 for ablation studies). Fig. compares the fixed sparsity used the previous sections and the gradual sparsity increase, across varied mask training epochs. We find that while simply extending the training process is conducive, gradual sparsity increase achieves better results. In particular, "gradual" outperforms "fixed" with lower training cost on all the three tasks, except for the PAWS dataset, A similar phenomenon is explained in Section 4.2.2
Conclusions and Limitations
In this paper, we investigate whether sparsity and robustness dataset bias be achieved simulta- neously for PLM subnetworks. Through extensive experiments, we demonstrate that BERT indeed contains sparse and robust subnetworks (SRNets) across variety NLU tasks and training and pruning setups. We further use the OOD information to reveal that there exist sparse and almost unbiased BERT subnetworks. Finally, we present analysis and solutions to refine the SRNet searching process in terms of subnetwork performance and searching efficiency.
The limitations of this work twofold. First, we focus on BERT-like PLMs and NLU tasks, while dataset biases are also common in other scenarios. For example, gender and racial biases exist in dialogue generation systems (6] and PLMs (15.. In the future work, would like extend our exploration to other types of PLMs and NLP tasks (see Appendix E.2 for discussion). Second, as we discussed in Section 5.I our analysis on "the timing to start searching SRNets" mainly serves as a proof-of-conee offccccept, and actually reducing the training cost requires predicting the exact timing.
Acknowledgments and Disclosure of Funding
This work was supported by National Natural Science Foundation of China (61976207 and 61906187).
10

---

References
[1] A. Agrawal, D. Batra, D. Parikh, and A. Kembhavi. Don't just assume; look and answer: Overcoming priors for visual question answering. In CVPR, pages 4971 Computer Vision Foundation IEEE Computer Society, 2018.
[2] Y. Bengio, N. Léonard, and A. C. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. CoRR, abs/ 1111110883333333 2013.
[3] Chen, Frankle, Chang, S. Liu, Y. Zhang, Z. Wang, and M. Carbin. The lottery ticket hypothesis for pre-trained BERT networks. In NeurIPS, pages 15834-15846, 2020.
[4] C. Clark, M. Yatskar, and L. Zettlemoyer. Don't take the easy way out: Ensemble based methods for avoiding known dataset biases. In EMNLP//JCNLP, pages 4069 4082. Association for Computational Linguistics, 2019.
[5] J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pages 4171-4186, 2019.
[6] E. Dinan, A. Fan, A. Williams, J. Urbanek. D. Kiela, and J. Weston. Queens are powerful too: Mitigating gender bias in dialogue generation. In EMNLP (?), pages 8173-815 1888.. Association for Computational Linguistics, 2020.
[7] M. Du, S. Mukherjee, Y. Cheng, M. Shokouhi, X. Hu, and A. H. Awadallah. What do compressed large language models forget? robustness challenges in model compression. CoRR. abs/21 10,211,,, 2021.
[8] Frankle and M. Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In ICLR. OpenReview.net, 2019.
[9] Y. Fu, Q. Yu, Y. Zhang, S. Wu, X. Ouyang. D. D. Cox, and Lin. Drawing robust scratch tickets: Subnetworks with inborn robustness are found within randomly initialized networks. In NeurlPS, pages 13059-13072, 2021.
[10] T. Gale, E. Elsen, and S. Hooker. The state of sparsity in deep neural networks. CoRR, abs/1900 10s/19922,9974, 2019.
[11] Ganesh, Chen, X. Lou, M. A. Khan, Y. Yang, H. Sajjad, P. Nakov, D. Chen, and M. Winslett. Compressing large-scale transformer- bassed models: A case study on BERT. Transactions of the Association for Computational Linguistics, 9:1061-1080, 2021.
[12] A. Ghaddar, P. Langlais, M. Rezagholizadeh, and A. Rashid. End-to-end self-debiasing frame- work for robust NLU training. In ACL/IJCNLP (Findings), volume ACL/IJCNPP 2021 of Findings of ACL, pages 1923-1929. Association for Computational Linguistics, 2021.
[13] M. A. Gordon, K. Duh, and N. Andrews. Compressing BERT: studying the effects weight pruning on transfer learning. In RepLANLP@ACL, pages 143-155, 2020.
[14] S. Gui, H. Wang, H. Yang, C. Yu, Z. Wang, and J. Liu. Model compression with adversarial robustness: A unified optimization framework. In NeurIPS, pages 1283-1294, 2019.
[15] Y. Guo, Y. Yang, and A. Abbasi. Auto-debias: Debiasing masked language models with auto- mated biased prompts. In ACL, pages 1012-1023. Association for Computational Linguistics, 2022.
[16] S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. R. Bowman, and N. Smith. Annotation artifacts in natural language inference data. In NAACL-HLT, pages 107-112. Association for Computational Linguistics, 2018.
[17] S. Han, J. Pool, J. Tran, and W. Dally. Learning both weights and connections for efficient neural network. In Advances Neural Information Processing Systems 28, pages 1135-1143 Curran Associates, Inc., 2015.

---

[18] H. He, S. Zha, and H. Wang. Unlearn dataset bias in natural language inference by fitting the residual. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resourg NLP (DeepLo 2019), pages 132-142. Association for Computational Linguistics, 2019.
[19] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. Binarized neural networks. In NIPS, pages
2016.
[20] X. Jiao, Y. Yin, Shang, X. Jiang. X. Chen, L. Li, Wang. and Q. Liu. Tinybert: Distilling BERT for natural language understanding. In EMNLP (Findings), pages 4163-4174, 2020.
[21] R. Karimi Mahabadi, Y. Belinkov, and J. Henderson. End-to-end bias mitigation by modelling biases in corpora. In ACL, pages 8706-8716. Association for Computational Linguistics, 2020.
[22] Z. Lan, M. Chen, S. Goodman, K. Gimpel, Sharma, and R. Soricut. ALBERT: lite BERT for self-supervised learning of language representations. In ICLR. OpenReview.net, 2020.
[23] Z. Li, E. Wallace, S. Shen, K. Lin, K. Keutzer, D. Klein, and J. E. Gonzalez. Train large, then compress: Rethinking model size for efficient training and inference of transformers. CoRR, abs/2002. 11794, 2020.
[24] C. Liang, S. Zuo, M. Chen, H. Jiang, X. Liu, P. He, T. Zhao, and W. Chen. Super tickets in pre-trained language models: From model compression improving generalization. In ACL/IJCNLP, pages
Association for Computational Linguistics, 2021.
[25] Y. Liu, Z. Lin, and F. Yuan. ROSITA: refined BERT compression with integrated techniques. In AAAI, pages 8715 5-722. AAAI Press, 2021.
[26] Y. Liu, Meng. Lin, W. Wang. and J. Zhou. Marginal utility diminishes: Exploring the minimum knowledge for BERT knowledge distillation. In ACL/JCNLP, pages 2928-12222 2221,, 2021.
[27] Y. Liu, F. Meng. Z. Lin, Fu, Y. Cao, W. Wang. and J. Zhou. Learning to win lottery tickets in BERT transfer via task-agnostic mask training. CoRR, abs/2204.1 11212, 2022.
(28) A. Mallya, D. Davis, and S. Lazebnik. Piggyback: Adapting single network to multiple tasks by learning to mask weights. In ECCV, volume 1 1200 of Lecture Notes Computer Science, pages 72 88. Springer, 2018.
[29] Y. Mao, Y. Wang, C. Wu, C. Zhang, Y. Wang, Q. Zhang, Y. Yang, Y. Tong, and J. Bai. Ladabert: Lightweight adaptation of BERT through hybrid model compression. In COLING, pages 3225 3234,, 2020.
[30] T. McCoy, E. Pavlick, and T. Linzen. Right for the wrong reasons: Diagnosing syntactic heuris- ties in natural language inference. In ACL, pages 3428-3448. Association for Computational Linguistics, 2019.
[31] P. Michel, O. Levy, and G. Neubig. Are sixteen heads really better than one? In NewrlPS, pages 14014-14024, 2019.
[32] Prasanna, A. Rogers, and Rumshisky. When BERT plays the lottery, all tickets are winning. In EMNLP, pages 1208-3229 2020.
[33] E. Radiya-Dixit and X. Wang. How fine can fine-tuning be? learning efficient language models. In AISTATS, volume 108 of Proceedings of Machine Learning Research, pages 2435-2443. PMLR, 2020.
[34] V. Sanh, L. Debut, J. Chaumond, and T. Wolf. Distilbert, distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01 108, 2019.
[35] Schuster, Shah, Y. J. S. Yeo, D. Filizzola, E. Santus, and R. Barzilay. Towards debiasing fact verification models. In EMNLP/IJCNLP, pages 3417-3423. Association for Computational Linguistics, 2019.
[36] V. Sehwag, S. Wang, P. Mittal, and S. Jana. HYDRA: pruning adversarially robust neural networks. In NewrlPS, 2020.
12

---

[37] E. Strubell, A. Ganesh, and A. McCallum. Energy and policy considerations for deep learning in NLP. In ACL, pages 3645-3650. Association for Computational Linguisties, 2019.
[38] Sun, Y. Cheng, Z. Gan, and Liu. Patient knowledge distillation for BERT model compres sion. In EMNLP/IJCNLP, pages 4322-4332 2019.
[39] T. Tambe, C. Hooper, L. Pentecost, E. Yang, M. Donato, V. Sanh, A. M. Rush, D. Brooks, and
G. Wei. Edgebert: Optimizing on-chip inference for multi-task NLP. CoRR, abs/2011.14200 2020.
[40] J. Thorne, A. Vlachos, O. Cocaraseu, C. Christoodoulopoullo, and A. Mittal. The fact extraction and verification (FEVER) shared task. CoRR, abs/ abs//181..000 1.10971, 2018.
[41] P. A. Utama, N. S. Moosavi, and I. Gurevych. Mind the trade-off: Debiasing NLU models without degrading the in-distribution performance. In ACL, pages 8717-8729. Association for Computational Linguistics, 2020.
[42] P. A. Utama, N. S. Moosavi, and I. Gurevych. Towards debiasing NLU models from unknown biases. EMNLP, pages 7597-7610. Association for Computational Linguisties, 2020.
[43] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In NIPS, pages 5998 6008, 2017.
[44] A. Williams, N. Nangia, and S. R. Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In NAACL-IILT, pages 1112-1122. Association for Computational Linguisties, 2018
[45] C. Xu, W. Zhou, T. Ge, K. Xu, J. McAuley, and F. Wei. Beyond preserved accuracy: Evaluating loyalty and robustness of BERT compression. In EMNLP (/), pages 10653-10659, Association for Computational Linguistics, 2021.
[46] Ye, X. Lin, K. Xu, S. Liu, Cheng, Lambrechts, Zhang, A. Zhou, K. Ma, and Y. Wang. Adversarial robustness vs. model compression, both? In ICCV, pages 111-120. IEEE, 2019.
[47] O. Zafrir, G. Boudoukh, P. Izsak, and M. Wasserblat. Q8BERT: quantized 8bit BERT. In EMC2 @ NeurlPS, pages 36-39. IEEE, 2019.
[48] D. Zhang, K. Ahuja, Y. Xu, Y. Wang, and A. C. Courville. Can subnetwork structure be the key out-of-distribution generalization? In ICML, volume 139 of Proceedings of Machine Learning Research, pages 12356-12367. PMLR, 2021.
[49] W. Zhang, L. Hou, Y. Yin, L. Shang, X. Chen, X. Jiang, and Q. Liu. Ternarybert: Distillation- aware ultra-low bit BERT. In EMNLP, pages 509-521 Association for Computational Linguis- tics, 2020.
[50] Y. Zhang, J. Baldridge, and I.. He. PAWS: paraphrase adversaries from word scrambling. In NAACL-HLT, pages 1208-1308. Association for Computational Linguistics, 2019.
[51] M. Zhao, T. Lin, F. Mi, M. Jaggi, and H. Schütze. Masking as an efficient alternative to finetuning for pretrained language models. In EMNLP, pages 2226-2241, 2020.
[52] M. Zhu and S. Gupta. To prune, not to prune: Exploring the efficacy of pruning for model compression. In ICLR (Workshop). OpenReview.net, 2018.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6
(c) Did you discuss any potential negative societal impacts of your work? [No] Currently,
we think there no apparent negative societal impacts related to our work.
13

---

(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material as a URL)? [No] We will release the codes and reproduction instructions upon publication.
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.1 and Appendix B.
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] See all the figures of our experiments.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix B.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 4.1
(b) Did you mention the license of the assets? [Yes) Licenses of some dataset we used are mentioned in Section 4.1 However, for the other datasets, we were unable find the licenses.
(c) Did you include any new assets either in the supplemental material or URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you'ne using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "yNDSOZx3ZZ-", "reviewer": "Reviewer_Bf72", "summary": "In this paper, the authors studied the network pruning through (1) iterative magnitude-based pruning and (2) mask training, while combing the debasing loss during the pruning process. For different settings, the paper provides thorough experiment with analysis on the results. Experiments reveal that there are subnetworks with a certain sparsity level and robustness on ODD datasets. ", "questions": "Please check the weakness section.", "limitations": "I don't have concern on negative societal impact for this paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\n1. The evaluation is thorough and clear, the authors present figures where readers can see how the sparsity level is correlated with the model performance\n2. The model scores reported in the figures are at a reasonably high level\n\nWeakness:\n\n1. Overall the results seems to be what I have expected. Take section 3.2.2 as example, (1) Existing papers have shown that the pruning can be done at around 70% sparsity level with modest performance drop. (2,3) We can consider both the imp and mask training as learnable sparsity inducing methods. We are expecting the OOD performance to be improved over standard loss by training the parameters in such a way during pruning. (4) This statement is describing the observation of experiments in this paper. How can we prove this 20% - 70% range can be generalized? \n\n2. The conclusion that the sparse and robust BERT subnetworks exist seems to be made from experiment results on three dataset settings. In figure 2, the authors show the ID and OOD performance of pruned model using different approaches. I agree that 65% ~ 70% accuracy on HANS dataset can be a reasonably good performance. \n\nHowever, is the experimental scores sufficient to draw conclusion that we have extracted subnetworks that are robust to OOD?  How do we define the robustness here? What network is robust and what is not robust? \n\nAre we confident that the conclusion that we obtained from these datasets are generalizable?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "oqmxQOSHEah", "reviewer": "Reviewer_6mgh", "summary": "The paper studies whether sparse networks pruned with different methods (i.e., mask train or IMP), or different loss functions (i.e., standard cross-entropy loss or product-of-export loss) can perform as well as unpruned models on the ID and OOD settings. The paper is presented in a very straightforward manner -- applying existing methods to various tasks in three evaluation settings and demonstrating the results for each setting. The experiments are solid and provide interesting results. The main concern with this paper is that it lacks technical contributions and deeper insights.", "questions": "- Why do you want to study these two problems together? What’s the motivation behind?\n", "limitations": "The paper only mentions the limitations of only conducting experiments on BERT and NLU datasets. See weaknesses above about how the paper can be further improved.\n", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is well organized and easy to follow in general.\n- The paper studies two important problems -- seeding for subnetworks and mitigating models’ biases. Both problems are very crucial in the field and addressing these problems can have high impacts.\n- The experiments are solid.\n\nWeaknesses:\n- The paper can be better motivated and better positioned. The paper claims the paper “presents the first systematic study on sparsity and dataset bias robustness for PLMs”. However, these two problems are not connected well. Each of these two problems can be viewed as an independent and important problem but I don’t see a clear motivation in the paper about why we want to study these two problems jointly.\n- The paper lacks technical contributions. The paper applies existing several approaches in different evaluation settings and does not propose new methodology.\n- Although the paper conducts extensive experiments, I still feel the paper lacks deeper discussion which may provide insightful ideas to facilitate further research. \n- For some conclusions in the paper regarding the experimental results, it may be hasty to draw. For example, the paper suggests that the ID performance of BERT depends on memorizing the dataset bias. This is not convincing to me only based on the ID performance and the OOD performance.\n\n=========================\n\nI have read the responses from the authors and my concerns are partially addressed. I have reassessed this paper and updated the score.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "T31fv2nzSck", "reviewer": "Reviewer_fVow", "summary": "This paper basically answers the question if there exist sparse subnetworks of pre-trained LMs that can work well under the out-of-domain setting. They focus on the BERT model and three NLU tasks and demonstrate that there exists subnetwork that do well on OOD setting for these three tasks. The findings are interesting but I found it lacks a conclusive result of how to efficiently find such sparse subnetworks.", "questions": "Questions:\nIn figure 2, which are the OOD plots and which are the ID plots?\n", "limitations": "They slightly mention the limitation of not exploring other types of pre-trained LMs and other tasks.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- They present some interesting findings about subnetworks of PLMs: (1) there exist subnetworks that can achieve great OOD performance meanwhile preserve the in-domain (ID) performance and (2) even the fine-tuned BERT model is biased, there also exist unbiased subnetwork in it. \n- They tried both IMP (iterative magnitude pruning) and Mask training, and CE loss and PoE (product of experts) loss and found that with PoE, you can directly learn effective subnetworks from the biased fine-tuned BERT model.\n- They show that gradually increasing sparsity could help the training.\n\nWeaknesses:\nThe proposed approach to fine-tune with biased models (PoE) seems not generalizable to arbitrary tasks. For all experiments, it uses biased models that are trained using known spurious features. I wonder how such a technique could be generalized to other tasks?\n\nOne concern I have is that, after reading the paper, I am still not sure how to find such subnetworks that do well on both ID and OOD efficiently. \n- How to determine the best sparsity to trade-off performance in ID and OOD?\n- How you improve the subnetwork searching process. How do you determine when to find subnetworks during the fine-tuning?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "QV34CVhpNqi", "reviewer": "Reviewer_Spmm", "summary": "* The paper presents an analysis of the intersection of sparsification methods and robustness vis-a-vis in-domain (ID) and out-of-domain (OOD) performance, showing that sparse and robust networks (dubbed SRNets in the paper) can be found for pre-trained BERT models, finetuned BERT models and sparsified pre-trained models that are then finetuned in isolation.\n* Concretely, the paper considers 2 pruning methods: Iterative Magnitude Pruning and Mask Training; and 2 training paradigms: standard finetuning and Product of Experts (PoE), the latter to improve OOD performance. They present their results on 3 datasets: MNLI (with HANS as OOD), QQP (with PAWS-wiki and PAWS-qqp as OOD datasets) and FEVER (with FEVER Symmetric v1 and v2 as OOD datasets).\n* For pruned finetuned models, the authors show that the sparsified models achieve competitive performance compared to full model performance upto 70% sparsity, and achieve better than finetuned OOD performance. When trained with PoE, the sparsified models achieve competitive ID and OOD performance at 70% sparsity.\n* The authors show that for sparsified pre-trained models, finetuned in isolation, the models tend to achieve better OOD performance, showing that SRNets can be found in pre-trained BERT models, and can be fine-tuned in isolation.\n* Further the authors show that for BERT subnetworks without finetuning, the pruned networks are competitive with full finetuned models both while using standard and PoE objectives.\n* Finally, the authors show that even using intermediate finetuned checkpoints for pruning can be used to achieve good finetuning performance, and also present a methodology to achieve improved performance for high sparsity networks.", "questions": "Questions for the Authors\n1.  Was there any reason why PoE variant (as opposed to the Debiasing Focal Loss variant) was considered for the experiments ?\n2. I am not sure what line 63 means (\"Second, we ameliorate the mask training method ...\"). Would it be possible to rephrase it ?\n3.  For the results on BERT Subnetworks Fine-tuned in Isolation, it seems like IMP (PoE) + std ft. does considerably worse compared to full bert std finetuning on the OOD dataset (HANS) compared to Mask train (PoE) + std ft. I am curious if there is any hypothesis on why this might be the case ?\n4. The IMP methods seem to have a considerably higher variance compared to the Mask Train methods. Is there a hypothesis on why this might be the case ?\n\nTypographic Edits:\n1. Line 8: even inside 3) PLM without -> 3) even inside PLM without \n2. Line 49: BERT do contain -> BERT does contain", "limitations": "The authors have considered the limitations fairly well. One minor addition that might be helpful would be to elaborate a bit more on how would the exploration on other type of tasks look like (eg: what sort of biases would you want to investigate w.r.t language generation and how would you adapt the proposed methodology for the same).", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper presents a comprehensive analysis on a novel area in the intersection of OOD robustness and sparsification of finetuned models, which is quite relevant from a usability standpoint\n2. The existence of SPNets in the different setups considered is quite intriguing, and does present an interesting direction of work. \n3. The results of being able to combine PoE and pruning methods is quite useful from a practical standpoint for deploying models in the wild. \n\nWeaknesses:\n1. My main concern with is with Section 4.1. The authors claim that they \"refine the SRNet searching process to improve efficiency ..\" (Line 69). However, from Section 4.1, while the authors show that starting from earlier steps can yield competitive results compared to full finetuning, the authors make no recommendations on how to find such a step a-priori (eg: as a function of the model / dataset etc). For example, why should the 20k steps be what is considered, and not any other step-count. Or is it possible to predict at what step count should we start the pruning process, without actually finetuning the entire model. Overall, given that this is listed as a main contribution of the work, it would have been nice to have a more in-depth analysis of this topic.\n2. While the results presented as figures does help analyse trends, it would be helpful to also include the actual values in a Table as well. Otherwise, it makes comparisons (especially across the different setups) extremely hard (eg: between BERT subnetworks without Finetuning vs BERT subnetworks on finetuned models vs BERT subnetworks finetuned in isolation).\n3. It would be good to consider at least one additional pre-trained model to show some generality of observations across models (Eg: RoBERTa[1], especially since RoBERTa has been shown to have better OOD generalization than BERT [2,3] ).\n4. Additionally, OOD robustness is observed more for large models compared to base models ([2]). Thus it would be good to see how much value add would the proposed methods bring for those scenarios. Specifically, would it still be the case that pruning yields comparable to better OOD performance, or would the loss in model capacity result in worse OOD performance compared to the baseline methods.\n5. [Minor] Line 125 mentions example reweighing and confidence regularization also being considered for the work, but it is not mentioned otherwise in the paper. It would be good to include those results as well.\n\n\nReferences\n\n[1] Liu, Yinhan, et al. \"Roberta: A robustly optimized bert pretraining approach.\" arXiv preprint arXiv:1907.11692 (2019).\n\n[2] Tu, Lifu, et al. \"An empirical study on robustness to spurious correlations using pre-trained language models.\" Transactions of the Association for Computational Linguistics 8 (2020): 621-633.\n\n[3] Hendrycks, Dan, et al. \"Pretrained transformers improve out-of-distribution robustness.\" arXiv preprint arXiv:2004.06100 (2020).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept