Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper basically answers the question if there exist sparse subnetworks of pre-trained LMs that can work well under the out-of-domain setting. They focus on the BERT model and three NLU tasks and demonstrate that there exists subnetwork that do well on OOD setting for these three tasks. The findings are interesting but I found it lacks a conclusive result of how to efficiently find such sparse subnetworks.
strengths_and_weaknesses: Strengths:
- They present some interesting findings about subnetworks of PLMs: (1) there exist subnetworks that can achieve great OOD performance meanwhile preserve the in-domain (ID) performance and (2) even the fine-tuned BERT model is biased, there also exist unbiased subnetwork in it. 
- They tried both IMP (iterative magnitude pruning) and Mask training, and CE loss and PoE (product of experts) loss and found that with PoE, you can directly learn effective subnetworks from the biased fine-tuned BERT model.
- They show that gradually increasing sparsity could help the training.

Weaknesses:
The proposed approach to fine-tune with biased models (PoE) seems not generalizable to arbitrary tasks. For all experiments, it uses biased models that are trained using known spurious features. I wonder how such a technique could be generalized to other tasks?

One concern I have is that, after reading the paper, I am still not sure how to find such subnetworks that do well on both ID and OOD efficiently. 
- How to determine the best sparsity to trade-off performance in ID and OOD?
- How you improve the subnetwork searching process. How do you determine when to find subnetworks during the fine-tuning?

questions: Questions:
In figure 2, which are the OOD plots and which are the ID plots?

limitations: They slightly mention the limitation of not exploring other types of pre-trained LMs and other tasks.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors studied the network pruning through (1) iterative magnitude-based pruning and (2) mask training, while combing the debasing loss during the pruning process. For different settings, the paper provides thorough experiment with analysis on the results. Experiments reveal that there are subnetworks with a certain sparsity level and robustness on ODD datasets. 
strengths_and_weaknesses: Strength:

1. The evaluation is thorough and clear, the authors present figures where readers can see how the sparsity level is correlated with the model performance
2. The model scores reported in the figures are at a reasonably high level

Weakness:

1. Overall the results seems to be what I have expected. Take section 3.2.2 as example, (1) Existing papers have shown that the pruning can be done at around 70% sparsity level with modest performance drop. (2,3) We can consider both the imp and mask training as learnable sparsity inducing methods. We are expecting the OOD performance to be improved over standard loss by training the parameters in such a way during pruning. (4) This statement is describing the observation of experiments in this paper. How can we prove this 20% - 70% range can be generalized? 

2. The conclusion that the sparse and robust BERT subnetworks exist seems to be made from experiment results on three dataset settings. In figure 2, the authors show the ID and OOD performance of pruned model using different approaches. I agree that 65% ~ 70% accuracy on HANS dataset can be a reasonably good performance. 

However, is the experimental scores sufficient to draw conclusion that we have extracted subnetworks that are robust to OOD?  How do we define the robustness here? What network is robust and what is not robust? 

Are we confident that the conclusion that we obtained from these datasets are generalizable?
questions: Please check the weakness section.
limitations: I don't have concern on negative societal impact for this paper.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper studies whether sparse networks pruned with different methods (i.e., mask train or IMP), or different loss functions (i.e., standard cross-entropy loss or product-of-export loss) can perform as well as unpruned models on the ID and OOD settings. The paper is presented in a very straightforward manner -- applying existing methods to various tasks in three evaluation settings and demonstrating the results for each setting. The experiments are solid and provide interesting results. The main concern with this paper is that it lacks technical contributions and deeper insights.
strengths_and_weaknesses: Strengths:
- The paper is well organized and easy to follow in general.
- The paper studies two important problems -- seeding for subnetworks and mitigating models’ biases. Both problems are very crucial in the field and addressing these problems can have high impacts.
- The experiments are solid.

Weaknesses:
- The paper can be better motivated and better positioned. The paper claims the paper “presents the first systematic study on sparsity and dataset bias robustness for PLMs”. However, these two problems are not connected well. Each of these two problems can be viewed as an independent and important problem but I don’t see a clear motivation in the paper about why we want to study these two problems jointly.
- The paper lacks technical contributions. The paper applies existing several approaches in different evaluation settings and does not propose new methodology.
- Although the paper conducts extensive experiments, I still feel the paper lacks deeper discussion which may provide insightful ideas to facilitate further research. 
- For some conclusions in the paper regarding the experimental results, it may be hasty to draw. For example, the paper suggests that the ID performance of BERT depends on memorizing the dataset bias. This is not convincing to me only based on the ID performance and the OOD performance.

=========================

I have read the responses from the authors and my concerns are partially addressed. I have reassessed this paper and updated the score.

questions: - Why do you want to study these two problems together? What’s the motivation behind?

limitations: The paper only mentions the limitations of only conducting experiments on BERT and NLU datasets. See weaknesses above about how the paper can be further improved.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: * The paper presents an analysis of the intersection of sparsification methods and robustness vis-a-vis in-domain (ID) and out-of-domain (OOD) performance, showing that sparse and robust networks (dubbed SRNets in the paper) can be found for pre-trained BERT models, finetuned BERT models and sparsified pre-trained models that are then finetuned in isolation.
* Concretely, the paper considers 2 pruning methods: Iterative Magnitude Pruning and Mask Training; and 2 training paradigms: standard finetuning and Product of Experts (PoE), the latter to improve OOD performance. They present their results on 3 datasets: MNLI (with HANS as OOD), QQP (with PAWS-wiki and PAWS-qqp as OOD datasets) and FEVER (with FEVER Symmetric v1 and v2 as OOD datasets).
* For pruned finetuned models, the authors show that the sparsified models achieve competitive performance compared to full model performance upto 70% sparsity, and achieve better than finetuned OOD performance. When trained with PoE, the sparsified models achieve competitive ID and OOD performance at 70% sparsity.
* The authors show that for sparsified pre-trained models, finetuned in isolation, the models tend to achieve better OOD performance, showing that SRNets can be found in pre-trained BERT models, and can be fine-tuned in isolation.
* Further the authors show that for BERT subnetworks without finetuning, the pruned networks are competitive with full finetuned models both while using standard and PoE objectives.
* Finally, the authors show that even using intermediate finetuned checkpoints for pruning can be used to achieve good finetuning performance, and also present a methodology to achieve improved performance for high sparsity networks.
strengths_and_weaknesses: Strengths:
1. The paper presents a comprehensive analysis on a novel area in the intersection of OOD robustness and sparsification of finetuned models, which is quite relevant from a usability standpoint
2. The existence of SPNets in the different setups considered is quite intriguing, and does present an interesting direction of work. 
3. The results of being able to combine PoE and pruning methods is quite useful from a practical standpoint for deploying models in the wild. 

Weaknesses:
1. My main concern with is with Section 4.1. The authors claim that they "refine the SRNet searching process to improve efficiency .." (Line 69). However, from Section 4.1, while the authors show that starting from earlier steps can yield competitive results compared to full finetuning, the authors make no recommendations on how to find such a step a-priori (eg: as a function of the model / dataset etc). For example, why should the 20k steps be what is considered, and not any other step-count. Or is it possible to predict at what step count should we start the pruning process, without actually finetuning the entire model. Overall, given that this is listed as a main contribution of the work, it would have been nice to have a more in-depth analysis of this topic.
2. While the results presented as figures does help analyse trends, it would be helpful to also include the actual values in a Table as well. Otherwise, it makes comparisons (especially across the different setups) extremely hard (eg: between BERT subnetworks without Finetuning vs BERT subnetworks on finetuned models vs BERT subnetworks finetuned in isolation).
3. It would be good to consider at least one additional pre-trained model to show some generality of observations across models (Eg: RoBERTa[1], especially since RoBERTa has been shown to have better OOD generalization than BERT [2,3] ).
4. Additionally, OOD robustness is observed more for large models compared to base models ([2]). Thus it would be good to see how much value add would the proposed methods bring for those scenarios. Specifically, would it still be the case that pruning yields comparable to better OOD performance, or would the loss in model capacity result in worse OOD performance compared to the baseline methods.
5. [Minor] Line 125 mentions example reweighing and confidence regularization also being considered for the work, but it is not mentioned otherwise in the paper. It would be good to include those results as well.


References

[1] Liu, Yinhan, et al. "Roberta: A robustly optimized bert pretraining approach." arXiv preprint arXiv:1907.11692 (2019).

[2] Tu, Lifu, et al. "An empirical study on robustness to spurious correlations using pre-trained language models." Transactions of the Association for Computational Linguistics 8 (2020): 621-633.

[3] Hendrycks, Dan, et al. "Pretrained transformers improve out-of-distribution robustness." arXiv preprint arXiv:2004.06100 (2020).
questions: Questions for the Authors
1.  Was there any reason why PoE variant (as opposed to the Debiasing Focal Loss variant) was considered for the experiments ?
2. I am not sure what line 63 means ("Second, we ameliorate the mask training method ..."). Would it be possible to rephrase it ?
3.  For the results on BERT Subnetworks Fine-tuned in Isolation, it seems like IMP (PoE) + std ft. does considerably worse compared to full bert std finetuning on the OOD dataset (HANS) compared to Mask train (PoE) + std ft. I am curious if there is any hypothesis on why this might be the case ?
4. The IMP methods seem to have a considerably higher variance compared to the Mask Train methods. Is there a hypothesis on why this might be the case ?

Typographic Edits:
1. Line 8: even inside 3) PLM without -> 3) even inside PLM without 
2. Line 49: BERT do contain -> BERT does contain
limitations: The authors have considered the limitations fairly well. One minor addition that might be helpful would be to elaborate a bit more on how would the exploration on other type of tasks look like (eg: what sort of biases would you want to investigate w.r.t language generation and how would you adapt the proposed methodology for the same).
review_title: null

Meta-review: metareview: This paper proposes methods to fine sub-networks in BERT that would lead to good performance out of distribution. It considers different settings of when to search for sub-networks in the pre-train/fine-tune paradigm. 

This paper has received borderline reviews, three mildly positive and one mildly negative.  

The strengths in this work seem to be:
* interesting findings
* thorough evaluation
* important problem
* clarity

I agree with the importance of the problem, the very comprehensive evaluation in multiple settings, and the interesting findings, both from the perspective of interpretability and of robustness.  

The reviewers noted several weaknesses, which I'll discuss below. But I think the strengths outweigh the weaknesses and the paper would make interesting contributions to the community. 

Weaknesses:
* generalizability to arbitrary tasks and to other models
* how to trade off iid and ood performance
* how to choose which earlier steps to start from?

There was some discussion with the authors that have led reviewers to update their reviews. Of the weaknesses, I find the choice of tasks reasonable as they are common and well studies in OOD generalization settings. I agree with the need to experiment with other models besides BERT. The authors have added experiments with RoBERTa in their revision, for one task and setting. This makes me more confident of the applicability of the approach, but I'd suggest including similar experiments with the other tasks and settings. 

The question of trade-off of iid and ood performance is inherent to the field and there's no perfect solution. The authors have made reasonable effort by not using OOD data for model selection. One concern that I still have is the use of only product-of-experts for bias mitigation, which is known to be sub-par especially in terms of trade-off, compared to Confidence Regularization. Experiments with confidence regularization would have been great to add, but I also know that the Utama et al. results are sometimes difficult to replicate. Still, consider trying it out and reporting your results. 

On when to start pruning: I agree this is a major limitation that should be clearly stated and discussed. The author response gave some thoughts, please include a thoughtful discussion in the next revision. 
confidence: Less certain
Decision: Accept