Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The motivation behind this paper leaves from current limitations of fine-tuning, namely fine-tuning instability and poor generalizability (particularly in OOD settings). To address these limitations, the authors propose a Dynamic Parameter Selection (DPS) algorithm, which fine-tunes pre-trained networks adaptively in two stages: 1) Stage I accumulates gradients of updated parameters during fine-tuning, and 2) Stage II derives a subnetwork composed of the important parameters to update based on the accumulated values in Stage I. The authors propose two versions of their algorithm, DPS Dense (which optimizes the entire network update process) and DPS Mix (which optimizes the stochastic parameter update process). 

The authors conduct experiments in OOD setting and low-resource scenarios on GLUE and NLI datasets, showing their proposed method outperforms prior approaches. On the GLUE benchmark, DPS gains range between 0.44-1.33 points and 0.29-0.8- std. Moreover, the authors conduct ablation experiments where they analyze performance of DPS when varying the subnetwork size, using various pre-trained models, sufficient number of training examples and time duration. The authors conclude that their proposed DPS method leads to stable improvement in pre-trained model performance with sufficient number of training examples, is time-efficient and captures task-relevant parameters across a variety of subnetwork sizes.

strengths_and_weaknesses: Strengths: 
- The intuition behind the idea that fine-tuning a subnetwork instead of the entire network can yield more relevant parameters for a specific downstream task is valuable and does make a lot of sense


- The authors use an extensive set of baselines from the literature to compare their proposed method with, including Mixout (Lee et al, 2020), R3F (Aghanjanyan et al, 2021), R-Dropout (Wu et al, 2021), Child-tuning (Xu et al, 2021), and Re-init (Zhang et al, 2021)


- The authors convincingly demonstrate that their method performs well in Out-of-distribution and low-resource scenarios, and the ablation experiments show that the proposed method works when combined with different pre-trained models

Weaknesses: 
- DPS dense needs a first pass in Stage I that finetunes the whole network for gradient accumulation. While I understand the need for this pass, it seems wasteful, why would someone use DPS on top of fine-tuning the network? Unfortunately the authors do not discuss this limitation of their proposed method in the paper.


- While DPS Mix is proposed as an alternative to optimize the fine-tuning process, it requires a Frequency Accumulation Matrix (FAM) to store the frequency of every parameter being updated, in addition to the Gradient Accumulation Matrix (GAM) required by DPS Dense. The authors do not discuss the computational bottleneck and complexity of these design choices. Moreover, Stage I of DPS Mix is identical to previously proposed methods in the literature (Mixout).


- The authors are mainly comparing the obtained results with ChildTuning, without motivating why they focus particularly on this baseline method.

questions: Line 175: “DPS Dense outperforms all baselines” - from the results in Table 1, DPS Dense does not seem to outperform all baselines, particularly on MRPC and STS-B datasets. While on average it does, I would recommend clarifying this aspect in the presentation of the results. 

Lines 177-178: “DPS saves 177 additional computational overhead imposed by computing Fisher Information based on all training 178 samples”

Regarding the results in Table 2 (Out-of-Domain Generalization), how do you explain that DPS Dense is performing better than DPS mix on the MNLI dataset, while results on the SNLI dataset seem to show the opposite trend that DPS Dense outperforms DPS Mix? What about the standard deviation?

Table 4: Have any of the pre-trained models been trained on the datasets you are evaluating their performance on?

Section 4.3: “two datasets in GLUE with training samples > 50K” - which is the motivation behind selecting datasets with > 50K in particular? The paper does not provide an explanation for your particular choice of > 50K.

limitations: The work seems largely incremental in nature, and the methods proposed by the authors are largely inspired by previous work in the literature. The authors nevertheless convincingly show the good performance of their method.

The paper lacks a comprehensive discussion of the limitations of the proposed methods, the computational overhead introduced. While Section 4.4 talks about the slight computational costs compared to vanilla fine-tuning, this is only in time usage and not in computational complexity.

Line 80: typo - “Differ from fine-tuning whole network” (--> different)



The paper addresses a relevant problem on how to more efficiently fine-tune pretrained models, particularly in data-limited regime and out of distribution settings. The method proposed by the authors is inspired by previous work, nevertheless empirical results demonstrate it performs well. The paper would benefit from a more clear discussion of the limitations of the proposed approach, particularly in terms of the computational complexity and the need to first do fine-tuning before applying the proposed approached (DPS Dense), as well as the additional data structures needed to store the frequency of each parameter being update (DPS Mix).
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposed a new large-scale language model fine-tuning algorithm called Dynamic Parameter Selection (DPS). This algorithm essentially uses a two stage cyclic updating strategy, where in Stage-1 it accumulates in-batch gradients of updated parameters during fine-tuning, and in Stage-2, it derives a subnetwork based on the Stage-1 accumulated gradients and their relative importance. The paper has proposed two variants of the algorithm called DPS Dense and DPS Mix, and show that it consistently performs better than previous approaches on sub-network finetuning strategies. Also, the experimental results on the out-of-domain show the usefulness of DPS. 

strengths_and_weaknesses: Strengths: 
1) The paper is well written and easy to follow. 
2) The proposed DPS model is interesting and novel with better results than other algorithms related to sub-network optimization.
3) The results on out of domain generalization are very exciting. However, it is not clear why this approach leads to better generalization. 
4)  The ablations are very thorough and useful such as experiments on low resource scenario, effect of subnetwork size, different pre-trained language models, and DPS with sufficient number of training examples. 

Weaknesses: 
1) It would have been more interesting if the paper could show any theoretical guarantees or understanding of why this algorithm is superior over other sub-network optimizations. 
2) It would be interesting to check whether the observations hold for more than the current 4 tasks reported in Table-1. Is there any reason for not showing on a complete GLUE dataset?

questions: 1) In Table-1, what is the reason to choose only four tasks from GLUE and why only these tasks? 
2) Suggestion: Maybe it is better to merge Section-5 with Section-2. 
3) In all the ablations, comparison is based on vanilla fine-tuning, can you also add any sub-network optimization model as well? 

limitations: N/A
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a Dynamic Parameter Selection (DPS) algorithm to address the overfitting and catastrophic forgetting issues in fine-tuning pre-trained language models. 
The proposed DPS consists of a cyclic two-stage updating strategy:
1) accumulates the in-batch gradients of updated params during fine-tuning.
2) using the accumulated values in stage 1 to derive a subnetwork and keep non-subnetwork params constant.

The experiments are mainly carried out on the classification tasks of GLUE. 
The proposed DPS is tested under various settings, and the results show that DPS shows consistent improvements over various baseline systems, which demonstrates the effectiveness of DPS.
strengths_and_weaknesses: ### Strengths
1. The proposed method addresses an important topic of the overfitting and catastrophic forgetting issues in fine-tuning pre-trained language models.
2. The formulation of the proposed DPS is technically sound and reasonable. 
3. The experiments are carried out in various aspects to fully demonstrate the effectiveness of the proposed model. 


### Weaknesses
1. The experiments are only carried out on classification tasks. Using one more type of NLU task would have further strengthened the results.
2. The discussion of the limitation is not informative.
questions: ### Major comments
1. The experiments are only carried out on classification tasks. Did you apply your method to other tasks, such as machine reading comprehension (MRC), or named entity recognition (NER)?
2. In Section 4.4, the authors have demonstrated that the proposed DPS is more efficient than other methods in time usage. How about memory consumption?
3. line 187-188: Did you arbitrarily choose the size (8K) of subsampled MNLI/SNLI? 
4. It would be nice if the authors could explicitly discuss and compare two variants of DPS along with the experimental results. 
5. Section 3.5: If I understand correctly, all experiments are carried out on the same subsets (either 0.5k or 1k) of the tasks in GLUE. It would be nice if the authors could release what the exact subsets are sampled to enable the researchers to compare the results in the future.


### Minor comments
1. line 63: I know what is "std," but please use its full name when it first appears in a formal paper.
2. line 226: DEBERTA -> DeBERTa
3. Figure 1: it seems that "us" is a single variable. Thus, "2kus+i" seems to be quite confusing (might be interpreted as 2*k*u*s). Please try to make the naming of the variable more clear.
limitations: The authors claim in the checklist that they have discussed their limitation in Section 4.4, noting the additional training cost of their method. However, the Section 4.4 seems not to be discussing their limitation but advantages over other methods. The authors are encouraged to discuss some "true" limitations of their method. For example, does your model consume more memory than other methods? Please try to take a balanced view in interpreting the limitation of your method.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Large pre-trained LMs form the baselines of many impressive results, but fine-tuning can, among other challenges, lead to overfitting and degradation of internal representations. This paper introduces Dynamic Parameter Selection, which selects subnetworks to perform staging updates based on gradients. The GLUE benchmark is used for evaluation, and results are positive. 
strengths_and_weaknesses: _Strengths_

- the two-stage process of accumulating in-batch gradients during fine-tuning, and subsequently using those gradients to derive stage-specific subnetworks to update is conceptually simple, and therefore clear.
- GLUE and the NLI datasets are appropriate, generic datasets. 
- The models are very competitive, though in many cases the best means are within the indicated std (Table 1) therefore some statistical significance testing (correcting for multiple comparisons) is advised, as little can be conclusively concluded otherwise. The multitude of experimental scenarios (e.g., OoD generalization, low-resource scenarios) is a positive.


_Weaknesses_

- some minor spelling and grammatical mistakes (e.g., ‘has been prove’, L28; ‘to randomly replaces’, L30; ‘same sized’, L85; CHID-TUNING, L87, L92; ‘will not be update’, fig 1 caption) should be addressed. Along these lines, Figures 1 and 2 need to have their text made larger, for readability. References should be checked for proper formatting
- the treatment of related work (Sec 5) is quite superficial — more synthesis of the various approaches should be attempted.

questions: - why are so many (8) references needed at L17 to support the idea that pre-trained LMs are useful in NLP?
- while the limitations on pre-trained LMs in Sec 1 around overfitting and catastrophic forgetting are well known, how are the specific arguments (esp L34) related to those limitations?
- much of the information stored in neural networks concerns the relative differences between weights in the network — does altering only a subpart of a network affect not only its internal structure but its relationship with other information elsewhere in the network?
- As this approach is fairly generic, would it be straightforward to add vision datasets to those used in Sec 3.1?
limitations: Sec 4.4 is said to discuss limitations, though this is done only superficially, essentially only serving to further hype DPS.
review_title: null

Meta-review: metareview: The reviewers were very positive about this paper, which proposes a method for selective fine-tuning of language models on downstream tasks. The method works well in various settings including realistic out-of-distribution and low-resource scenarios. 

The main strengths are:
* useful idea 
* conceptually simple method
* the choice of datasets
* the use of competitive models
* convincing experiments in multiple settings, including OOD generalization 

The reviewers also noted some weaknesses, especially: 
* the need for a first pass, computational bottleneck, --> author response seems satisfactory 
* the choice of baseline to compare against 
* need for theoretical guarantees  --> to me, this is not strictly necessary for an empirical paper 
* experiments only with classification tasks --> to me, the choice of tasks (GLUE, NLI) is reasonable, as the authors argue 
* uninformative discussion of limitations

The authors did a fine job addressing most of these issues in their response and they should update their paper accordingly. Concerning the last point: the authors did not address this point, and are strongly encouraged to do so in their revision. 
confidence: Certain
Decision: Accept