Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning (2021)

Abstract: Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus arises in various downstream tasks. In this paper, we propose a straightforward yet effective fine-tuning technique, Child-Tuning, which updates a subset of parameters (called child network) of large pretrained models via strategically masking out the gradients of the non-child network during the backward process. Experiments on various downstream tasks in GLUE benchmark show that Child-Tuning consistently outperforms the vanilla fine-tuning by 1.5 8.6 average score among four different pretrained models, and surpasses the prior fine-tuning techniques by 0.6 1.3 points. Furthermore, empirical results on domain transfer and task transfer show that Child-Tuning can obtain better generalization performance by large margins.

## introduction pretrained language models (plms) have had a remarkable effect on the natural language processing (nlp) landscape recently @xcite @xcite . pretraining and fine-tuning have become a new paradigm of nlp, dominating a large variety of tasks. despite its great success, how to adapt such large-scale pretrained language models with millions to billions of parameters to various scenarios, especially when the training data is limited, is still challenging. due to the extremely large capacity and limited labeled data, conventional transfer learning tends to aggressive fine-tuning @xcite , resulting in: 1) degenerated results on the test data due to overfitting @xcite @xcite , and 2) it forwards on the whole network while backwarding on a subset of network (i.e., child network). right: to achieve this, a task-free or task-driven mask is performed on the gradients of the non-child network, resetting them to zero (grey diagonal grids). in this paper, we propose to mitigate the aggressive fine-tuning problem from a new perspective. based on the observation that it is unnecessary to update all the parameters within the large-scale model during fine-tuning, we propose an effective fine-tuning technique, child-tuning, which straightforwardly updates a subset of parameters (called child network) via strategically masking out the gradients of non-child network in the backward process, as illustrated in figure 1 . note that it is different from model pruning, since it still forwards on the whole network, thus making the full use of knowledge hidden in the pretrained weights. in detail, we propose two variants, child-tuning f and child-tuning d , which respectively detect the child network in a task-free and a task-driven way. child-tuning f chooses out the child network in the absence of task data via a bernoulli distribution. it introduces noise to the full gradients, playing a role of regularization, hence preventing overfitting to small datasets and leading to better generalization. furthermore, child-tuning d utilizes the downstream task data to detect the most task-related parameters as the child network and freezes the parameters in non-child network to their pretrained weights. it decreases the hypothesis space of the model via a task-specific mask applied to the full gradients, helping to effectively adapt the large-scale pretrained model to various tasks and meanwhile greatly maintain its original generalization ability. our extensive experiments on the glue benchmark show that child-tuning can be more excellent at fine-tuning different plms, with up to 8.60 average score improvement on cola/rte/mrpc/sts-b tasks compared to vanilla fine-tuning (section. 3.3). moreover, it achieves better generalization ability in transferring to out-of-domain data and other related tasks (section. 3.4). experimental results also demonstrate that child-tuning yields consistently greater improvements than state-of-the-art fine-tuning methods. more importantly, since child-tuning is orthogonal to these prior methods, integrating child-tuning with them can even lead to further improvements (section. 4.1). in summary, our contributions are three-fold: ## methodology to better adapt large-scale pretrained language model to various downstream tasks, we propose a simple yet effective fine-tuning technique, child-tuning. we firstly introduce a gradient mask in the backward process to achieve the aim of updating a subset of parameters (i.e., child network), while still utilizing the knowledge of the whole large model in the forward process (section 2.1). then, we explore two ways to detect the child network (i.e., generate different gradient masks): child-tuning f that are in a task-free way (section 2.2), and child-tuning d that are in a taskdriven way ( section 2.3). ## related work explosion of plms. there has been an explosion of studies on pretrained language models (plms). @xcite propose bert that is pretrained on large quantities of unannotated corpus with self-supervised tasks. many plms also emerged such as gpt-2 @xcite , gpt-3 (brown et al., 2020), electra @xcite , xlnet @xcite , roberta @xcite , and bart @xcite . the number of parameters of plms also explodes. bert large has 340 millions of parameters, and the number for gpt-3 is even up to 175 billions. effective and generalizable fine-tuning. with a mass of parameters, fine-tuning large plms tend to achieve degenerated performance due to overfitting and have poor generalization ability, especially on small datasets @xcite @xcite . therefore, different finetuning techniques have been proposed. some of them utilize the pretrained weights to regularize the deviation of the fine-tuned model @xcite @xcite , while others compress the output information @xcite or injects noise into the input @xcite . moreover, @xcite and @xcite point out that the omission of bias correction in the adam optimizer used in @xcite is also responsible for the degenerated results. orthogonal to these methods, child-tuning address the problems by detecting the child network within the model in a task-free or task-driven way. it only updates parameters within the child network via a gradient mask, which is proved to be effective in adapting large plms to various tasks, along with better generalization ability. ## conclusion to mitigate the overfitting problem and improve generalization for fine-tuning large-scale plms, we propose a straightforward yet effective fine-tuning technique, child-tuning, which only updates the child network during fine-tuning via strategically masking out the gradients of the non-child network. two variants are introduced, child-tuning f and child-tuning d , which detect the child network in a task-free and task-driven way, respectively. extensive experiments on various downstream tasks show that both of them can outperform vanilla fine-tuning and prior works by large gains among four different pretrained language models, and meanwhile largely enhance the generalization ability of the fine-tuned models. since child-tuning is orthogonal to most prior fine-tuning techniques, integrating child-tuning with them can further boost the performance.