Evaluating Parameter Efficient Learning for Generation (2022)

Abstract: Parameter efficient learning methods (PERMs)have recently gained significant attention asthey provide an efficient way for pre-trainedlanguage models (PLMs) to adapt to a downstream task. However, these conclusions aremostly drawn from in-domain evaluations overthe full training set. In this paper, we presentcomparisons between PERMs and finetuningfrom three new perspectives: (1) the effect ofsample and model size to in-domain evaluations, (2) generalization to unseen domains andnew datasets, and (3) the faithfulness of generations. Our results show that for in-domainsettings (a) there is a cross point of samplesize for which PERMs will perform better thanfinetuning when training with fewer samples,and (b) larger PLMs have larger cross points.For cross-domain and cross-dataset cases, weshow that (a) Adapter (Houlsby et al., 2019)performs the best amongst all the PERMs studied here, and (b) it outperforms finetuning ifthe task dataset is below a certain size. Wealso compare the faithfulness of generationsand show that PERMs can achieve better faithfulness score than finetuning, especially forsmall training set, by as much as 6%. Finally,we apply Adapter to MT-NLG 530b (Smithet al., 2022) and achieve new state-of-the-artresults on Xsum (Narayan et al., 2018) for allROUGE scores (ROUGE-1 49.17, ROUGE-227.20, ROUGE-L 40.98).

## introduction parameter efficient learning methods (perms) serve as potential alternatives to finetuning for adapting and deploying language models in real world scenarios @xcite . they allow users to finetune only a small number of parameters while freezing the rest of the shared parameters of pre-trained language models (plms). this is especially important for large language models (e.g. gpt-3 @xcite and mt-nlg @xcite ) as finetuning the entire model will be very expensive or infeasible due to their model size. prefix tuning @xcite , which is one of the perms, draws inspiration from prompting and introduces a small set of continuous vectors as virtual prompts to allow subsequent tokens to attend to, which obtains comparable performance to finetuning in the full data setting. prompt tuning @xcite shows the power of scaling plms and that tuning only a few extra embeddings is sufficient to achieve similar performance to finetuning the entire 11b t5-xxl (raffel et al., 2020) model. p-tuning v2 @xcite further demonstrates that small plms can also achieve comparable results to finetuning with prefix tuning. different from adding new parameters through prompts, adapter @xcite injects trainable parameters through lowrank structure in a skip-connection way. other perms includes lora @xcite , mix-and-match adapter @xcite , compactor @xcite , @xcite , diff-pruning @xcite and etc. most conclusions about perms so far are drawn from their in-domain evaluations over full training samples. to the best of our knowledge, it is not yet investigated (1) how these conclusions apply to different training sizes and model sizes, and (2) how perms generalize to unseen domains and new datasets, which are both important aspects for deploying perms in real-world applications. in addition, faithfulness in natural language generation has become an important topic as it is vital to real-world applications. various efforts are made to systematically measure and mitigate factual errors in many generation tasks, including summarization @xcite and dialogue generations @xcite @xcite . however, existing work on faithfulness only focuses on faithfulness of finetuning, and the impact of perms on the faithfulness of generation is not yet explored. in this paper, we provide an in-depth study of perms for generation tasks through three important aspects when deploying perms in practical applications: (1) in-domain evaluation by scaling both training dataset size and model size of plms, (2) cross-domain and cross-dataset generalization, and (3) faithfulness assessment. two generation tasks are used for evaluation: summarization and dialogue generation. we study four representative methods: p-tuning, prompt tuning, prefix tuning, and adapter, but mainly focus on prefix tuning and adapter as our preliminary results show that they are better than the others. our contributions are summarized as follows: (1) to the best of our knowledge, we present the first comparisons of faithfulness for perms. our experimental results show that perms, especially prefix tuning can achieve better faithfulness than finetuning by up to 6%. (2) for in-domain settings, there is always a cross point of sample size for which perms will be better than finetuning when training on fewer samples. larger plms have larger cross points. users need to choose which method to use based on their own training sample size and model size. ## methodology we compare the following four perms to finetuning (ft) using gpt-style models from megatron-lm @xcite . (1) adapter (ap) adds an extra layer with a bottleneck structure by first projecting input h to a low dimension using trainable weights w down and then projecting up to the original dimension using trainable weights w up . it is incorporated into backbone model in a skipconnection way. we also add reparameterization trick suggested by @xcite . (3) prompt tuning (pt) adds extra parameters to the embedding layer and uses these trainable embeddings to prompt the input. (4) p-tuning @xcite adds a prompt encoder to encode pseudo prompts and the encoded representation is used to prompt the input. ## conclusion in this paper, we extensively compare perms with finetuning over three main areas: (1) in-domain evaluation by scaling both the sample size and model size (2) cross-domain and cross-dataset generalization (3) faithfulness of generations. for in-domain settings, we find (a) there is a cross point of sample size for which parameter efficient learning will be better than finetuning when training with fewer samples and (b) larger plms have larger cross points. this suggests users to choose the method based on their own sample size and model size. simply apply adapter to mt-nlg, we achieve new state-of-the-art results on xsum for all rouge scores (rouge-1 49.17, rouge-2 27.20, rouge-l 40.98). compared to finetuning, not all perms can easily achieve better crossdomain and cross-dataset scores than finetuning even with large plm (e.g. 8.3b). adapter is a better choice than other perms in such cases. lastly, we provide the first comparison of perms over faithfulness of generations and show that prefix tuning is the best method for faithfulness. we believe our findings will help users better choose and deploy perms.