A dapter F usion: Non-Destructive Task Composition for Transfer Learning (2021)

Abstract: Sequential fine-tuning and multi-task learning are methods aiming to incorporate knowledge from multiple tasks; however, they suffer from catastrophic forgetting and difficulties in dataset balancing. To address these shortcomings, we propose AdapterFusion, a new two stage learning algorithm that leverages knowledge from multiple tasks. First, in the knowledge extraction stage we learn task specific parameters called adapters, that encapsulate the task-specific information. We then combine the adapters in a separate knowledge composition step. We show that by separating the two stages, i.e., knowledge extraction and knowledge composition, the classifier can effectively exploit the representations learned from multiple tasks in a non-destructive manner. We empirically evaluate AdapterFusion on 16 diverse NLU tasks, and find that it effectively combines various types of knowledge at different layers of the model. We show that our approach outperforms traditional strategies such as full fine-tuning as well as multi-task learning. Our code and adapters are available at AdapterHub.ml.

## introduction the most commonly used method for solving nlu tasks is to leverage pretrained models, with the dominant architecture being a transformer @xcite , typically trained with a language modelling objective @xcite @xcite . transfer to a task of interest is achieved by fine-tuning all the weights of the pretrained model on that single task, often yielding state-of-the-art results @xcite @xcite . however, each task of interest requires all the parameters of the network to be fine-tuned, which results in a specialized model for each task. @xcite . the adapterfusion component takes as input the representations of multiple adapters trained on different tasks and learns a parameterized mixer of the encoded information. there are two approaches for sharing information across multiple tasks. the first consists of starting from the pretrained language model and sequentially fine-tuning on each of the tasks one by one @xcite . however, as we subsequently fine-tune the model weights on new tasks, the problem of catastrophic forgetting @xcite can arise, which results in loss of knowledge already learned from all previous tasks. this, together with the nontrivial decision of the order of tasks in which to fine-tune the model, hinders the effective transfer of knowledge. multi-task learning @xcite @xcite is another approach for sharing information across multiple tasks. this involves fine-tuning the weights of a pretrained language model using a weighted sum of the objective function of each target task simultaneously. using this approach, the network captures the common structure underlying all the target tasks. however, multi-task learning requires simul-taneous access to all tasks during training. adding new tasks thus requires complete joint retraining. further, it is difficult to balance multiple tasks and train a model that solves each task equally well. as has been shown in @xcite , these models often overfit on low resource tasks and underfit on high resource tasks. this makes it difficult to effectively transfer knowledge across tasks with all the tasks being solved equally well @xcite , thus considerably limiting the applicability of multi-task learning in many scenarios. recently, adapters @xcite have emerged as an alternative training strategy. adapters do not require fine-tuning of all parameters of the pretrained model, and instead introduce a small number of task specific parameters -while keeping the underlying pretrained language model fixed. thus, we can separately and simultaneously train adapters for multiple tasks, which all share the same underlying pretrained parameters. however, to date, there exists no method for using multiple adapters to maximize the transfer of knowledge across tasks without suffering from the same problems as sequential fine-tuning and multi-task learning. for instance, stickland and @xcite propose a multi-task approach for training adapters, which still suffers from the difficulty of balancing the various target tasks and requiring simultaneous access to all target tasks. in this paper we address these limitations and propose a new variant of adapters called adapter-fusion. we further propose a novel two stage learning algorithm that allows us to effectively share knowledge across multiple tasks while avoiding the issues of catastrophic forgetting and balancing of different tasks. our adapterfusion architecture, illustrated in figure 1 , has two components. the first component is an adapter trained on a task without changing the weights of the underlying language model. the second component -our novel fusion layer -combines the representations from several such task adapters in order to improve the performance on the target task. contributions our main contributions are: (1) we introduce a novel two-stage transfer learning strategy, termed adapterfusion, which combines the knowledge from multiple source tasks to perform better on a target task. (2) we empirically evaluate our proposed approach on a set of 16 diverse nlu tasks such as sentiment analysis, commonsense reasoning, paraphrase detection, and rec-ognizing textual entailment. (3) we compare our approach with stickland and @xcite where adapters are trained for all tasks in a multi-task manner, finding that adapterfusion is able to improve this method, even though the model has simultaneous access to all tasks during pretraining. (4) we show that our proposed approach outperforms fully fine-tuning the transformer model on a single target task. our approach additionally outperforms adapter based models trained both in a single-task, as well as multi-task setup. the code of this work is integrated into the adapterhub.ml @xcite . ## background in this section, we formalize our goal of transfer learning @xcite @xcite , highlight its key challenges, and provide a brief overview of common methods that can be used to address them. this is followed by an introduction to adapters @xcite and a brief formalism of the two approaches to training adapters. ## adapterfusion adapters avoid catastrophic forgetting by introducing task-specific parameters; however, current adapter approaches do not allow sharing of information between tasks. to mitigate this we propose adapterfusion. ## experiments in this section we evaluate how effective adapter-fusion is in overcoming the issues faced by other transfer learning methods. we provide a brief description of the 16 diverse datasets that we use for our study, each of which uses accuracy as the scoring metric. ## results we present results for all 16 datasets in ## analysis of fusion activation we analyze the weighting patterns that are learned by adapterfusion to better understand which tasks impact the model predictions, and whether there exist differences across bert layers. we plot the results for layers 1, 7, 9, and 12 and st-a in figure 4 (see appendix figure 6 for the remaining layers). we find that tasks which do not benefit from adapterfusion tend to more strongly activate their own adapter at every layer (e.g. argument, hellaswag, mnli, qqp, scitail). this confirms that adapterfusion only extracts information from adapters if they are beneficial for the target task m. we further find that mnli is a useful intermediate task that benefits a large number of target tasks, e.g. boolq, sick, csqa, sst-2, cb, mrpc, rte, which is in line with previous work @xcite @xcite . similarly, qqp is utilized by a large number of tasks, e.g. sick, imdb, rte, cb, mrpc, sst-2. most importantly, tasks with small datasets such as cb, rte, and mrpc often strongly rely on adapters trained on large datasets such as mnli and qqp. interestingly, we find that the activations in layer 12 are considerably more distributed across multiple tasks than adapters in earlier layers. the potential reason for this is that the last adapters are not encapsulated between frozen pretrained layers, and can thus be considered as an extension of the pre-diction head. the representations of the adapters in the 12 th layer might thus not be as comparable, resulting in more distributed activations. this is in line with @xcite who are able to improve zero-shot cross-lingual performance considerably by dropping the adapters in the last layer. ## contemporary work in contemporaneous work, other approaches for parameter efficient fine-tuning have been proposed. @xcite train sparse "diff" vectors which are applied on top of pretrained frozen parameter vectors. ravfogel and goldberg (2021) only finetune bias terms of the pretrained language models, achieving similar results as full model finetuning. li and liang (2021) propose prefix-tuning for natural language generation tasks. here, continuous task-specific vectors are trained while the remaining model is kept frozen. these alternative, parameter-efficient fine-tuning strategies all encapsulate the idiosyncratic task-specific information in designated parameters, creating the potential for new composition approaches of multiple tasks. @xcite analyse the training and inference efficiency of adapters and adapterfusion. for adapterfusion, they find that adding more tasks to the set of adapters results in a linear increase of computational cost, both for training and inference. they further propose approaches to mitigate this overhead.