S ca L earn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale (2024)

Abstract: Multi-task learning (MTL) has shown considerable practical benefits, particularly when using language models (LMs). While this is commonly achieved by learning tasks under a joint optimization procedure, some methods, such as AdapterFusion, divide the problem into two stages: (i) task learning, where knowledge specific to a task is encapsulated within sets of parameters (e.g., adapters), and (ii) transfer, where this already learned knowledge is leveraged for a target task. This separation of concerns provides numerous benefits (e.g., promoting reusability). However, current two stage MTL introduces a substantial number of additional parameters. We address this issue by leveraging the usefulness of linearly scaling the output representations of source adapters for transfer learning. We introduce ScaLearn, a simple and highly parameter-efficient two-stage MTL method that capitalizes on the knowledge of the source tasks by learning a minimal set of scaling parameters that enable effective transfer to a target task. Our experiments on three benchmarks (GLUE, SuperGLUE, and HumSet) and two encoder LMs show that ScaLearn consistently outperforms strong baselines with a small number of transfer parameters (~0.35% of those of AdapterFusion). Remarkably, we observe that ScaLearn maintains its strong abilities even when further reducing parameters, achieving competitive results with only 8 transfer parameters per target task. Our proposed approach thus demonstrates the power of simple scaling as a promise for more efficient task transfer. Our code is available at https://github.com/CPJKU/ScaLearn.

## introduction with the wide availability of pre-trained language models (lms) as the backbone of language processing, multi-task learning (mtl) has shown significant benefits, especially for tasks with possible con-ceptual commonalities @xcite @xcite . the traditional paradigm in mtl is to formulate a joint optimization objective based on a set of tasks and train a single model to simultaneously learn and transfer the knowledge relevant to the tasks. this joint mtl approach can be realized by fine-tuning an lm @xcite , or, more recently, by using parameter-efficient, often modularized, mtl approaches @xcite @xcite @xcite . as an alternative to the joint mtl paradigm, some works, such as adapterfusion @xcite , clearly distinguish task training from transfer learning, assigning dedicated parameters to each of these aspects. in this paradigm, referred to as two-stage mtl, first each source task is trained separately and stored into a separate module like an adapter @xcite , and then a task transfer layer is trained for a given target task using information from an arbitrary set of source tasks. this separation of concerns between task and transfer learning offers valuable benefits: (1) learning a separate transfer layer for each target task in a two-stage mtl approach reduces the potentially destructive effects of transfer learning on specific tasks, as the transfer layer parameters corresponding to each target task can independently decide what information should be used from the available source tasks. as shown in our experiments with encoder lms, this supports the effectiveness of transfer learning, making it less sensitive to task selection. (2) since the source tasks can simply be taken from already trained modules (no need for retraining), two-stage approaches promote reusability -a principle of green ai @xcite . further, they provide a practical solution to cases involving issues such as data privacy and/or fairness constraints, as a pre-trained module can readily provide the (e.g., 10 6 10 4 10 2 10 0 10 @xcite and @xcite using roberta base @xcite . the reported values for the two-stage mtl methods only consider the ones in the respective transfer layers. the full details of the learnable parameters and performance results are provided in §5. despite these benefits, current two-stage mtl solutions introduce significantly more learnable parameters compared to recent joint mtl ones, exacerbated by the linear increase in the number of parameters with the number of target tasks. in our experiment setup with eight target tasks using roberta base @xcite , adapterfu-sion introduces ∼ 134% new parameters for transfer learning, while hyperformer++ @xcite conducts joint mtl by adding ∼ 4% (≈ 5m) trainable parameters (details in contributions. we build on insights gained from analyzing the effects of scaling the output representations of adapters, and introduce scalearn, a novel two-stage mtl method that learns to transfer the knowledge of the source adapters using a small set of scaling parameters. for a given target task, scalearn introduces parameters that scale the output representation of each source adapter and combine the resulting scaled representations by simply taking the element-wise sum. this approach results in high parameter-efficiency, such that -following the mentioned experiment setting -scalearn only adds ∼ 0.47% (≈ 0.5m) parameters. we further introduce an even more parameter-efficient variant via uniform scaling (scalearnuniform), where each scaling vector is reduced to a single scaling parameter. finally, by sharing parameters across layers, we achieve our most efficient variation (scalearnuniform++), only containing 64 parameters for transfer learning. we conduct a large set of transfer learning experiments on the glue @xcite , super-glue @xcite , and humset @xcite benchmarks using encoder lms, namely the popular roberta @xcite and xlm-r @xcite ) models, both in their base and large configurations. figure 1 summarizes our results on glue and superglue, showing that scalearn, while providing high efficiency and the benefits of the twostage mtl paradigm, consistently outperforms the baselines. the overall performance of scalearn remains highly competitive in its more parameterefficient variations. our results also show the advantage of two-stage models in avoiding destructive effects during transfer learning. overall, with scalearn we leverage the power of scaling as a viable, non-destructive, simple-to-implement, and highly parameter-efficient solution to the current shortcomings of existing mtl methods, paving the future for more effective and efficient task transfer. ## background in task transfer learning, we consider a pre-trained lm as well as two sets s and t , representing the source and target tasks, respectively. the aim of mtl is to leverage the information of tasks in s to improve the generalization on tasks in t . single task learning (stl). in this basic set-ting, a separate set of parameters is optimized on each task (@xmath0. stl can be done by finetuning the lm parameters or by introducing more parameter-efficient modules into the model, such as adapter modules (pfeiffer adapters @xcite , propetl @xcite , or compacter++ @xcite ), (ia) 3 (liu et al., 2022), prefix-tuning (li and liang, 2021), or lora @xcite , each with θ s parameters for each task s. joint mtl. this approach is commonly done by having a unified model for all tasks (@xmath1, and a joint optimization objective that simultaneously optimizes the model using samples from all tasks @xcite . the general joint mtl objective can be formulated as l @xmath2, where α s is the sampling weight of task s. this optimization objective can be used to fine-tune the parameters of an lm @xcite @xcite , or those of a modularized architecture @xcite @xcite . despite the benefit of having one unified model, the joint loss often causes tasks to compete with each other for learning capacity, leading to the task interference problem @xcite @xcite . this makes the joint mtl paradigm particularly sensitive to the selection of tasks @xcite , while various methods in the literature have aimed to address this issue (e.g., @xcite ; @xcite ; a brief review is provided in § 6). two-stage mtl. in contrast to joint mtl, twostage mtl methods optimize each target task independently, bypassing the issue of task interference @xcite . similarly to stl, a parameter-efficient module is first learned for each source task s with parameters θ s . in principle, twostage mtl methods can simply use already pretrained modules (such as adapters), saving the costs of re-training modules on each task. this facilitates the re-use of existing parameter-efficient modules for each source task, foot_1 which may vary in performance and/or take into account additional constraints such as fairness and bias mitigation @xcite @xcite . moreover, it also removes the need for accessing the training data of the source tasks (e.g., due to data privacy) so far as the source task's functionality is solely provided via parameter-efficient modules. next, given |s| (pre-trained and frozen) source task modules, two-stage mtl methods define and optimize a transfer layer for each target task to leverage the knowledge of source tasks to solve the target task. this stage introduces ω t new parameters for each target task t. adapterfusion @xcite introduces an implementation of the two-stage approach with strong performance @xcite . it uses an attention mechanism as its transfer layer, inserted into each lm layer after the source adapters. more specifically, given the output vector of each source adapter s in each layer l, referred to as o l s , the attention layer (with target task t as query and source tasks s as keys and values) learns to assign a weight ω l s to each source task. the final output of the target task t in this layer is calculated as: regardless of how the weights are calculated, the method can be seen as a weighted summation of source output vectors, where the weights form a categorical probability distribution. ## scalearn -learning to scale for knowledge transfer to understand the effect of scaling the output representations of adapters, we conducted initial experiments on scaling them, both in isolation and when combining two of them. in these experiments, we observed that (1) scaling output vectors is an effective method for controlling the (partial or full) activation of the knowledge contained in an adapter module; based on that, we present scalearn, a novel two-stage transfer learning method to combine the knowledge of source adapters by scaling their output representations. our core contribution regards the transfer layer, built on the output of the tasks' modular networks. similar to @xcite , we utilize adapter modules for the task learning layer. in particular, the output representation of the adapter of source task s at layer l is defined as: o l @xmath3, where x l s is the input vector, and u l s and d l s denote the up-and down-projection parameter matrices, respectively. our introduced scalearn linearly scales and combines the output representations of source adapters, o l 1 , . . . , o l |s| , to achieve the objective of target task t. we define two variations of the scaling operation: non-uniform which applies a scaling vector to each output vector using the element-wise product (scalearn), and the more parameter-efficient uniform that scales each vector only with a scalar parameter (scalearnuniform). these variations are formulated below: ## related work parameter-efficient task learning in nlp. various parameter-efficient methods have emerged as a more sustainable alternative to full fine-tuning, enabling modularization, efficient sharing, and reusability of knowledge. a common modularization approach is to introduce a small number of additional parameters into an lm, realized by various methods such as adapters @xcite , compacter @xcite , and propetl-adapter @xcite . similarly, lora @xcite injects trainable low-rank matrices into each transformer layer, and bitfit @xcite updates only the bias terms. another line of research identifies sparse subnetworks within the model to tune @xcite @xcite , while he et al. (2022) and mao et al. (2022) propose to merge various distinct modules. we refer to pfeiffer et al. (2023) for a full survey on this topic. learning by scaling. besides the common approach of learning a feed-forward layer for a (non-) linear transformation of an input vector, several recent methods explore the merit of learning a scaling vector applied to the input vector in various scenarios. liu et al. (2022) learn a modular network for stl that rescales lm vectors through elementwise multiplication. ilharco et al. (2023) and ortiz-jiménez et al. (2023) introduce task arithmetic to control lm behavior by extracting task vectors from pre-and post-fine-tuning model weights, then scaling and combining them to improve mtl performance. masoudian et al. (2024) learn a gating adapter that adjusts the scaling of representations to control the behavior of the model at inference time. finally, lian et al. (2022) learn to shift and scale the output vectors of a vision transformer in an stl setting. our work contributes to this line of research by leveraging scaling for highly parameter-efficient and effective mtl. joint mtl. interference and imbalance between tasks have been shown to impede performance in joint mtl @xcite @xcite . several studies have aimed to address these issues and improve generalization. for example, ( liu et al., 2019a) 2023 ) leverage continuous prompts learned on large-scale source tasks, leading to competitive performance in mtl benchmarks, although both methods depend on the selection of typically high-resource source tasks. in contrast to the mentioned methods that highly depend on the selection of tasks and/or apply the combination to the weights, @xcite combines the output representations of several independent source adapters through an attention mechanism. our work is directly related to this line of research and introduces a novel highly parameter-efficient transfer layer applied to the output representation. ## conclusion we propose scalearn, a highly parameterefficient and effective two-stage mtl method leveraging simple scaling of output vectors. our proposed approach directly learns the coefficients that scale the representations of source adapters and combines them simply by taking the sum. we conduct transfer learning experiments using encoder lms on the three benchmarks of glue, super-glue, and humset, consisting of a diverse set of tasks, domains, and languages. our results show that scalearn and even its extremely parameterefficient variants obtain strong improvement over existing mtl methods without any negative crosstask effects. we further show that these improvements are also present in few-shot transfer learning.