ATTEMPT : Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts (2022)

Abstract: This work introduces a new multi-task, parameter-efficient language model (LM) tuning method that learns to transfer knowledge across different tasks via a mixture of soft prompts—small prefix embedding vectors pre-trained for different tasks. Our method, called ATTEMPT (ATTEntional Mixtures of Prompt Tuning), obtains source prompts as encodings of large-scale source tasks into a small number of parameters and trains an attention module to interpolate the source prompts and a newly initialized target prompt for every instance in the target task. During training, only the target task prompt and the attention weights, which are shared between tasks in multi-task training, are updated, while the original LM and source prompts are intact. ATTEMPT is highly parameter-efficient (e.g., updates 2,300 times fewer parameters than full fine-tuning), while it overcomes instability of prompt tuning and achieves high task performance using learned knowledge from high-resource tasks. Moreover, it is modular using pre-trained soft prompts, and can flexibly add or remove source prompts for effective knowledge transfer. Our experimental results across 21 diverse NLP datasets show that ATTEMPT significantly outperforms prompt tuning and outperforms or matches fully fine-tuned or other parameter-efficient tuning approaches that use 10 times more parameters. Finally, ATTEMPT outperforms previous work in few-shot learning settings.

## introduction fine-tuning all the parameters of large language models (lms) given target task training data is the most common practice for optimizing task performance @xcite . a recent line of research introduces parameterefficient tuning methods @xcite @xcite that only update a small number of lm parameters; however, increasing efficiency often decreases the task performance @xcite . moreover, these models are trained only using the task training data and do not benefit from large collection of other nlp tasks @xcite . we posit that parameter-efficient tuning methods can leverage rich knowledge of high-resource tasks to improve both training efficiency and task performance. this work introduces a new parameter-efficient, modular multi-task tuning method called attempt (attentional mixtures of prompt tuning, previewed in figure 1 ). attempt efficiently integrates knowledge from multiple tasks via a mixture of trainable soft prompts preprended to the input, keeping the original lm completely frozen. it first pre-trains transferable soft embeddings @xcite , called source prompts, on large-scale source tasks, which are likely to contain knowledge beneficial to other tasks. then, attempt initializes a new target prompt for a given target task and learns an attention-weighted combination of source prompts and the target prompt. the attention module is a light-weight network that can be shared and trained simultaneously across tasks. attempt offers three key advantages over previous multi-task fine-tuning or parameter-efficient tuning methods: first, it is highly parameterefficient and achieves competitive performance despite updating only 0.4% of the parameters in full fine-tuning. second, it enables modular multi-task learning using pre-trained soft prompts, where knowledge from different tasks can be flexibly combined, reused, or removed, and new tasks can be added to the lists of source or target tasks. unlike prior work that relies on precomputed priors on which tasks are related, attempt learns to focus on useful tasks from many source tasks. moreover, at inference, a single lm with multiple pre-loaded soft prompts can perform multiple tasks without parameter reloading. lastly, it improves interpretability on underlying task similarities in multi-task learning by generating attention distributions. we conduct experiments on 21 datasets across diverse tasks, domains and output formats. at-tempt significantly outperforms previous prompt tuning-based approaches and matches state-of-theart parameter-efficient transfer approaches or fully fine-tuned models that train orders of magnitude more parameters, especially on smaller datasets. attempt is also effective on few-shot domain adaptations (i.e., 4-32 shots). our analysis further shows that attempt is particularly parameter-efficient and competitive with larger backbone lms, where other parameterefficient transfer approaches shows rapid increases of the trainable parameters. our ablation studies suggest that learned attentions, multi-task learning and modular transfer from multiple tasks largely contribute to the performance improvements. the attention distributions show the underlying similarities among seemingly different tasks (e.g., entailment and paraphrase detection), indicating signal for effective knowledge transfer across tasks. ## background and problem setup we first enlist common paradigms in nlp for learning a target task, which differ in terms of available data and resources. we then describe our problem setup with respect to these paradigms. ## results we present main results in section 5.1 and fewshot domain transfer experiments on sampled tasks in section 5.2, demonstrating the effectiveness of attempt especially when the data is scarce. section 5.3 further provides set of analyses. ## conclusion we present a new parameter-effluent tuning method attempt, which learns to produce instance-wise prompts by interpolating multiple reusable soft prompts trained on source tasks and a new taskspecific prompt, while keeping the original lm frozen. our large-scale experiments demonstrate that attempt achieves a great trade-off between task performance and efficiency, introducing an interpretable and modular task transfer.