Prototype-based H yper A dapter for Sample-Efficient Multi-task Tuning (2023)

Abstract: Parameter-efficient fine-tuning (PEFT) has shown its effectiveness in adapting the pre-trained language models to downstream tasks while only updating a small number of parameters. Despite the success, most existing methods independently adapt to each task without considering knowledge transfer between tasks and are limited to low-data regimes. To overcome this issue, we propose Prototype-based HyperAdapter (PHA), a novel framework built on the adapter-tuning and hypernetwork. It introduces an instance-dense retriever and a prototypical hypernetwork to generate the conditional modules in a sample-efficient manner. This leads to comparable performance improvements against existing PEFT methods on multi-task learning and few-shot transfer learning. More importantly, when the available data size gets smaller, our method outperforms other strong baselines by a large margin. Based on our extensive empirical experiments across various datasets, we demonstrate that PHA strikes a better trade-off between trainable parameters, accuracy on stream tasks, and sample efficiency. Our code is publicly available at https://github.com/Bumble666/PHA

## introduction fine-tuning a pre-trained language model (plm) yields extraordinary potential for simultaneous adaptation to multiple downstream tasks in a multitask setting. however, fine-tuning all the parameters of models induces substantial storage and deployment costs, especially as pre-trained model sizes are growing rapidly. to address this issue, several works @xcite @xcite @xcite @xcite @xcite have developed parameterefficient fine-tuning which trains compact modules * equal technical contribution, co-first authors. † corresponding authors. per task and adapts plms to downstream tasks. nonetheless, these methods require learning different modules to adapt to diverse tasks, and the cost of the parameter increases proportionally with the number of tasks. on the other hand, training taskspecific modules separately fails to reap benefits from other relative tasks. recent work @xcite has proposed training a hypernetwork to generate the parameters of these modules to achieve a better trade-off between parameter efficiency and adaption for downstream tasks. these methods encourage the multi-task learning model to capture the shared information by leveraging task-shared hypernetwork while eliminating negative task interference by generating con-ditioned modules individually. despite these methods' success in multi-task learning, there are still some issues:(1) hypernetwork-based methods generally optimize the specific embedding and shared hypernetwork together by end-to-end training without any regularization. task-specific information is inseparably intertwined, which suppresses the efficiency of the hypernetwork, especially in resourcelimited settings. (2) these existing approaches generalize to new tasks that require task-specific prior knowledge or knowledge from frozen pre-trained models. these works @xcite indicate that the task-shared hypernetwork serves as a cross-task information captor, while a specific embedding should encapsulate task-level semantic features in order to extract pertinent information from the hypernetwork for generating corresponding module parameters. empirically, task-level features are typically implicitly represented by related instance features. a natural idea to encourage embedding generation is to calculate the central points (prototypes) of task-specific instance features. in this paper, we introduce the prototype-based hyperadapter(pha), a novel framework built on adapter-tuning that achieves both multi-task learning and generalization to new tasks in a sampleefficient manner. as depicted in figure 1 , pha consists of two main components, instance-dense retriever and prototypical hypernetworks. the first part aims to train a retriever to discriminate the instances from different tasks in embedding space. for the second part, we aim to estimate the task-specific prototypes with the instance-level features and keep the prototypes as embeddings to be trained with the hypernetwork. specifically, we project the encoded instance features into embedding space using the retriever. to avoid instances interference in embedding space, we train the retriever with the infonce estimator @xcite . as a result, it clusters intratask instances and increases the distances between inter-task instances. the projected features here can be deemed as instance-level semantic features that are used to estimate task-level embeddings. inspired by pcl @xcite , we estimate the task-specific embedding using the contrastive prototypical loss, which encourages the prototypes to become the center points of instance-level features. compared with the existing method, where the specific embeddings are optimized directly during tuning, our method efficiently learns specific embedding with side information, which helps to optimize the embedding space in low-data regimes. during the adaptation of new tasks, since we maintain the previous task-level semantic features as prototypes that align with the instances in the embedding space, we match the corresponding prototype for the current new task by calculating the distance between the new instances and the previous prototypes. we evaluate pha on 13 nlp datasets across diverse tasks. extensive experiments show the effectiveness of pha, especially in low-data regimes. meanwhile, pha is able to achieve a few-shot domain adaption with 4-32 shots. for example, pha outperforms the strong multitask adapter transfer baseline by 1.0% with lower trainable parameters on the glue benchmark. in low resource regimes where only 100 samples per task from the glue benchmark is available, pha outperforms adaptertuning by 8.0%. our analysis shows that pha efficiently captures specific information and shared information while reducing negative transfer. we also present a detailed analysis to demonstrate that the instances from different tasks can be identified by corresponding prototypes and used for new task adaption. ## background hypernetworks. a hypernetwork @xcite can generate parameters to be used by networks or modules. specifically, the hypernetwork, denoted as h w , leverages an embedding i to generate the module parameters ϕ: adapter. adapter @xcite is commonly used in parameter-efficient tuning that aims to apply plm to downstream tasks. specifically, adapter-based tuning inserts trainable task-specific modules into transformer layers, while keeping the plm fixed. the adapter a l (x) for layer l is defined as: ## conclusion we introduce prototype-based hyperadapter (pha), a novel framework built on adapter-tuning. pha achieves both multi-task adaption and adaption to a new task in a sample-efficient manner. it generates parameters of adapter layers conditioned on task-specific prototypes which are calculated by the corresponding instance-level features. in addition, the specific prototype is retrieved and transferred to new tasks to be further tuned. the resulting method significantly outperforms previous sota on full/low-data multi-task adaption and few-shot adaption.