Hit the Nail on the Head: Parameter-Efficient Multi-task Tuning via Human Language Intervention (2024)

Abstract: Parameter-Efficient Fine-Tuning (PEFT) on small Pre-trained Language Models (PLMs) has emerged as a promising approach to enhance their multi-tasking capabilities. Prevalent methods simultaneously train additional modules (i.e., one task-shared module and multiple task-specific modules) for adapting PLMs to downstream tasks. However, their adaptability to new tasks is constrained, as the task-specific modules independently adapt to each task, overlooking the potential for knowledge transfer across tasks. In this paper, we propose a novel multi-task learning framework, Inspirational Pointer (IP), that enables the transfer of prior knowledge across tasks through human language intervention. Specifically, we attach task descriptions to the input samples, which are then mapped to corresponding task embeddings. Based on those embeddings, we adapt PLMs for downstream tasks. Similar tasks share akin descriptions, allowing new task samples close to similar trained tasks in the task embedding space, hitting the memory about trained tasks of the model. Our experiments on the T5 model demonstrate performance improvements of our method in multi-task learning and few-shot transfer learning. Further, we implemented the IP in decoder-only models including GPT2 and large language models (LLMs), and the results show that IP enhances the capabilities of decoder-only models.

## introduction the goal of multi-task learning is to equip a single model with the capability to address multiple tasks simultaneously. recent large language models (llms) have demonstrated outstanding performance in the field. however, deploying llms necessitates significant computational resources, which is inefficient for scenarios requiring rapid processing of massive amounts of data. consequently, enhancing the multitasking abilities of smaller pre-trained language models (plms), especially those operating on one consumer-grade gpu, is essential. recent studies indicate that parameter-efficient fine-tuning (peft) on small plms significantly improves their multitasking capabilities @xcite . the peft method freezes all parameters of plm and trains additional compact modules per task, allowing plm to adapt to multiple downstream tasks @xcite . recent developments in peft on samll plms for multi-task learning focus on a crucial challenge: effectively capturing both task-specific and shared information across tasks. a potential solution is the integration of peft modules, typically involving two training phases @xcite . initially, individual task-specific peft modules are trained separately. subsequently, an integration module is trained to consolidate those task-specific modules for downstream tasks. prior works show the effectiveness of this approach in multi-task learning and few-shot transfer learning. however, the additional integration training consumes more resources, which limits its practicality. an alternative solution provides a more stream-lined training process, which is called the hypernetwork-based method @xcite @xcite . it optimizes a hypernetwork and multiple task-specific embeddings from multi-task tuning. guided by distinct task embeddings, this hypernetwork generates parameters for various task-specific adapters, a kind of peft module. during training, the hypernetwork captures shared information across tasks, while each task embedding learns the task-specific information. this one-time training method achieves a better balance between resource consumption and performance. however, this method often shows limited transfer learning effectiveness, particularly in few-shot scenarios. this is due to two main reasons: (1) adapting to a new task requires training a task-specific embedding independently, without leveraging previously trained task embeddings. (2) the limited number of samples fails to provide enough task-specific information for task embedding. this paper primarily aims to improve the fewshot transfer learning in one-time training methods. we introduce the inspirational pointer (ip), a multi-task learning framework that leverages human language intervention as a knowledge transfer bridge between various tasks. as shown in figure 1 , it first incorporates task descriptions into input samples and then projects the modified data into the task embedding space. finally, these task embeddings are transformed into modification vectors, which are used to adapt the hidden states of plms for downstream tasks. due to similar tasks having comparable descriptions, this approach enables new samples to align closely with trained tasks in the task embedding space, thereby leveraging the existing knowledge of the model. additionally, task descriptions also offer task-specific prior knowledge to plms @xcite . furthermore, we moved away from the 'hypernetworkadapter' structure and explored more parameterefficient methods. specifically, we infuse taskspecific knowledge into the plm by directly performing a hadamard product between the modification vectors and the hidden states of the plm. previous work @xcite has demonstrated that the hadamard product can convey more information with fewer parameters, which leads to the parameter efficiency of ip (shown in figure 2 ). following previous works, we evaluated the performance of ip on t5 model @xcite with 13 nlp datasets. the results show that ip requires only a small number of samples to adapt to new tasks, and it also has considerable advantages in traditional multi-task learning performance. further, we implemented the ip in decoder-only models, and the results showed that ip enhances their capabilities (in appendix a.5). contributions can be listed as follows: ## conclusion this paper introduces the inspirational pointer (ip), a multi-task learning framework. ip modifies input samples by attaching task descriptions, then influences the plm hidden states based on the ask embeddings of these samples. the experimental results show that our method achieves superior performance, applicable in multi-task and few-shot transfer learning scenarios, and ip with various sizes plms also perform well. finally, the ablation study highlights the significance of task descriptions. for future work, we aim to explore more effective methods for constructing task descriptions.