Learning to Predict Task Transferability via Soft Prompt (2023)

Abstract: Fine-tuning pretrained language models on helpful intermediate tasks often greatly improves the performance of target tasks. However, how to efficiently find the source tasks that can successfully transfer still remains under-explored. In this work, we propose to learn an affinity scoring function to predict transferability between tasks. Specifically, we conduct prompt tuning and regard soft prompts as task embeddings that summarize task-specific information. Then we randomly sample task pairs to train an affinity scoring function. The goal is to predict the transfer gain (i.e., affinity) between a task pair, by conditioning on their task embeddings. Once the scoring function is trained, given a novel target task, we use it to predict the most transferable source tasks, without a brute-force search for all possible source-target pairs. Experimental results across 50 tasks show that our method efficiently identifies beneficial tasks for transfer learning.

## introduction fine-tuning pretrained language models (plms), such as bert @xcite and roberta @xcite , achieves remarkable performance on various natural language processing tasks. in addition, fine-tuning on intermediate source tasks yields further gains @xcite @xcite . while incorporating intermediate stages of knowledge transfer has shown compelling benefits on various tasks, choosing inappropriate source tasks results in negative transfer performance on the target tasks @xcite @xcite . previous work use data size, task complexity, or cosine similarity between tasks as well as domains @xcite @xcite to model the relationship between datasets. however, the conditions for successful transfer remain unclear @xcite . given a target task, which source task is the most helpful is not well investigated. how to effectively predict the transferability between tasks is still challenging. to shed light on the task relationship, in this paper, we conduct a comprehensive study of the transferability between 50 tasks, covering a wide range of text genres and degrees of difficulty. as fine-tuning and deploying a separate instance of the entire large model for each downstream task is prohibitively expensive, we conduct prompt tuning @xcite on a diverse set of source tasks. prompt tuning is a parameter-efficient approach which freezes all parameters of plms and merely tunes soft prompts @xcite . as prompts effectively simulate the knowledge of a large language model @xcite , we interpret soft prompts as task embeddings following @xcite and store task prompts in a prompt pool. then we randomly sample task prompt pairs from the pool, along with their transfer gain (i.e., the performance difference between "with transfer" and "without transfer") to learn an affinity scoring function. once the scoring function is trained, given a novel target task, we use it to predict the most beneficial source task in the prompt pool. using the affinity scoring function to predict the most helpful source task costs substantially less than using a brute-force search over all possible source-target pairs. experimental results show that the proposed affinity scoring function efficiently identifies beneficial tasks for transfer learning and obtains larger transfer gain on the target task than baseline methods. moreover, our method effectively predicts task transferability with much fewer supervision demands. our contributions are as follows: ## background: prompt tuning soft prompts are trainable continuous embeddings prepended to the input. during training, we keep the large pretrained language model frozen and only update the parameters of the prompt. we use soft prompts in our work since discrete prompts need carefully handcrafted design @xcite @xcite . even if significant effort is invested, discrete prompts are likely to be suboptimal @xcite . it is difficult and time-consuming to finding proper discrete prompts for each task @xcite . given a series of k input tokens, @xmath0, ..., w k }, where w i denotes i-th token and k denotes the input length. we first utilize plm to embed the tokens, forming a matrix @xmath1, where h is the dimension of the embedding space. following reparametrization methods @xcite which can lead to more stable optimization @xcite , we use a two-layer feed-forward neural network with parameter θ as our prompt encoder. then we use the prompt encoder to embed the randomly initialized prompt tokens, forming a matrix @xmath2, where l denotes the length of the prompt. we concatenate it to the embedded input as [p ; x] ∈ r (l+k)×h which then passes through the downstream model as normal while only parameters of the prompt encoder, i.e., θ, are updated. given the input embedding x and its label y, the model is updated by maximizing the likelihood: notice that we do not use verbalizers and prompt engineering in our experiments. ## methodology as shown in figure 1 , we first construct a prompt pool which consists of a wide range of source task prompts. then, we randomly sample task pairs from the pool to train a scoring function that learns affinity scores between tasks. once the affinity scoring function is learned, given a novel target task, we use it to effectively predict the most transferable source task from the pool. then we initialize the target prompt encoder with the best source prompt and conduct prompt tuning on the target task. ## conclusions in this paper, we conduct an extensive study of the transferability between 50 tasks and propose an affinity scoring function to predict transferability between tasks. we interpret soft prompts as task embeddings, and store task prompts in a prompt pool. the affinity scoring function is trained with task pairs sampled from the pool to learn to estimate task affinity. then given a novel target task, we use the affinity scoring function to predict helpful source tasks from the pool. experimental results show that the proposed method outperforms the baseline methods on task affinity prediction. moreover, our method effectively predicts task transferability with much fewer supervision demands.