B ayesian Multi-Task Transfer Learning for Soft Prompt Tuning (2023)

Abstract: Prompt tuning, in which prompts are optimized to adapt large-scale pre-trained language models to downstream tasks instead of fine-tuning the full model parameters, has been shown to be particularly effective when the prompts are trained in the multi-task transfer learning setting. These methods generally involve individually training prompts for each source task and then aggregating them to provide the initialization of the prompt for the target task. However, this approach critically ignores the fact that some of the source tasks could be negatively or positively interfering with each other. We argue that when we extract knowledge from source tasks via training source prompts, we need to consider this correlation among source tasks for better transfer to target tasks. To this end, we propose a Bayesian approach where we work with the posterior distribution of prompts across source tasks. We obtain representative source prompts corresponding to the samples from the posterior utilizing Stein Variational Gradient Descent, which are then aggregated to constitute the initial target prompt. We show extensive experimental results on the standard benchmark NLP tasks, where our Bayesian multi-task transfer learning approach outperforms the state-of-the-art methods in many settings. Furthermore, our approach requires no auxiliary models other than the prompt itself, achieving high degree of parameter-efficiency.

## introduction large-scale pre-trained language models (plms) have been recently fine-tuned for various nlp tasks @xcite . due to the computational challenges of training the extensive parameters in plms, there is a growing focus on methods that efficiently tune fewer parameters @xcite . figure 1 : two key steps for bayesian multi-task prompt tuning (bmtpt) are illustrated. first, we merge the posterior distributions of each source task to form a global posterior distribution. this distribution is approximated using stein variational gradient descent (svgd), a particle-based variational inference method. finally, we adapt to the target task by using the derived posterior from the source tasks as a prior. one of the promising approaches is prompt tuning (pt, @xcite , where a few adaptable vectors are added as prompts to the input of the downstream task @xcite . pt freezes the plm model parameters and limits the learning to prompts, yet it achieves impressive performance. however, it is still challenging to achieve the same level of performance as the full fine-tuning, as well as to mitigate sensitivity to initialization @xcite . to address these challenges, recent works @xcite @xcite proposed to adopt multi-task transfer learning approach, where the prompt is trained from multiple source tasks to be applied to the target task. specifically, they train real-valued vectors for prompts (i.e. soft prompts) on source tasks and use them as the initialization of prompt for the target task. however, it is unclear whether aggregating such individually-trained prompts provides a reliable initialization point and fully harnesses the benefits of multi-task transfer learning. in this paper, we propose bayesian multi-task prompt tuning (bmtpt) as a practical yet effective solution to this challenge. unlike traditional methods of prompt tuning grounded in transfer learning, our approach engages with the posterior distribution of prompts across a multitude of source tasks. for the transference of knowledge gleaned from source tasks, we utilize the source prompts' posterior distribution as the prior for the designated target task. this bayesian method of transfer learning augments the conventional transfer learning framework, which primarily learns the initialization point of the target prompt from the source tasks. specifically, bmtpt employs stein variational gradient descent (svgd, liu and wang 2016), a particle-based variational inference (vi) method, to approximate the source prompts' posterior distribution. further elaboration on this method is provided in section 2.2. we validate our approach through experiments on 21 datasets across diverse nlp tasks and output formats. the experimental results demonstrate that bmtpt achieves comparable or superior performance to strong state-of-the-art parameter-efficient fine-tuning methods @xcite as well as full fine-tuning, while utilizing a very small number of parameters and requiring no auxiliary models other than the prompt itself. ## problem setting in this section, we formally introduce core elements, symbols, and problem statements that form the basis of our approach. we denote the trainable parameter of the soft prompt as θ ∈ r l×d , characterized by its length l and the dimension d of the language model (lm). for clarity, we use θ s and θ t to denote the soft prompts for source tasks and target task(s), respectively. this implies that the soft prompt θ is prepended to the sequence x, prior to its processing by the lm. the underlying objective is to predict the target sequence y. we denote the dataset for the k-th source as d s k , and define the target task is represented as d t during task adaptation. thus, the i-th instance in the d s k dataset will be represented as (x k i , y k i ). note that the log-likelihood log p(d s k |θ s ) for the k-th source task can be represented as follows: in this formulation, p lm denotes the likelihood determined by the lm and the corresponding criterion. next we state our bayesian objective, aiming to optimize the target task prompt using the posterior of source prompts for a transfer learning scheme. problem statement. the objective is to maximize the posterior probability of the target prompt θ t , as expressed by the following equation: in this context, the prior distribution p(θ t | d s ) serves as a guide for the target task adaptation. we model p(θ t |θ s ) as the multivariate gaussian with mean θ s , since without any information on the target task, it is natural to have θ @xmath0. this problem formulation provides a general framework subsuming conventional transfer learning method for prompt tuning. for example, we could approximate the above integral in eq. ( 2 ) defining the prior on θ t using a prompt trained from source tasks θ s * , i.e. p(θ s |d s ) =δ θ s * (θ s ), which would be roughly equivalent to the conventional transfer learning setting where the source prompt serves as the initialization of the target prompt. assuming an uninformative prior for the source prompt θ s (e.g. uniform distribution) as well as independent selection of source tasks, the posterior distribution p(θ s |d s ) for source tasks is formulated as the product of the posteriors of each task. remark. assuming the uniform prior for θ s and independent selection of source tasks, the global posterior p(θ s | d s ) is proportional to the product of posteriors: ## approach instead of optimizing individual prompts for each source task in isolation, our method primarily revolves around learning the posterior distribution of source prompts across all source tasks. this approach assigns a larger probability mass to those prompts capable of addressing a greater number of source tasks, thereby potentially becoming more suitable candidate prompts for the target task as well. we implement this concept by using particles to approximate the posterior distribution. the following subsections provide a detailed explanation of this methodology. ## results in section 6.1, we provide the main findings on glue and superglue benchmarks. for findings on mrqa and "others" benchmarks, please refer to appendix d. in section 6.2, we further provide a set of analyses. ## conclusion we present bayesian multi-task prompt tuning (bmtpt), a bayesian approach for transferring soft prompt. our method defines a posterior distribution over prompt on source tasks, and approximates the posterior using svgd, then initializes the target prompt with aggregation of source prompts while regularizing the training of the target prompt using transferred posterior. empirically we found this approach achieves comparable or superior performance over strong parameter-efficient fine-tuning baselines. limitations while showing compelling experimental results with only the use of a soft prompt, bmtpt has its limitations. primarily, appending the soft prompt to the input text leads to an extension in the overall input length, consequently increasing the memory footprint. this is a well-known issue in pt (karimi mahabadi et al., 2021a), and bmtpt is not immune to it. furthermore, in bmtpt, since multiple particles are used and source task sentences are appended to each, the batch size grows to be a multiple of the number of particles. this could potentially increase memory demands during the source posterior learning. however, this can be mitigated by implementing source task sampling or reducing the number of particles. experiments determining the optimal number of particles have not been performed in our study, and future research could potentially explore this aspect to ascertain the most appropriate number of particles.