S har PT : Shared Latent Space Prompt Tuning (2023)

Abstract: Prompt tuning is an efficient method for adapting large language models, and Soft Prompt Transfer (SPoT) further narrows the gap between prompt tuning and full model tuning by transferring prompts learned from source tasks to target tasks. It is nevertheless difficult and expensive to identify the source task that provides optimal prompts. In this work, we propose to learn a shared latent space which captures a set of basis skills from a mixture of source tasks. Given an instance, its embedding queries the latent space, yielding a basis skill vector. This vector generates soft prompts, via a lightweight prompt generator, which modulates a frozen model. The latent space and prompt transformation are learned end-to-end by training on source tasks. Transfer learning from source tasks to a target task simply amounts to finetuning the prompt generator, accounting for roughly 0.3% parameters of the frozen backbone model, while the shared latent space is also frozen in finetuning. Our approach outperforms prior soft prompt methods by a significant margin on a variety of tasks such as NLI, sentence completion, QA, conference resolution, word sense disambiguation. We also find, on various model scales, our method achieves competitive performance compared to finetuning the full model.

## introduction adapting pre-trained large language models (llms) has advanced the progress in many nlp areas @xcite . this is typically done by finetuning all parameters of a model on a downstream task (i.e., @xcite . this approach is however expensive, especially given the growing sizes of sota llms. this limitation motivates recent research on parameter-efficient methods which only tune a small amount of parameters @xcite @xcite ; x , and then queries the skill latent space, resulting in a skill vector e spot @xcite improves over prompt-tuning by leveraging knowledge from source tasks. they first learn a task-specific soft prompt for each task in a set of source tasks. given a target task, they search over the set of source prompts and use the best one or some weighted combination to initialize the prompt for the target task and then tune the prompt. it further narrows the performance gap to modeltuning on smaller models. but it is complicated and expensive to identify the source task that provides optimal prompts. in this work, we propose a novel prompt-based transfer learning method, sharpt (shared latent space prompt tuning). figure 1 illustrates the general idea. sharpt assumes a shared (discrete) latent space by all source and target tasks. we call each vector in the latent space as a skill vector, since we assume each one captures a basis nlp capacity or skill after training on the source tasks. given an instance (from either a source task or a target task), an instance encoder embeds it into an instance vector, which is then used to query the latent space to find the nearest neighbor, yielding a skill vector for this instance. a lightweight prompt generator then generates soft prompts as a function of the selected skill vector. the soft prompts condition a frozen llm. the latent space and prompt generator are learned end-to-end on a mixture of source tasks. in target task training, the latent space is frozen and only the prompt generator is tuned. sharpt retains the key advantage of prior prompt methods, parameter-efficiency. it only updates approximately 0.1% to 0.3% parameters compared to modeltuning. different from prior methods, we add an instance encoder to encode each instance. the instance encoder is lightweight and frozen in all scenarios. sharpt and spot both exploit a generic idea, leveraging knowledge shared across tasks. the approaches to achieve this are however distinctly different. spot assumes task-to-task transfer based on task-level prompts and the knowledge is encoded in task prompts. it is not straightforward to identify a source prompt for a target task. they illustrated two approaches: (1) spot-oracle and (2) spot-retrieval. spot-oracle involves using oracle test labels and expensive search (e.g., 48 times more expensive than regular prompt tuning in their experiments). in spot-retrieval, they first tuned a task prompt for each source and target task independently and retrieved a prompt based on prompt similarity. note that the retrieval tun-ing is only for searching a source prompt, which is in addition to final prompt tuning on the target task. in contrast, sharpt assumes the knowledge is encoded in a shared latent space and utilizes instance-level prompts, which are generated based on latent vectors from the shared space. these designs make source-to-target transfer simple. we learn the shared latent space with all source tasks in a single training run. also, the tuning on the target task only requires a single run. given an instance from a target task, we use the instance embedding to identify a skill vector, learned from all source tasks, which is then transformed to soft prompts. in summary, we design an instance-promptbased method by learning a shared skill latent space. we apply sharpt to a diverse set of tasks covering diverse domains and task categories. we find that our method outperforms prior prompt-based methods and matches full-model-tuning across model scales. sharpt assumes there exists a discrete latent space, consisting of a set of skill vectors @xmath0. the soft prompt is a simple transformation of one of the skill vectors e i , that is, @xmath1. the transformation or prompt generator (f α ) is a light-weight mlp. in target task training aforementioned, e is known and fixed. we next specify how to learn it from source tasks. suppose we have n source tasks, {t (s) j } n @xmath2. we simply mix all tasks together, s) and its embedding e in summary, the forward pass for training on source and target tasks are exactly the same (also see figure 1 ). the only difference is the loss function, equation 5 (source) versus equation 3 (target). ## experiments high-to-low resource transfer in this setting, the target tasks are low-resource tasks (less than 10k training examples), while the source tasks are high-resource tasks. it consists of 25 tasks in total. there are 15 source tasks (e.g., docnli, drop) and 10 target asks (e.g., boolq, cola). please see appendix a for the complete list or transfer across different task categories we here investigate the transferability from datasets in some task categories to datasets in other heldout task categories. following @xcite , we assume datasets in each category measures a general nlp ability, and use the same taxonomy defined in @xcite . the source tasks include (1) qa tasks: record, squad, drop, multirc, and race; (2) sentiment analysis tasks: yelp-2 and sst-2; (3) a paraphrase detection task: qqp; (4) a semantic similarity task: cxc. the target tasks include (1) a sentence completion task: copa; (2) nli tasks: cb and rte; (3) a coreference resolution tasks: wsc; (4) a word sense disambiguation task: wic. training details as in prior works @xcite , all datasets are converted to a text-to-text format. all experiments are conducted with t5-base-lm-adapted as the backbone unless stated otherwise. we use a simcse @xcite model (bert-base) as the instance encoder. since the instance encoder is always frozen, we can pre-compute the embeddings of all instances and only keep the embeddings. however, we find that memory and time saved in this approach is negligible foot_1 . in source task training, the model (skill latent space and prompt generator) is simply tuned on the mixture of all source tasks for each setting. the model is tuned for 80k steps. in learning and testing on target tasks, we closely follow the procedure in @xcite . the model is tuned for 100k on each target task. we save a checkpoint every 500 steps and report results on the checkpoint with the highest validation performance. the prompt generator generates 64 soft tokens. the following hyperparameters are shared in all target and source task training: learning rate (0.3), the number of warmup steps (4000), optimizer (adam). ## conclusion we introduce sharpt, which learns a shared latent space which captures a set of basis nlp capacities from a mixture of source tasks. target instance queries this space to retrieve a skill vector, which then generates prompt tokens to condition a frozen llm. our approach outperforms prior soft prompt methods by a significant margin on a variety of tasks. our method also matches full-model-tuning across model scales.