Efficiently Enhancing Zero-Shot Performance of Instruction Following Model via Retrieval of Soft Prompt (2023)

Abstract: Enhancing the zero-shot performance of instruction-following models requires heavy computation, either by scaling the total number of training datasets or the model size. In this work, we explore how retrieval of soft prompts obtained through prompt tuning can efficiently assist hard prompts in zero-shot task generalization. Specifically, we train soft prompt embeddings for each prompt through prompt tuning, store the samples of the training instances mapped with the prompt embeddings, and retrieve the corresponding prompt embedding of the training instance closest to the query instance during inference. While only adding 0.007% additional parameters, retrieval of soft prompt enhances the performance of T0 on unseen tasks by outperforming it on 10 out of 11 datasets as well as improving the mean accuracy of T0 on BIG-bench benchmark by 2.39% points. Also, we report an interesting finding that retrieving source embeddings trained on similar answer choice formats is more important than those on similar task types.

## introduction training large language models (llms) on huge amounts of data has enabled lms to perform downstream tasks without any fine-tuning with the aid of natural prompts or concatenation of a few demonstration instances @xcite @xcite . additionally, recent works have shown that adding a instruction tuning stage, an additional training step that helps pretrained lms understand prompts and demonstrations results in a significant performance boost on zero-shot task generalization even for moderate-sized lms @xcite @xcite @xcite . this extra 1 model checkpoints and code implementation are available at github.com/seonghyeonye/rospr. dense retriever mips keys values source prompt library instance 1 instance 2 instance 3 ... instance x ti output t0 embedding e tm ## method in this section, we introduce retrieval of prompt tuning (rospr) for zero-shot task generalization. a detailed overview is shown in figure 2 . we first train source prompt embeddings of lm for each hard prompt given a source task using prompt tuning (section 3.1). then, we save training instance samples along with their prompt embeddings in the source prompt library and use it to retrieve embeddings at inference to perform tasks in a zero-shot manner (section 3.2). we additionally introduce interpolation of multiple source prompt embeddings (section 3.3) and variance-based ranking (section 3.4) to increase robustness and accuracy. ## experimental settings in this section, we explain the experimental settings of training of source prompt embedding and construction of our source prompt library. we also explain our evaluation setting during zero-shot inference and baseline models. we provide detailed experiment configurations in appendix f. ## experimental results rospr also outperforms finetuning baselines even without utilizing any training instances of the target task. we first observe that pt harms the performance of the backbone model, which aligns with the result of @xcite ; @xcite that prompt tuning is unstable when the training instances or the training steps are small. by comparing attempt with rospr, rospr outperforms attempt on 7 out of 10 tasks and 1.21% points on the mean accuracy of 10 tasks. this shows that rospr is more applicable for efficient adaptation because it requires 3 times fewer additional parameters compared to attempt and also does not require any further fine-tuning of the target task. ## analysis of rospr zero-shot task adaptation of lms is often seen as a problem of task location, locating the target task to where the model can solve it using the intrinsic ability obtained at pretraining stage with the aid of prompts and demonstrations @xcite . in this section, we analyze which factors contribute to the performance enhancement in the perspective of identifying better task location. we find that although the target task performance depends on the source task types, heuristic features such as the answer choice format are more important. this agrees with previous findings that a metatrained lm focuses on simple features such as the label space, the input distribution, and sequence format, instead of complex semantics @xcite . ## ablation studies in this section, we analyze the effect of the number of (1) prompts, (2) source datasets, and (3) queries sampled for evaluation. we evaluate variations of our proposed methods on 4 datasets: rte for nli, copa for sentence completion, winogrande for coreference, and wic for word sense disambiguation task. we report the average of the mean accuracy of all evaluation prompts for each dataset by running 3 different runs. scaling number of prompts vs. number of datasets. recent works on meta-trained lms show that the number of source datasets and prompts is an important factor for zero-shot task generalization @xcite @xcite . we also show ablations for rospr and measure how the zero-shot generalization performance changes when we vary the number of prompts and datasets available during the prompt tuning stage (shown in figure 7a ). first, we vary (1) the total number of source prompts by 60, 120, and 230 by increasing the number of prompts per dataset and (2) the number of datasets by 8, 16, and 30 by increasing the number of datasets per task cluster. 9 note that in (1), the total number of datasets is fixed while in (2), we use all available prompts for each dataset while varying the number of datasets per task cluster. in contrast to ( 1 ), (2) does not always lead to a linearly increasing performance boost; the performance saturates as more source datasets are included. by comparing the effect of scaling datasets and scaling prompts for similar source prompt library sizes, we observe that the number of prompts has more impact on the accuracy of the target task (figure 7a ). this ablation study also supports the analysis of the previous section; diverse answer choice formats of prompts, which are mostly influenced by the total number of source prompts, are more important than source task types which are influenced by the number of source datasets. 10 therefore, if 9 task cluster is defined as a cluster of the same task types. 10 although the total number of prompts also increases as the number of task clusters is sufficient to some extent, scaling the number of source prompts per dataset is more crucial than scaling the number of source datasets per task cluster. ## conclusion in this paper, we introduce rospr, a method that efficiently enhances zero-shot generalization capabilities of a meta-trained lm by retrieving promptspecific source prompt embeddings (soft prompts) for a given target task. we accomplish this by first training the soft prompts for hard prompt of the source tasks. after training source prompt embeddings, we construct the source prompt library by storing the mean representation of training instances as keys and the corresponding prompt embeddings as values. at inference, we search for training instances stored in the library similar to sample instances from the target task, retrieve the corresponding prompt embedding, select the most frequently retrieved embedding, and append it to each of the target task instances for prediction. our results show that rospr efficiently enhances the zero-shot performance of the backbone model while introducing minimal additional parameters during inference. we additionally provide analysis of which factors attribute to the performance of rospr and find that heuristic cues such as the answer choice format are critical for generalization performance, implying that it may play a role similar to demonstrations in in-context learning.