Universal Self-Adaptive Prompting (2023)

Abstract: A hallmark of modern large language models (LLMs) is their impressive general zero-shot and few-shot abilities, often elicited through in-context learning (ICL) via prompting. However, while highly coveted and being the most general, zero-shot performances in LLMs are still typically weaker due to the lack of guidance and the difficulty of applying existing automatic prompt design methods in general tasks when ground-truth labels are unavailable. In this study, we address this by presenting Universal Self-Adaptive Prompting (USP), an automatic prompt design approach specifically tailored for zero-shot learning (while compatible with few-shot). Requiring only a small amount of unlabeled data and an inference-only LLM, USP is highly versatile: to achieve universal prompting, USP categorizes a possible NLP task into one of the three possible task types and then uses a corresponding selector to select the most suitable queries and zero-shot model-generated responses as pseudo-demonstrations, thereby generalizing ICL to the zero-shot setup in a fully automated way. We evaluate USP with PaLM and PaLM 2 models and demonstrate performances that are considerably stronger than standard zero-shot baselines and often comparable to or even superior to few-shot baselines across more than 40 natural language understanding, natural language generation, and reasoning tasks.

## introduction the recent advancements in large language models (llms) are among the most astonishing breakthroughs in artificial intelligence. the modern, massive attention-based @xcite llms not only surpass human and previous models in specific natural language processing tasks, but they have also demonstrated impressive general capabilities @xcite . indeed, thanks to both the scaling of llm sizes and advances in * work done during internship at google. training and fine-tuning techniques @xcite @xcite , one of the most prominent and impressive abilities of modern llms is their zero-shot generalizability handling diverse and sophisticated tasks, even if the models have not been explicitly trained on them. beyond zero-shot abilities, when a few demonstrations are available, the few-shot capabilities can take advantage of the information in them with in-context learning (icl) @xcite , leading to further improvements. such few-shot capabilities are often observed to improve as the llms scale @xcite . along with careful prompting, in many cases, llms can perform similarly to, or even better than, fine-tuning, even though the latter is both more computationally expensive (due to gradient back-propagation) and more data-intensive. as such, in many scenarios, prompt-based learning has drastically reduced the barrier to the use of even the most massive llms. notwithstanding the breakthroughs, many open questions remain. while the zero-shot performances of llms are highly valued and widely used as a key yardstick of llm capabilities @xcite , llms still often show weaker performances and/or larger performance fluctuations in the zero-shot setting because of the lack of guidance or readily-available template solutions. while many automatic prompting methods have been proposed (refer to §4 for details), few existing works target the zero-shot setup, and heuristic manual prompt design is still often heavily relied upon @xcite . on the other hand, even though the icl paradigm has reduced the cost of data collection and labeling considerably, given that modern llms are typically used for an extremely diverse set of tasks, obtaining even a small number of labeled examples per task can easily become expensive for many tasks. furthermore, in some tasks, obtaining even a few examples might require a non-trivial amount of human effort (e.g., summarization of long articles, translation of low-resource languages, and/or domain-specific question answering requiring research or expertise), or simply impossible for novel tasks that are only revealed at test time. to address this, we introduce usp (universal self-adaptive prompting) that specifically pushes the state-of-the-art with icl in zero-shot settings (while remaining compatible with fewshot) via pseudo-demonstrations (pseudo-demos) constructed from unlabeled queries and modelgenerated outputs. usp works with fully blackbox, inference-only llms, and the use of pseudodemos ensures that usp may operate entirely in the transductive zero-shot setup @xcite using only unlabeled data. this makes usp ex-tremely versatile, as unlabeled data is typically readily available via, e.g., continuous, on-the-fly collections of user queries. unlike alternative methods often requiring task knowledge beforehand (e.g., class names), usp requires only the task type information to select an appropriate confidencequantifying metric (e.g., natural language understanding (nlu) or generation (nlg) -these need to be known anyway), while still remaining capable of using additional information like class names if they are indeed available ( §3.3). this enables usp to work in arbitrary, potentially novel tasks at test time and/or tasks that simply cannot be cast as classification problems (e.g., open-domain qa and other generative tasks). usp is inspired by recent works leveraging confident predictions for model self-improvements on chain-of-thought tasks @xcite @xcite but inherits the benefits of these works and generalize them considerably in terms of the scope of applicability. to achieve this, we derive various criteria capable of selecting high-quality pseudo-demos in the absence of any ground-truth labels. to summarize: 1) we propose usp, a versatile and black-box automatic prompting method that can be zero-shot using only unlabelled data. ## preliminaries in-context learning (icl). icl enables llms to perform few-shot learning by processing several labeled, exemplary queries similar to the test queries we are interested in solving as demonstrations, or demos in the prompts @xcite @xcite (fig. 2b ). formally, denoting a test query as x and if we have k pairs of related concatenated queries and labels s (i) = concat(x (i) , y (i) ) ∀ @xmath0, ..., k} serving as demos, we augment the test query by prepending the demos (and instructions, if any) to it: icl is achieved by obtaining the prediction ŷ by querying c(x) instead of just x. in our zero-shot setup, none of the ground-truth labels (i.e., the ys) are available, and we propose to use the llm predictions themselves as pseudo-demos. thus, our zero-shot icl instead has the form of: ## related works besides those covered in §2, here we discuss other related works in zero-shot automatic prompting. we include an additional literature review in the app. a. autocot @xcite also uses modelgenerated output as pseudo-demos but differs in the selection procedure -it computes a sentence embedding of available queries and uses clustering to select the centroid queries as pseudo-demos. this process, unlike usp, is purely based on the query (dis)similarity rather than the output quality, and the quality of the selected pseudo-demos is thus, in expectation, the same as the average model performance -we empirically compare against a generalized version of it in §5, which is originally designed for reasoning tasks only (hence the name). another method, z-icl @xcite , generates pseudo-demos with synonyms of random class names. it, however, by assuming label knowledge, is limited to a subset of cls tasks where it is reasonable to do label synonym replacement. for example, while it is reasonable to replace simple sentiment-describing labels like "good" or "bad", the same may not be possible for factual labels or when labels are beyond single words (e.g., the race{h,m} examples shown in few-shot usp. on the bbh tasks on palm 2, we also test a few-shot variant of usp (termed uspfs) to generate additional pseudo-demos on top of scarce, manual demos. we show the results in fig. 9 in app. d, and uspfs outperforms both the zero-shot usp reported in fig. 3 and standard 7444 1-shot, thereby highlighting the generality of usp. ## conclusion we propose usp, a versatile, zero-shot automatic prompting technique applicable to a wide range of nlu, nlg, and reasoning tasks. we show large improvement over standard zero-shot prompting and other baselines in over 40 tasks with 3 llms.