Two Examples are Better than One: Context Regularization for Gradient-based Prompt Tuning (2023)

Abstract: Prompting has gained tremendous attention as an efficient method for the adaptation of large-scale language models. However, prompts often act against human intuition and report unstable performances, which has motivated methods that automatically find effective prompts. One popular approach is gradient-based search, which iteratively updates a (randomly) initialized prompt towards the optimal one with the guide of gradients. We propose a novel regularization method, CoRe, for gradient-based prompt tuning techniques, which guides a prompt to produce a task context properly. CoRe realizes two regularization effects — context attuning and context filtering — that improve prediction performance in a zero-shot in-context learning setting where a model makes inferences only with the prompt tuned by CoRe, without any demonstration examples for in-context learning. Context attuning guides the context generated by the input and the tuned prompt toward embedding the appropriate context for the task. In our theoretical analysis, regularizing the context extends to improving zero-shot in-context learning performance. Context filtering steers the prompt to select only the task-related context so that context attuning solely focuses on creating and sending the right task context. We evaluate CoRe on natural language understanding datasets and two large language models, GPT2-XL and GPT-J.Our training scheme shows performance improvements up to 11.9% on GPT2-XL, and up to 6.3% on GPT-J in zero-shot settings.

## introduction a prompt is a carefully composed input to adapt a pretrained language model (lm) with in-context learning, and has gained attention as a new method for the adaptation of lms. @xcite and @xcite showed that pretrained lms can be transferred to various downstream nlp tasks by designing prompts to be aligned with pretraining objectives and downstream tasks. this method has a unique capability in that it does not mandatorily require training examples or parameter access for lm adaptation. moreover, @xcite suggested, and @xcite systematically explored that providing demonstration examples in the prompt can also boost the performance of prompting. initially, a prompt is manually crafted by a developer with human intuition. however, because of the unstable performances and unpredictable behaviors of manual prompts @xcite , automatic methods for prompt tuning have been proposed. one of the most popular paradigms is optimizing prompts using stochastic gradient descent (sgd) @xcite @xcite -gradientbased prompt tuning, which updates prompt tokens or parameterized embeddings with a guide of gradients. to further boost the performance of gradientbased prompt tuning methods, this paper suggests a new context regularization scheme, core. core introduces two regularizers: context attuning and context filtering. context attuning guides a prompt ω and an example {x, y} to generate an appropri-ate task context without being biased towards the example. this is realized as a concatenation of multiple examples, resulting in a single sequence including the examples, as the concatenation of the three examples in figure 1 . context attuning then optimizes the first prompt (ω that comes before x 0 ) towards minimizing the losses of the succeeding examples (y 1 , y 2 ). therefore, the prompt does not become biased to {x 0 , y 0 }, but is generalized for the appended examples as well and generates less biased task contexts. however, context attuning alone does not show the effect of regularization because the following examples receive too diverse contexts, such as styles, topics, relationships between entities, or task information. such diversity hinders the optimization of the regularizer towards the task context. to resolve this problem, we add another regularizer, context filtering, to adequately extract the task context from the preceding examples. specifically, following the example in figure 1 , context filtering (blue line) optimizes prompts given preceding examples -optimization of ω 1 with respect to y 1 when {x 0 , y 0 , x 1 } is given. we evaluate the effect of core in a zero-shot in-context learning (icl) setting as most prompt tuning methods have focused on the setting. in zero-shot icl, a tuned prompt and a test input is only given without any additional demonstration examples. while icl often assumes demonstration examples are given, for simplicity we call this setting zero-shot icl. it can be also seen as a typical zero-shot instruction following. context attuning and context filtering improve the model's performance in a zero-shot in-context learning (icl) setting even though the prompts have been trained in a few-shot icl setting (i.e., concatenation of examples). we conjecture that this successful transfer from a few-shot setting to a zero-shot setting is possible since prompts have been tuned with the guide of core to generate a more generalized task context. such a generalized task context is effective regardless of concatenated examples, thus improving the performance in a zero-shot setting. we highlight that this transferability from few-shot settings to zero-shot settings is noteworthy and verify it on a theoretical framework based on the hidden markov model and bayesian inference @xcite . to the best of our knowledge, this work is the first to leverage interactions between examples in a sequence for the purpose of regularization. there are several works training on multiple examples in a sequence @xcite @xcite . they focused on improving few-shot in-context learning by maximizing the likelihood of an example given demonstration examples. our work is distinct from the above works in two aspects. first, we target zero-shot settings. second, we introduce a novel regularization technique -context attuning-to improve zero-shot performance by leveraging the influence between multiple examples within a sequence. we evaluate core on two large autoregressive language models, gpt2-xl @xcite and gpt-j @xcite , and 8 nlu datasets: five from superglue, sst2, agnews, snli. core enhances the performances of zero-shot inference up to 11.9% on gpt2-xl and 6.3% on gpt-j when applied to p-tuning @xcite , prefix-tuning (li and liang, 2021), and softprompt-tuning @xcite . ## background in-context learning (icl) is a relatively new paradigm in the adaptation of pretrained language models (lms). icl have been used to adapt pretrained lms to numerous downstream tasks such as natural language inference @xcite ), text classification @xcite @xcite , question answering @xcite @xcite and many more. in icl, a prompt is a task specification given as part of the input and aims to guide the language model into generating the desired output according to the given task. a prompt usually consists of a task description, a template of the input with placeholders for data, and optionally demonstration examples. for example, when given an input "translate english to spanish: i love @xmath0, the lm is expected to finish the input by generating the answer "te amo". this is different from fine-tuning, where the parameters of the lm are updated to optimize the translation performance. compared to fine-tuning, which is a representative lm adaptation technique, icl is more memory-efficient when we want the lm to process various different tasks. in icl, the lm parameters are typically frozen, and developers only have to keep at most dozens of tokens per task, which are task descriptions and input templates. however, for fine-tuning, developers have to keep different versions of parameters per task, which can be as huge as 540 billion parameters @xcite in the status quo. ## approach our key approach to improve sgd-based prompt tuning is to concatenate multiple examples and regularize a context generated by them, which transfers to improvement in zero-shot prediction. our training scheme first concatenates multiple examples from the same task. taking an example of concatenating three examples (x 0 , y 0 , x 1 , y 1 , x 2 , y 2 ) as depicted in figure 1 , a model is given an input @xmath1, x 0 , y 0 , ω, x 1 , y 1 , ω, x 2 , y 2 } where ω is the tunable prompt. then, core introduces two regularizers: context attuning and context filtering. the context attuning regularizer steers an example to send the correct task context to the succeeding examples, while the context filtering regularizer guides the succeeding examples to receive only the task context among diverse contexts from preceding examples. we target zero-shot icl where only a prompt and a test input are provided (i.e., no demonstration examples). during inference, a model is given ω, x i and expected to predict y i where ω has been trained with core. we denote this setting as zeroshot in-context learning to imply that the model learns from the context provided by the tuned prompt, but we do not give any demonstration examples (zero-shot example for in-context learning). please note that this differs from the typical zeroshot setting. although the model still does not see any training example for making predictions, training data has been used when optimizing the prompt with core. we would like to highlight our finding that prompts tuned with core in a few-shot icl setting show improved performances in a zero-shot setting. core's two regularizers -context attuning and context filtering-are applied on a concatenation of multiple examples (i.e., few-shot icl), and the regularizers steer the prompt during train-ing time to generate a more generalized task context for following examples (x 1 , x 2 ) in a sequence. such a well-generated task context (generated by ω) is propagated not only to the following examples (x 1 , x 2 ) but also to the example (x 0 ) paired with the prompt. therefore, a prompt tuned with core helps the model's prediction when given an input of a single example (@xmath2, x i }), which is equivalent to a zero-shot icl setting. throughout this section, we use a simple form of prompt for conciseness of the explanation but without loss of generality: {ω, x i , y i } where ω is a tunable prompt. our work focuses on autoregressive language models (lms), so our method is not explored or analyzed upon autoencoding lms such as bert @xcite . ## experimental setup in this section, we briefly specify the setup that our experiments are conducted on: models, datasets, and hyperparameters. for more details, please check appendix b. ## experimental results we evaluate core on the setup we presented in section 4. we first show the performance gain of our method on the three baselines. then, we present how the performance changes according to sequence sizes, and further analyze the effect of two regularizers of core with the ablation studies. in addition, we present how the similarity of concatenated examples affects performance. finally, we present the performance of core in few-shot inference settings. ## conclusion core is a novel gradient-based prompt tuning method that regularizes task contexts among multiple examples, which finally regularizes a prompt to improve zero-shot performance. two regularizers of core, context attuning and context filtering, regularize the prompt to create proper task context and guides the prompt to convey only the task-related context to succeeding examples on the concatenation of multiple examples. we provide a theoretical analysis for the effect of context attuning and context filtering and our experimental results back this up. following the theoretical analysis, core achieves performance gain over three different baseline prompt tuning methods in zeroshot setting up to 11.9% on gpt2-xl and 6.3% on gpt-j. it implies that core can serve as an effective and memory-efficient adaptation method for hyper-scale lms.