Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding (2022)

Abstract: Prompt Tuning has been largely successful as a parameter-efficient method of conditioning large-scale pre-trained language models to perform downstream tasks. Thus far, soft prompt tuning learns a fixed set of task-specific continuous vectors, i.e., soft tokens that remain static across the task samples. A fixed prompt, however, may not generalize well to the diverse kinds of inputs the task comprises. In order to address this, we propose Vector-quantized Input-contextualized Prompts (VIP) as an extension to the soft prompt tuning framework. VIP particularly focuses on two aspects—contextual prompts that learns input-specific contextualization of the soft prompt tokens through a small-scale sentence encoder and quantized prompts that maps the contextualized prompts to a set of learnable codebook vectors through a Vector quantization network. On various language understanding tasks like SuperGLUE, QA, Relation classification, NER and NLI, VIP outperforms the soft prompt tuning (PT) baseline by an average margin of 1.19%. Further, our generalization studies show that VIP learns more robust prompt representations, surpassing PT by a margin of 0.6% - 5.3% on Out-of-domain QA and NLI tasks respectively, and by 0.75% on Multi-Task setup over 4 tasks spanning across 12 domains.

## introduction with the increase in network size, it has become more computationally expensive to fine-tune pretrained language models (plms) on downstream tasks. recent studies have shown that promptbased learning is a quite effective parameterefficient method of conditioning the behavior of plms for a given predictive task @xcite , 2021; @xcite . the goal of prompt-based learning, in contrast to traditional supervised learning approaches, is to find the task-specific template that is augmented to the input text in order to guide the plm toward generating the correct output @xcite . the initial efforts in this direction involved manually designed templates to solve natural language processing (nlp) tasks @xcite @xcite . in order to reduce the human effort required in finding appropriate task and lm-specific prompt templates, more recent works aim to automate the process of prompt designing. one of the popular ways is to search for templates in the discrete space of tokens @xcite @xcite . these are known as hard prompts. another popular line of research aims at learning prompt token embeddings in a continuous vector space, i.e., soft prompts @xcite . our work is an extension of the parameter-efficient soft prompt tuning framework, pt, which we describe below. given a frozen plm, the soft prompt tuning method pt @xcite aims to learn a set of soft tokens (vectors) to solve a downstream task. there is a critical limitation of this approach, that is, the learned prompt tokens are expected to perform well across the samples of the task. thus they are static against the change in input. henceforth, we refer to them as static tokens. we hypothesize that learning to attend and adapt the prompt tokens to the input provides more flexibility to the system to find the distribution of prompts that generalizes well to diverse forms of the input. to address the limitation of pt, i.e., inputinadaptability of static tokens, we introduce a novel soft prompt tuning technique, vector-quantized input-contextualized prompts (vip), with two integral properties that are our main contributions: while there has been some recent interest in input-dependent prompt generation @xcite @xcite , we observe that a straightway utilization of the contextualized tokens as a replacement for static tokens leads to performance degradation (vip-c in similar to the other prompting methods, the original input text is concatenated with the obtained quantized (contextual) prompt tokens and subsequently fed into the frozen plm. the contextualizer and static tokens are learned using backpropagation to minimize the task-specific prediction loss. whereas the codebook vectors are learned using a more stable update method of exponential moving average @xcite . empirically, we demonstrate the utility of our proposed approach vip on a wide range of nlp tasks. on superglue, qa, relation classification, ner and nli, vip improves over the baseline soft prompt tuning method pt by an average margin of 1.19%. additionally we find that vip outperforms pt by a margin of 0.6% -5.3% on out-of-domain qa and nli tasks respectively and by 0.75% on multi-task setup over 4 different tasks. overall, our experiments showcase that vip learns more richer robust prompt representations than static soft prompts pt. ## problem formulation following @xcite , we cast all the tasks into a text-to-text format. therefore, tasks such as text and relation classification are reformulated as text generation and the model is expected to generate the class label y. formally, given a pretrained language model (plm) parameterized by θ, we formulate a task t as conditional text generation pr θ (y | x) where x and y are respectively the sequences of token embeddings corresponding to the texts x and y, generated by the tokenizer provided by the frozen plm. the prompting approach of @xcite prepends x with the set of learnable soft tokens @xmath0, . . . , p n }. soft tokens p i are vectors that lie in a d-dimensional continuous space ir d . the idea is to condition the model output y directly on p and x as pr θ (y |p, x), where the static vectors in p are conditionally independent of x given task t i.e. pr(p | x, t) = pr(p | t). t denotes the set of tokens obtained from task-specific attributes such as task description and class labels. however, such a setting restricts the system to find a singular static set of prompt tokens that is effective for all the diverse samples of a given task. motivation. we hypothesize that to solve a complex language understanding task, the dependence of prompt on input x is a critical characteristic that aids in generalization over the unseen in and out-of-domain samples. in the parameter-efficient prompt tuning methods such as @xcite , @xcite is the soft prompt tuning which learns the vectors p i through backpropagation. the plm is not allowed to tune its parameters and learn task-specific contextualization. this motivates us to learn compact prompt representations in vip that can encode task-specific information and contextual information from each of its samples. we represent vip tokens by the set of vectors p @xmath1, . . . , p q n } ∈ ir d , which unlike static tokens p , holds the desirable property pr(p q |x, t ) ̸ = pr(p q |t ). ## methodology in order to obtain input-dependent soft prompts p q , we start with trainable static prompt tokens p similar to @xcite . p along with the original input token sequence x is then fed to a parameterized submodule called cq, which performs contextualization and quantization. formally, p @xmath2, x]) where ϕ is the parameter set of cq module. our final prompt token representation is obtained by combining the cq module's output p q and the static tokens p through a skip connection i.e. p + p q . this we substitute in place of p in pr θ (y |p, x) from @xcite pr(y |t, p, p q , x) @xmath3, p + p q , x]). (1) next, we elaborate on functioning of the cq module, which performs contextualization followed by quantization of the soft prompts and is the primary contribution of this work. ## experiments tasks. we base our experiments on a wide range of language understanding tasks including natural language inference, question answering of extractive and multiple choice nature, relation classification, and named entity recognition. we frame the same text-to-text format suitable for t5 as in @xcite . baselines. we consider the following baselines. to our idea of making prompt tokens dependent on the input @xcite . idpg cannot be directly treated as a baseline, since it uses roberta-large as plm and has only restricted applicability to text classification tasks, where it fine-tunes a classifier head over the frozen plm. instead, we adopt the idpg-style prompt generation with our backbone t5-base model and reduce the trainable parameters to be comparable to ours. this version, named vip-idp, fetches the input representation from a frozen t5-base encoder and applies two nonlinear projections -first to a low dimension and then upscaling to a space of dimension 768*n to generate n prompt tokens. one limitation is that this network requires two forward passes through the t5-encoder. in results tables 1 to 3 , bold numbers represent the best performing prompt-based model, as ft and adapter enjoy a much larger parameter size. for cross-domain qa, nli, and multi-task settings we compare vip against only primary baseline pt. ## discussions training-time and memory overhead. as compared to pt, we did not observe an adverse impact of vip on training time even with more trainable parameters, and notice a similar convergence pattern. the number of training steps required on superglue datasets in order (dataset, pt, vip) are-(wsc, 180, 90), (cb, 320, 300), (copa, 377, 442), (wic, 4080, 1530), (rte, 546, 760), (boolq, 5015, 2360), (race-m, 17500, 22000), (multirc, 5000, 6000), @xcite , where per-iteration time-overhead of vip is 0.07 seconds (1.96 (vip) vs 1.89 (pt)). thus, for wsc, cb, wic, boolq, vip converges faster. regarding memory overhead, vip (97.7mb) is slightly worse (3.3mb) than pt (94.4mb), with the main memory bottleneck being the common frozen t5-base plm (892mb). while codebook in vip increases the parameter size, it can be shared across multiple tasks, thus reducing the parameter overhead. in our multi-task experiment, vip with codebook shared across 3 tasks still outperforms the pt baseline. learning codebook for small datasets. even on small superglue datasets such as cb, copa, and wsc with only 200-400 training instances, vip performs an average 1.8% better than pt. furthermore for cb, the codebook visualization and interpretability analysis (appendix a.3, figure 4 ) shows that vip indeed learns meaningful representations even for small scale datasets. comparison with l2 regularization. while l2regularization can also limit the capacity of prompt space, quantization achieves a clustering over the contextualized prompts with the discrete latent variables (codebook vectors) being the clusters. this encourages the quantized prompts to have similar representations for semantically similar inputs, which can neither be achieved in vip-c nor through l2-regularization. experiments with l2regularization did not show any performance gain, in fact, it degraded the performance for a large regularization constant. on superglue datasets cb and copa, our pilot-study in the standard pt setting yielded these results in the form (l2reg constant, accuracy): for cb-(0,75.8), (1e-4,66.78), (1e-3,70.933), (1e-2,66.77), (1e-1,23.71). for copa-(0,59.0), (1e-4,53.99), (1e-3,59.20), (1e-2,54.34), (1e-1,45.24). lastly, learning the codebook can also lead to better interpretability of the prompt space (appendix a.3, figure 4 ). ## conclusion we propose a novel extension of soft prompt tuning (pt) -vector quantized input-contextualized prompt tuning (vip), designed to have two desirable characteristics -(i) contextualizing the soft prompt tokens w.r.t input text using a learnable sentence encoder (ii) discretizing the contextual prompts using a vector quantization network. on an extensive set of language understanding tasks -superglue, qa, nli, ner, and relation classification, vip outperforms pt baseline. further, our generalization studies on out-of-domain evaluations of qa and nli and multi-task settings over 4 tasks also show that vip is able to learn richer and more robust prompt representations than pt.