Contrastive Demonstration Tuning for Pre-trained Language Models (2022)

Abstract: Pretrained language models can be effectively stimulated by textual prompts or demonstrations, especially in low-data scenarios. Recent works have focused on automatically searching discrete or continuous prompts or optimized verbalizers, yet studies for the demonstration are still limited. Concretely, the demonstration examples are crucial for an excellent final performance of prompt-tuning. In this paper, we propose a novel pluggable, extensible, and efficient approach named contrastive demonstration tuning, which is free of demonstration sampling. Furthermore, the proposed approach can be: (i) Plugged into any previous prompt-tuning approaches; (ii) Extended to widespread classification tasks with a large number of categories. Experimental results on 16 datasets illustrate that our method integrated with previous approaches LM-BFF and P-tuning can yield better performance. Code is available in https://github.com/zjunlp/PromptKG/tree/main/research/Demo-Tuning.

## introduction pre-trained language models (plms) have been applied to widespread natural language understanding and generation tasks, which are proven to obtain significant gains across benchmarks @xcite @xcite @xcite @xcite . one paradigm of plms is the pre-train-fine-tune, which has become the de facto standard for natural language processing (nlp), where task-specific objectives and additional parameters are leveraged in the tuning procedure. recently, the paradigm of the adaptation of plms has been shifting. a new finetuning methodology named prompt-tuning with a natural language prompt and a few demonstrations has made waves in the nlp community by proving astounding few-shot capabilities on myriad language understanding tasks. further studies try to mitigate the labour-intensive prompt engineering with discrete prompt searching @xcite or continuous prompt optimization @xcite @xcite . however, few studies have focused on the demonstration, which is an indispensable component in prompt-oriented methodologies. in previous studies, demonstrations are sampled examples in the training set. gpt-3's naive "incontext learning" paradigm picks up to 32 randomly sampled instances as demonstrations and directly concatenates them with the input sequence @xcite . since informative demonstrations are crucial for model performance, @xcite develop a refined strategy via sampling input pairs with similar examples, thereby providing the model with more discriminative comparisons. however, it is still not guaranteed to prioritize the most informative demonstrations as (1) the similarity-based sampling may obtain degraded demonstrations in different classes but have similar distances to the input; (2) the number of usable demonstrations is still bounded by the model's maximum input length. for example, as shown in figure 1 , the purple lines refer to the random sampling while the blue lines indicate similarity-based sampling. note that similaritybased sampling may obtain examples very similar to the input sequence. however, those sampled examples with different labels may tend to have a similar representation and thus confuse the discriminability of the model. moreover, for datasets with many classes, it is still non-trivial to concatenate all sampled demonstrations. those above-mentioned challenges hinder the applicability of demonstration in prompt-tuning. to address those issues, in this paper, we propose contrastive demonstration tuning (demotuning) for pre-trained language models. specifically, we leverage learnable continuous embeddings (e.g., one or two learnable tokens) as virtual demonstrations to relax the maximum number of categories. we concatenate those virtual demonstrations to the input sequence; thus, our approach can be extended to a wide variety of classification tasks with many categories. to optimize those continuous embeddings, we explore a simple contrastive framework without negative pairs @xcite since it is difficult to find an appropriate negative pair in semantic space for nlp. in each training batch, we randomly sample a real example and regard the virtual and real examples as positive pairs. with contrastive learning, we can obtain informative, optimized virtual demonstrations with more discriminative comparisons. we conduct extensive experiments on 16 nlp datasets. our contrastive demonstration tuning can yield better performance when integrated with previous prompt-based methods (e.g., lm-bff @xcite , p-tuning @xcite ). moreover, our approach can be applied to datasets with many categories and outperform baselines. note that our approach is model-agnostic and can be plugged into lots of prompt-based methods without the effort to select suitable demonstrations. the main contributions of this study are as follows: ## preliminaries in this work, we focus on classification tasks in the few-shot setting, including text classification and natural language understanding, where the input x in is either a sentence x @xmath0, x 2 ). here, we let prompt-based fine-tuning prompt-based finetuning @xcite ) is an efficient work by designing cloze-style template t and verbalizer m : @xmath1. template in prompt-based fine-tuning paradigm, template t is mainly comprised of inputs x in and a prompt @xmath2, where the prompt could be a series of discrete tokens @xcite or continual pseudo tokens @xcite . for instance, in the sentiment analysis task (see figure 2 ), a template with handcraft prompt may be: verbalizer a verbalizer m defines a mapping of label tokens from label space of a specific task. in figure 2a , the verbalizer maps "negative/postive" to "terrible/great". in this way, we could re-use the output weight w @xmath3. demonstration let d c train be the subset of all examples of class c. we sample demonstrations ## contrastive demonstration tuning in this work, we focus on how to learn a compact and differentiable virtual demonstration to serve as prompt augmentation instead of designing specific sampling strategies for demonstrationbased learning. we propose a learning framework based on a contrastive learning approach that can be compatible with the current prompt-based learning paradigm. this section introduces the concepts of contrastive demonstration tuning (demo-tuning) and provides details of this approach. next, we study how to obtain the optimal virtual demonstrations, which are initialized as a series of pseudo tokens at the start of fine-tuning. to address this challenging problem, we propose to use contrastive learning, which aims to obtain effective representation by pulling semantically close neighbors together. intuitively, we believe the optimal virtual demonstrations may be analogous with "prototype" @xcite , the representative for corresponding class, and we will discuss in §6. positive instances a key element of contrastive learning is how to construct reasonable x in , x + in pairs. here, we design a new template t + (x) based on template t (x) by randomly replacing one of virtual demonstrations [d (c) ] with real demonstration d c as shown in the figure 2b : in , i.e., t (x in ), t + (x in ) is a positive training instance. in this way, aligning virtual demonstration [d (c) ] with d c , the only difference between x in and x + in , and pulling representations (h in , h + in ) closer in semantic space could effectively alleviate the problem that the existing of terrible or irrelevant demonstration by previous sampling strategies. optimization similar to @xcite , we can randomly sample a minibatch of n examples from d train to construct positive pairs {(x i , x + i )} n @xmath4, x + i ): in this work, we also explore a simple contrastive framework without negative pairs foot_1 similar to recent non-contrastive self-supervised learning @xcite . regarding the difficulty to find a appropriate negative pair in semantic space for nlp, specially in few-shot setting, we only construct positive pairs and define the following mean squared error between h i and h + i with ℓ 2 -normalization, when supervised examples d train are available, the pre-trained language model could be fine-tuned to minimize the joint objective comprised of crossentropy and contrastive objective of eq. ( 4 ). in this way, during inference, we can concatenate the input x in with trained virtual demonstrations in template t (x), which does not need to sample real demonstrations. besides, we provide empirical analysis of negative sampling in §5.4. ## conclusion and future work in this work, we propose contrastive demonstration tuning, a simple model-agnostic approach for pre-trained language models, which improves stateof-the-art prompt-tuning performance without the necessity of demonstration selection. in the future, we plan to explore the following directions: 1) studying the connection between virtual demonstration and prototypes and theoretically analyzing the optimal solution of demonstration for prompttuning. 2) applying our work to more nlp tasks and trying to adapt to natural language generation.