Contrastive Learning for Prompt-based Few-shot Language Learners (2022)

Abstract: The impressive performance of GPT-3 using natural language prompts and in-context learning has inspired work on better fine-tuning of moderately-sized models under this paradigm. Following this line of work, we present a contrastive learning framework that clusters inputs from the same class for better generality of models trained with only limited examples. Specifically, we propose a supervised contrastive framework that clusters inputs from the same class under different augmented “views” and repel the ones from different classes. We create different “views” of an example by appending it with different language prompts and contextual demonstrations. Combining a contrastive loss with the standard masked language modeling (MLM) loss in prompt-based few-shot learners, the experimental results show that our method can improve over the state-of-the-art methods in a diverse set of 15 language tasks. Our framework makes minimal assumptions on the task or the base model, and can be applied to many recent methods with little modification.

## introduction the prompt-based fine-tuning method reduces the gap between pre-training and fine-tuning by forming the fine-tuning task into a masking language problem. a language prompt is a piece of text appended to the query input enabling the model to come up with better predictions @xcite . for instance, by feeding a language model with "the story is not worth reading, a truly one.", the model assigns a higher probability for the blank to be filled with "terrible" than "great". here, "a truly one." is called the template of the prompt and "terrible" or "great" is the label word. recently, lm-bff @xcite shows that appending demonstrations (e.g."this is an amazing movie, a truly great one") to inputs can help the model to better understand the label word, leading to further improved results. in this work, we show that supervised contrastive learning (supcon) @xcite at the feature space can be beneficial during the finetuning of prompt-based few-shot language learners, with proper data augmentation. data augmentation is the key component of supcon. while there exists many augmentation techniques like cutmix @xcite , mixup @xcite in computer vision and eda @xcite , aeda @xcite for text, data augmentation remains challenging. however, prompt-based few-shot learners with demonstrations actually provide us with a natural way to create multiple "views" (augmentations) of a single example, i.e., for a fixed set of label words, we can sample different templates and different demonstrations to append to the input text (shown in figure 1 ). this allows us to construct diverse input texts that are consistent and complete. by applying supcon to cluster the above two example inputs with very different contents but the same label, our method is able to obtain an additional supervision at the feature space which is crucial if we are only given a few labeled examples. the main contributions of our paper are: the story is not worth reading, a truly @xcite one. this is an amazing movie, a t r u l y g r e a t o n e . t h e c l a s s h a s n o attendance, a truly terrible one. prof. ronaldo won nobel prize, it is [mask]. superb performance by messi, it is great. kobe passed away, it is terrible. hidden state for [mask] primary prompt auxiliary prompts … great terrible … √ … great terrible … √ sent_0 sent_1 the story is not worth reading, i think it is [mask] .the flight is comfortable, i think it is great. i didn't go to the show, i think it is terrible. sent_2 the story is not worth reading, i think it is [mask] .the flight is comfortable, i think it is great. i didn't go to the show, i think it is terrible. sent_2 sent_3 kobe passed away, all in all [mask]. prof. ronaldo won nobel prize, all in all great. what a chance he missed, all in all terrible. sent_3 kobe passed away, all in all [mask]. prof. ronaldo won nobel prize, all in all great. what a chance he missed, all in all terrible. figure 1: overview of our proposed method. besides the standard prompt-base mlm loss on label words "great" and "terrible", we introduce a supcon loss on multi-views of input text. the positive pair is sentences (with sampled templates and/or demonstrations) in the same class, e.g. sent 1 and sent 3 , or itself with a different template and demonstrations, e.g. sent 1 and sent 2 . the negative sentence pair is input sentences (with sampled templates and/or demonstrations) in different classes, e.g. sent 1 and sent 0 . @xcite . inspired by the in-context learning of gpt-3, prompt-based fine-tuning @xcite @xcite recently becomes dominant in nlp. @xcite applies contrastive learning in their few-shot semi-supervised intent classification, by using eda @xcite as augmentation method. different from @xcite , our method applies to prompt-based fine-tuning, and we show in experiments that our proposed augmentation outperforms eda. supervised contrastive loss. supcon is a special form of contrastive learning @xcite @xcite ) that clusters two augmented batches at the class level in the feature space. let x2k-1 , x2k be two augmented views of an input batch x k ; and z 2k , z 2k-1 to be the features of x2k-1 , x2k . then supcon loss can be computed as fine-tuning with prompts and demonstrations. prompt-based methods treat a classification problem as a masked language modeling (mlm) problem. they take as input a sentence (sent) and a masked template (temp) (i.e., x @xmath0, temp([mask])), and find the best token to fill in the @xcite . this leads to a mlm loss l @xmath1, y), where y is the label word corresponding to x prompt . lm-bff @xcite further appends demonstrations of label words to improve the results: x prompt+@xmath2, temp 0 ([mask]), sent i , temp 0 (word i ) , where word i is the label word for sent i , and sent i is sampled from the training set. then the classification loss becomes: more mathematical formulation can be found in lm-bff or our appendix b. language-based supervised contrastive loss. for applying supcon on multi-views of an input text, we need to first obtain two views of a text: see our appendix c for more mathematical details. computational overhead. we show the algorithm of our method in algorithm 1. in general, our method learns from l @xmath3, whereas baseline lm-bff learns with l mlm only. learning from l supcon requires one additional forward and backward pass (highlighted in blue in algorithm 1), leading to an increase of computational cost by ×1.5. ## improvements vs. task difficulty here, we show that the improvements achieved by our method are greater for tasks with higher difficulty. to show this, we first sort the 15 tasks by base (lm-bff) performance and use this ranking as a proxy for the difficulty of the task. next, we report the average improvements achieved by our method on the top k hardest tasks, where k goes from 1 to 15. figure 2 shows these results. the first bar corresponds to the improvement achieved by our method on the hardest task, the second bar corresponds to the average improvement achieved by our method on the hardest and second-hardest tasks, and so on. the last bar corresponds to the average improvement on all 15 tasks. the results in ## limitations since supcon clusters examples on class level, our framework applies only to classification tasks. also, our framework requires large gpu memory, as supcon is an in-batch contrastive loss that needs a large batch size. ## conclusion we proposed a novel supervised contrastive learning framework and an effective augmentation method using prompts that can boost the performance of prompt-based language learners and outperform recent work on 15 few-shot tasks.