LM - BFF - MS : Improving Few-Shot Fine-tuning of Language Models based on Multiple Soft Demonstration Memory (2022)

Abstract: LM-BFF (CITATION) achieves significant few-shot performance by using auto-generated prompts and adding demonstrations similar to an input example. To improve the approach of LM-BFF, this paper proposes LM-BFF-MS — b etter f ew-shot f ine-tuning of l anguage m odels with m ultiple s oft demonstrations by making its further extensions, which include 1) prompts with multiple demonstrations based on automatic generation of multiple label words; and 2) soft demonstration memory which consists of multiple sequences of globally shared word embeddings for a similar context. Experiments conducted on eight NLP tasks show that LM-BFF-MS leads to improvements over LM-BFF on five tasks, particularly achieving 94.0 and 90.4 on SST-2 and MRPC, respectively.

## introduction the gpt-3 model @xcite has achieved remarkable few-shot performance on natural language understanding tasks given a natural language prompt and |k| labeled samples as demonstrations in the inputs without updating the model's weights. however, the gpt-3 model consists of 175b parameters, making it challenging to perform task-specific fine-tuning, which is often required in real-world applications. to enable task-specific fine-tuning, promptbased few-shot fine-tuning has been widely studied to encourage the few-shot capabilities of pretrained language models (plms) equipped with label-specific verbalizers and prompts that are compatible with language models @xcite . prompt-based fine-tuning reformulates downstream tasks as a masked language modeling problem, where a token (label word) is generated on a given prompt with a task-specific template. however, constructing optimal prompts requires domain expertise and the use of manual prompts can be suboptimal @xcite @xcite . among the various methods of prompt-based fine-tuning, this study is based on the lm-bff method @xcite , which uses a demonstration-aware prompt where a demonstration is produced by unmasking the example prompt in contexts similar to the input, inspired by the findings from the gpt-3 model @xcite . with demonstration-aware prompts, the lm-bff outperforms the conventional fine-tuning approach and gpt-3's in-context learning. to improve lm-bff, we propose lm-bff-ms, better few-shot fine-tuning of language models with multiple soft demonstration memory, based on the following two extensions: while lm-bff uses single demonstration foot_0 , our model uses multiple demonstrations with different label phrases, where each demonstration is constructed per label phrase. given that label phrases are semantically related or similar, it is expected that resulting demonstrations indirectly augment the vocabulary of the verbalizer with label phrases foot_1 . following the previous setting of the lm-bff, the experimental results on eight nlp datasets show that the proposed lm-bff-ms leads to a better and more stable few-shot performance compared to the previous models. the contributions of this study are summarized as follows: ## related work prompt-based few-shot fine-tuning, which finetunes based on few-shot examples under a prompting setting, has been widely studied for moderately sized plms such as bert @xcite and roberta @xcite . for example, pet reformulates downstream tasks as a masked language modeling problem and performs gradient-based fine-tuning @xcite . auto-prompt creates appropriate prompts for a set of discrete tokens using a gradient-guided search @xcite . null prompts-simple concatenations of the inputs and [mask] token-achieve a free of prompt engineering @xcite . instead of using hard prompts, there have also been works of using continuous vectors of prompt tokens, called soft prompting foot_2 , including the work of @xcite , which proposes soft prompts composed of learnable continuous embeddings while freezing the weight of plms; and @xcite proposes pre-training prompts by adding soft prompts into the pre-training stage to obtain a better initialization. the demonstration-aware prompt has also been explored by @xcite with their proposed lm-bff, where a demonstration is constructed by unmasking the masked prompt on a similar input example. unlike lm-bff, which uses a single demonstration per label, our work uses 'multiple' demonstrations that are provided for automatically generated label phrases. in addition, inspired by the method of soft prompting, we use 'soft' demonstration memory based on globally shared soft vectors for prompt tokens, without using hard tokens of the similar context. @xcite is reported. full: fine-tuning using full training set. ## experiments the implementation details are provided in appendix b. for a fair comparison, the same manual prompts for t label in lm-bff and lm-bff-ms are used. ## conclusion this study proposed lm-bff-ms-manual prompts with multiple soft demonstration memory based on the automatic generation of multiple label words and an auxiliary ndp task. experiments showed that the proposed method outperforms prior works on five tasks: sst-2, mr, subj, mrpc, and mpqa. extending our work to a large soft demonstration memory and a combination of local and global memory is valuable for future investigations.