Instance-Guided Prompt Learning for Few-Shot Text Matching (2022)

Abstract: Few-shot text matching is a more practical technique in natural language processing (NLP) to determine whether two texts are semantically identical. They primarily design patterns to reformulate text matching into a pre-trained task with uniform prompts across all instances. But they fail to take into account the connection between prompts and instances. This paper argues that dynamically strengthening the correlation between particular instances and the prompts is necessary because fixed prompts cannot adequately fit all diverse instances in inference. We suggest IGATE: Instance-Guided prompt leArning for few-shoT tExt matching, a novel pluggable prompt learning method. The gate mechanism used by IGATE, which is between the embedding and the PLM encoders, makes use of the semantics of instances to regulate the effects of the gate on the prompt tokens. The experimental findings show that IGATE achieves SOTA performance on MRPC and QQP, outperforming strong baselines. GitHub will host the release of codes.

## introduction prompt learning has made significant progress in few-shot text matching due to the widespread use of pre-trained language models (plms) in natural language processing (nlp). this approach reformulates the text matching task as a cloze-style question which requires plms to predict what the blank should be filled with. more specifically, after being added prompt tokens and masked tokens, a plm is used to predict the masked words and map these words to the real labels. recent studies have shown that prompt learning achieves better performance on few-shot text matching. for example, with only thirty-two examples in mrpc @xcite , prompt learning achieves * * co-corresponding author about 85% of the performance of fully supervised fine-tuning models @xcite . brown et al. introduce the concept of prompt in the in-context method for the first time. subsequently, schick and schütze propose pet which gains improvement by exploiting patterns in natural language understanding. some studies @xcite @xcite search prompts automatically to reduce the reliance on human experts for manual pattern design. all of these methods use natural language as prompts thus they are named discrete prompts. other methods such as p-tuning @xcite , @xcite , and p-tuning-v2 @xcite replace natural language prompts with trainable continuous tokens on this basis to automatically search for optimal prompts in a high-dimensional space. correspondingly, these methods are also called continuous prompts. however, current prompt learning methods typically train models for specific task goals with little regard to the applicability of samples to the prompt. even though some recent works @xcite attempt to use contextual information for generating prompts, they frequently ignore how samples affect prompts and concentrate on how prompts contribute to instances. in addition, they usually fix the prompts during inference so that all test samples share the same pattern. to solve these problems, this paper proposes a novel method, igate: instance-guided prompt learning for few-shot text matching, to guide the construction of a prompt with instance semantics. we introduce the gate mechanism based on continuous prompts to regulate the flow of features in the prompts. meanwhile, since the semantics of instances are used to construct the prompt, each instance can play a restrictive role in building the pattern. so gate mechanism alleviates the problem of limited adaptation of the same pattern to instances in the reasoning process. in short, this study has the following contributions: this method can be easily transferred to other prompt methods. ## methodology in this section, we introduce conventional prompt learning approaches for few-shot text matching. then we introduce the architecture of igate, which adds an additional layer between the embedding layer and the encoder of plms. the architecture is shown in figure 1 . ## conclusion in this paper, we propose an instance-guided prompt learning method named igate. igate constructs prompts with the weight matrix which is extracted from the instance. thus the prompts are restricted by instance semantics in the training and inference process. experimental results show that igate achieves improvement on the text matching task, and can be generalized in nli tasks. meanwhile, igate can be applied to both discrete and continuous templates. in the future, we will explore more methods to construct prompts with the semantics of instances.