An Exploration of Prompt-Based Zero-Shot Relation Extraction Method (2022)

Abstract: “Zero-shot relation extraction is an important method for dealing with the newly emerging relations in the real world which lacks labeled data. However, the mainstream two-tower zero-shot methods usually rely on large-scale and in-domain labeled data of predefined relations. In this work, we view zero-shot relation extraction as a semantic matching task optimized by prompt-tuning, which still maintains superior generalization performance when the labeled data of predefined relations are extremely scarce. To maximize the efficiency of data exploitation, instead of directly fine-tuning, we introduce a prompt-tuning technique to elicit the existing relational knowledge in pre-trained language model (PLMs). In addition, very few relation descriptions are exposed to the model during training, which we argue is the performance bottleneck of two-tower methods. To break through the bottleneck, we model the semantic interaction between relational instances and their descriptions directly during encoding. Experiment results on two academic datasets show that (1) our method outperforms the previous state-of-the-art method by a large margin with different samples of predefined relations; (2) this advantage will be further amplified in the low-resource scenario.”

## introduction relation extraction (re) aims to extract the relation between entity pairs from unstructured text. the extracted relation facts can benefit various downstream applications such as knowledge graph completion @xcite , web search @xcite and dialog systems @xcite . however, many effective re methods @xcite work within predefined relation sets. they failed to deal with a real-world environment where new relations will emerge after the training phase. these fast-growing new relations make it impossible for us to gather labeled training data for all of them. to recognize the newly emerging relations lacking labeled data, zero-shot re is of the utmost practical interest. despite the great potential of zero-shot re in real-world applications, there have been relatively few studies focusing on this challenging task. to enable models to predict unseen relations, previous works usually model zero-shot relation extraction as a well-designed task form. @xcite consider relation extraction as a machine reading comprehension. they first associate a few question templates for each relation and then determine which relation satisfies the given sentence and question by model prediction. however, a reasonable and effective question template usually needs careful design, which cannot meet the extraction needs of rapidly growing new relations @xcite . therefore, instead of manually constructing question templates, subsequent works @xcite take advantage of the readily available textual description to represent the new relations, and formulate zero-shot re as a semantic matching task achieving superior results. however, current methods usually require a large number of in-domain labeled data of predefined relations to train the model parameters. the learned relational knowledge is mainly from labeled data figure 1 : when shifting to some special domains (e.g. medicine, finance) where large-scale labeled data are not available, the performance of these methods on new relations decreases significantly. by inducing the knowledge in the pre-trained language model, our method can approach the results of previous state-of-the-art method zs-bert @xcite using only 200 labeled data. when using all data, our method improves the f1 score by 8.83%. to answer this question, in this work, we propose a prompt-based zero-shot re method. different from previous methods, in which the learned relational knowledge mainly comes from the labeled data of predefined relations, we leverage prompt to stimulate the rich knowledge distributed in plms to reduce dependence on these labeled data. specifically, we model zero-shot re as a semantic matching task between relational instance and description. in order to induce the knowledge in plms, we fuse the original input with the prompt template to formulate a cloze-style task. then, we count the probability distribution of the model output and take the words with significant differences between classes as label words. in addition, each predefined relation corresponds to many instances and one description. the significant quantity gap makes the two-tower methods unable to effectively model the semantics of relation description. therefore, we directly model the semantic interaction between instances and descriptions during training. based on the reformulated input and these selected label words, we optimize a semantic matching model, which predicts whether the relation and the textual description match. experimental results show that our method has very significant advantages when the large-scale labeled data of predefined relations are not available. to summarize, the main contributions of our work are as follows: (1) we propose a prompt-based zero-shot relation extraction method, which maintains high generalization ability when using even one labeled data per predefined relations. (2) we design comprehensive experiments to analyze the impact of predefined relations and prompt composition on the generalization performance of the model in the low-resource scenario, which may enlighten the following work. (3) experiment results on two academic datasets show that our method outperforms the previous state-of-the-art method by a large margin and this advantage will be further amplified in low resource scenarios. ## method we reformulate the task of zero-shot relation extraction as a semantic matching task optimized by prompttuning. in this section, we will introduce our proposed method in detail. we start by defining the problem we will tackle. then we introduce how we reformulate zero-shot relation extraction, our prompt design and the selection of label tokens. finally, we introduce the strategy of making predictions with our model. ## experimental setup in this section, we describe the datasets for training and evaluating the proposed method. we also detail the baseline models for comparison. finally, we clarify the implementation details and hyperparameter configuration of our method. ## conclusions in this work, we introduce a prompt-based zero-shot relation extraction method, which still maintains superior generalization performance under low-resource settings. we clarify the limitations of the two-tower architecture in previous state-of-the-art methods, and directly model the interaction between instances and descriptions during encoding, which breaks the performance bottleneck of the previous model. the introduce of prompt-tuning effectively elicit the knowledge in plms and significantly reduces the dependence on predefined relations. we believe that these are the reasons why our method achieves excellent results. experiment results on two academic datasets show that our method outperforms the previous state-of-the-art method by a large margin and this advantage will be further amplified in low resource scenarios.