Pre-training to Match for Unified Low-shot Relation Extraction (2022)

Abstract: Low-shot relation extraction (RE) aims to recognize novel relations with very few or even no samples, which is critical in real scenario application. Few-shot and zero-shot RE are two representative low-shot RE tasks, which seem to be with similar target but require totally different underlying abilities. In this paper, we propose Multi-Choice Matching Networks to unify low-shot relation extraction. To fill in the gap between zero-shot and few-shot RE, we propose the triplet-paraphrase meta-training, which leverages triplet paraphrase to pre-train zero-shot label matching ability and uses meta-learning paradigm to learn few-shot instance summarizing ability. Experimental results on three different low-shot RE tasks show that the proposed method outperforms strong baselines by a large margin, and achieve the best performance on few-shot RE leaderboard.

## introduction relation extraction (re) aims to extract the relation between two given entities in the context. the most popular approaches to build re models are based on supervised learning @xcite . despite the superior performance, supervised relation extraction approaches severely suffer from the data bottleneck, which restricts their application to more relation types in real scenarios. consequently, low-shot relation extraction has become a recent research hotspot in re area. there are two mainstream learning paradigms widely explored in low-shot relation extraction, namely zeroshot re @xcite and few-shot re @xcite . few-shot relation extraction aims to identify instances of novel relation type with only a few illustrative instances, while zero-shot re is more progressive, which only uses external knowledge and the name or definition of the novel relations to recognize them. because low-shot re only requires very limited manually annotated data, it can effectively alleviate data bottlenecks in conventional re and therefore attached great attention. however, even with similar goals, zero-shot re and few-shot re actually require different fundamental abilities. specifically, zero-shot re is built on label semantic matching ability, which requires models to sufficiently exploit the label semantic of given novel relations, and matches relations and queries based on their underlying semantics. while few-shot re is built on instance semantic summarizing ability, which requires a model to quickly generalize to novel relations by summarizing critical information from few-shot instances. due to this fundamental difference, current state-of-the-art architectures are separately learned to deal with these two low-shot re tasks. for zero-shot re, the most popular solution is to transform it into a textual entailment @xcite , word prediction @xcite or mrc problem @xcite and use external resources from these tasks to pre-training the label semantic matching ability. however, the divergence between relation extraction and these tasks will inevitably undermine the performance. besides, mrc and tex-tual entailment architecture can only deal with one novel relation each time, which significantly increases the computational and storage cost of deploying such models in real-world scenarios. for few-shot re, current methods mainly focus on summarizing better prototypes from a few illustrative instances @xcite , or learning a model that can generalize to novel types within a few steps @xcite . these approaches require few-shot examples to fine-tune or summarize prototypes, and therefore can not be directly applied to zero-shot re. as a result, current relation extraction models can not be effectively and efficiently to apply to all low-shot re settings. in this paper, we propose to unify low-shot relation extraction by returning to the essence of relation extraction. fundamentally, relation extraction can be viewed as a multiple choice task. given two entities in context, a re system needs to match the most appropriate relation -or others for none-ofthe-above -from a set of pre-defined relation types. the information required to accomplish the multichoice matching can be summarized from either the surface form of relation name or from few-shot instances. motivated by this, we propose multi-choice matching network (mcmn) for unified low-shot re, which is shown in figure 2 . specifically, mcmn converts all candidate relation descriptions into a multi-choice prompt. then the input instance is concatenated with the multi-choice prompt and passes through a pre-trained encoder to obtain the semantic representations of the input instance and candidate relations. ## multi-choice matching networks in this section, we introduce our multi-choice matching networks (mcmn). different from previous unifying models, mcmn adopts a much more efficient and lightweight decoding module. following are the detailed descriptions. ## training strategies for multi-choice matching networks as mentioned above, the required abilities for zeroshot and few-shot are different. in this paper, we propose triplet-paraphrase meta pre-training, which jointly learn the label semantic matching ability required by zero-shot re and instance summarizing ability required by few-shot re. following is the detailed description of the pre-training framework. ## conclusions in this paper, we propose multi-choice matching networks to unify low-shot relation extraction. mcmn introduces a multi-choice prompt to formulate relation extraction as in a multichoice paradigm. to equip mcmn with different zero-shot and few-shot abilities, we propose the triplet-paraphrase meta pre-training, which leverages triplet paraphrase to pre-train zero-shot label matching ability and uses meta-learning paradigm to learn few-shot instance summarizing ability. experimental results on three different re tasks show mcmn outperforms strong baseline models by large margins.