A Simple yet Effective Relation Information Guided Approach for Few-Shot Relation Extraction (2022)

Abstract: Few-Shot Relation Extraction aims at predicting the relation for a pair of entities in a sentence by training with a few labelled examples in each relation. Some recent works have introduced relation information (i.e., relation labels or descriptions) to assist model learning based on Prototype Network. However, most of them constrain the prototypes of each relation class implicitly with relation information, generally through designing complex network structures, like generating hybrid features, combining with contrastive learning or attention networks. We argue that relation information can be introduced more explicitly and effectively into the model. Thus, this paper proposes a direct addition approach to introduce relation information. Specifically, for each relation class, the relation representation is first generated by concatenating two views of relations (i.e., [CLS] token embedding and the mean value of embeddings of all tokens) and then directly added to the original prototype for both train and prediction. Experimental results on the benchmark dataset FewRel 1.0 show significant improvements and achieve comparable results to the state-of-the-art, which demonstrates the effectiveness of our proposed approach. Besides, further analyses verify that the direct addition is a much more effective way to integrate the relation representations and the original prototypes.

## introduction relation extraction (re) @xcite is a fundamental task of natural language processing (nlp), which aims to extract the relations between entities in sentences and can be † corresponding author. 1 the code is released at https://github.com/ lylylylylyly/simplefsre . inspired by the success of few-shot learning in the computer vision (cv) community @xcite , various methods are introduced into fsre. one of the popular algorithms is the prototype network @xcite , which is based on the meta-learning framework @xcite . in detail, collections of few-shot tasks sampled from the external data containing disjoint relations are used as the training set for the model optimization. for each few-shot task, the center of each relation class is calculated and used as the prototype of the relation class. then, the model can be optimized by reducing the distances between the query samples and their corresponding prototypes. given a new sample, the model calculates which of the class prototypes is nearest to the new sample and assign it to this relation class. in order to get better results, many works have utilized relation information (i.e., relation labels or descriptions) to assist model learning. td-proto @xcite ) enhanced prototypical network with both relation and entity descriptions. cteg @xcite proposed a model that learns to decouple high co-occurrence relations, where two types of external information are added. another intuitive idea is to hope that the model can learn good prototypes or representations , that is, to reduce the distances of the intra-class while widening the ones among different classes @xcite , where @xcite introduced a novel approach based on supervised contrastive learning that learns better prototype representations by the utilization of prototypes and relation labels and descriptions during the model training; @xcite considered a semantic mapping framework, mapre, which leverages both label-agnostic and label-aware knowledge in pre-training and fine-tuning processes. however, there are two limitations in how these works introduce relation information. the first is that most of them take implicit constraints, like contrastive learning or relation graphs, instead of the direct fusion, which can be weak facing the remote samples. the second is that they usually adopt complicated designs or networks, like hybrid features or elaborate attention networks, which can bring too many or even harmful parameters. therefore, in this paper, we propose a straightforward yet effective way to incorporate the relation information into the model. specifically, on the one hand, the same encoder is used to encode relation information and sentences for mapping them into the same semantic space. on the other hand, we generate the relation representation for each relation class by concatenating two relation views (i.e., [cls] token embedding and the mean value of embeddings of all tokens), which allows relation representations and prototypes to form the same dimension. afterwards, the generated relation representation is directly added to the prototype for enhancing model train and prediction. figure 1 shows an intuitive illustration of the difference in ways to introduce relation information between most existing works and our proposed approach. based on the mentioned two limitations of previous works, we provide two possible high-level ideas about why our proposed approach should work for few-shot relation extraction. the first is that the direct addition is a more robust and denote the concatenation and addition operations, respectively. we conduct experiments on the popular fsre benchmark fewrel 1.0 @xcite under four few-shot settings. experimental results show considerable improvements and achieve comparable results to the state-of-the-art, which demonstrates the effectiveness of our proposed approach, i.e., the direct addition operation. ## proposed method in this section, we present the details of our proposed approach. figure 2 shows the overall structure, where the blue and yellow lines represent the flow of sentences and relation information, respectively. in order to map the representations of sentences and relation information into the same semantic space, the shared sentence encoder is utilized. then, we concatenate two views of the relation representations for obtaining the same dimension as prototypes and integrate relation representations into original prototypes by direct addition. ## conclusion in this paper, we proposed a simple yet effective approach with relation information based on prototype network. the core idea is to introduce relation representations by the direct addition operation instead of designing complex structures. experimental results on fewrel 1.0 achieve comparable results to the state-of-the-art and demonstrate the effectiveness of our proposed approach. moreover, we provide two high-level ideas, i.e., explicit constraints and fewer parameters, about why the direct addition is so effective. we believe that the idea of finding global information to perform the direct addition with original prototypes is general and can be extended to other few-shot tasks that can be modeled based on the prototype network. since the direct addition way of introducing relations is simple and efficient, we also believe that future work should focus more on generating better relation representations rather than designing fusion methods between relations and prototypes.