Learn from Relation Information: Towards Prototype Representation Rectification for Few-Shot Relation Extraction (2022)

Abstract: Few-shot Relation Extraction refers to fast adaptation to novel relation classes with few samples through training on the known relation classes. Most existing methods focus on implicitly introducing relation information (i.e., relation label or relation description) to constrain the prototype representation learning, such as contrastive learning, graphs, and specifically designed attentions, which may bring useless and even harmful parameters. Besides, these approaches are limited in handing outlier samples far away from the class center due to the weakly implicit constraint. In this paper, we propose an effective and parameter-less Prototype Rectification Method (PRM) to promote few-shot relation extraction, where we utilize a prototype rectification module to rectify original prototypes explicitly by the relation information. Specifically, PRM is composed of two gate mechanisms. One gate decides how much of the original prototype remains, and another one updates the remained prototype with relation information. In doing so, better and stabler global relation information can be captured for guiding prototype representations, and thus PRM can robustly deal with outliers. Moreover, we also extend PRM to both none-of-the-above (NOTA) and domain adaptation scenarios. Experimental results on FewRel 1.0 and 2.0 datasets demonstrate the effectiveness of our proposed method, which achieves state-of-the-art performance.

## introduction relation extraction (re) is one of the fundamental natural language processing (nlp) tasks, which aims to detect the relation between two entities contained in a sentence. most re models @xcite @xcite require large labeled datasets while constructing such datasets is usually high-costing and timeconsuming. thus, the few-shot relation extraction (fsre) has become a hot topic to alleviate data scarcity. there are two main steps in fsre. the model is first trained on collections of fewshot tasks (i.e., meta tasks) sampled from the largescale data containing disjoint relations and then fast adapted to the unseen relation classes with few samples. recently, many approaches have been proposed for addressing fsre problems @xcite @xcite . one of the popular algorithms is the prototype network @xcite , which is based on the meta-learning framework @xcite , and the basic framework used in the paper. prototype network generates a prototype representation for each relation class in the meta task with the given instances (generally average instances in each relation class). then, the distance of query instances and each class prototype are calculated for model train and prediction. to achieve better performance, many works have integrated the relation information into the model to assist prototype representation learning. tdproto @xcite enhanced prototypical network with both relation and entity descriptions. cteg @xcite proposed a model that learns to decouple high co-occurrence relations, where two types of external information are added. mapre @xcite proposed a framework considering both label-agnostic and label-aware semantic mapping information in pre-training and fine-tuning. hcrp @xcite introduced three modules containing hybrid prototype learning, relation-prototype contrastive learning, and task adaptive focal loss for the model improvement. however, most existing methods mainly utilized instances given in each relation class to obtain the prototype representation (generally to average these instances). although they implicitly incorporate relation information to constrain the prototype representations learning by contrastive learning, graphs, or attentions, such insufficiently and weakly implicit constraints are limited in dealing with the outlier samples. we provide an example in fig. 1 , where classa and classb represent two different relation classes; the circles and squares represent the few instances provided by each relation class, and the green circle represents the instance that needs to be predicted. it can be seen that if the provided instances are remote and not "good", the model will tend to classify the instance (green circle) into the classb. besides, existing methods also introduced more parameters into the model owing to their specific designs, which is detrimental to fsre. more parameters mean a more complex model, which increases the overfitting risk on the training set, thereby reducing the generalization ability of the model @xcite . to address aforementioned issues, this paper proposes a prototype representation rectification method, named prm, which focuses on obtaining a better prototype for each relation. specifically, we propose a prototype rectification module, capable of explicitly utilizing relation information and instances to generate the rectified prototype representations together instead of implicitly using the relation information to guide the generation of the prototype representation. stated in another way, our model tries to use relation information to rectify the distribution of the original few instances to make it more global and more representative for the overall distribution of the class. prm transforms the problem of perceiving the class distribution from local instances to perceiving the class distri-bution from local instances and global information. in addition, we extend prm to an advanced version of the existing n -way k-shot setting in fewshot learning (i.e., none-of-the-above (nota) scenario), where queries could also be none-of-theabove instead of assuming that all query instances belong to the sampled n classes of supports. although this task brings one more option in classification and is more challenging for the general fsre model, our model can easily extend to nota by introducing an external description "the relation of the query is not the same as this prototype.". experiments on fewrel 1.0 (general scenario) and fewrel 2.0 (nota scenario) demonstrate the effectiveness of our proposed method with state-ofthe-art results. the contribution of our work mainly lies in three folds: ## related work relation extraction (re) @xcite is a fundamental task for information extraction, aiming to recognize the relation types that exist between entity pairs in one sentence. the labeling of relations is usually time-consuming and laborious. in addition, in some specific fields, such as the medical field, the available data are few and additional expertise is required. therefore, the fewshot relation extraction (fsre) task has attracted more and more attention recently. fsre aims to fast adapt to unseen relation classes with few samples through training on known relation classes. @xcite ; @xcite proposed a large-scale supervised few-shot relation classification dataset, namely, fewrel, and provided the current state-of-the-art results on fewrel, i.e., proto-bert @xcite and bert-pair. most subsequent work is evaluated on fewrel. regrab @xcite proposed to incorporate an external global relation graph based on a bayesian meta-learning method. except for relation descriptions, td-proto @xcite and conceptfere @xcite also introduced entity descriptions to provide clues for relation prediction and enhancing the prototype network. cteg @xcite proposed a model that learns to decouple high co-occurrence relations, where two external information is added. mapre @xcite proposed a framework considering both label-agnostic and label-aware semantic mapping information for low resource relation extraction in both pre-training and fine-tuning. hcrp @xcite introduced three modules containing hybrid prototype learning, relationprototype contrastive learning, and task adaptive focal loss for the model improvement. however, these methods always introduced relation information implicitly, which may introduce more parameters and are limited in dealing with outlier samples. thus, explicitly rectifying the prototypes with relation representations can be a more effective way to incorporate relation information. ## task definition we follow a typical few-shot task setting, namely the n -way-k-shot setup, which contains a support set s and a query set q. the support set s includes n novel classes, each with k labeled instances. the query set q contains the same n classes as s. and the task is evaluated on the query set q, trying to predict the relations of instances in q. what's more, an auxiliary dataset d base is given, which contains abundant base classes, each with a large number of labeled examples. note the base classes and novel classes are disjoint with each other. the few-shot learner aims to acquire knowledge from base classes and use the knowledge to recognize novel classes. one popular approach is the meta-learning paradigm, which mimics the few-shot learning settings at the training stage. specifically, in each training iteration, we randomly select n classes from base classes, each with k instances to form a support set @xmath0; @xmath1, ..., n, @xmath2, ..., k}. meanwhile, g instances are sampled from the remaining data of the n classes to construct a query set @xmath3; @xmath4, ..., g}. the model is optimized by collections of few-shot tasks sampled from base classes so that it can rapidly adapt to new tasks. for an fsre task, each instance consists of a set of samples (x, e, y), where x denotes a natural language sentence, @xmath5, e t ) indicates a pair of the head entity and tail entity, generally called statements, and y is the relation label. the name and description for each relation are also provided as auxiliary support evidence for relation extraction. ## proposed method in this section, we present the details of our proposed approach. figure 2 shows the overall structure, where three colors are used to represent different relation types. the inputs are n -way k-shot tasks (sampled from the auxiliary dataset d base ), where each task contains a support set s and a query set q. meanwhile, we take the names and descriptions of these n classes (i.e., relations) as inputs as well. all input items share the same sentence encoder. the prototype rectification module utilizes relation representations and the mean value of representations of the given instances, called original prototypes, to generate the rectified prototypes together. then, the model calculates the distance between the rectified prototypes and each query for both training and predicting. ## conclusion in this paper, we proposed a prototype rectification method, prm, with relations based on prototype framework, where a prototype rectification module is used for obtaining rectified prototypes. we further extended prm to a none-of-the-above (nota) setting in few-shot learning. extensive experiments demonstrate the effectiveness of the proposed method. we believe that the idea of finding global information to rectify prototypes explicitly with fewer parameters is general and can be extended to other few-shot tasks.