Exploring Task Difficulty for Few-Shot Relation Extraction (2021)

Abstract: Few-shot relation extraction (FSRE) focuses on recognizing novel relations by learning with merely a handful of annotated instances. Meta-learning has been widely adopted for such a task, which trains on randomly generated few-shot tasks to learn generic data representations. Despite impressive results achieved, existing models still perform suboptimally when handling hard FSRE tasks, where the relations are fine-grained and similar to each other. We argue this is largely because existing models do not distinguish hard tasks from easy ones in the learning process. In this paper, we introduce a novel approach based on contrastive learning that learns better representations by exploiting relation label information. We further design a method that allows the model to adaptively learn how to focus on hard tasks. Experiments on two standard datasets demonstrate the effectiveness of our method.

## introduction relation extraction aims to detect the relation between two entities contained in a sentence, which is the cornerstone of various natural language processing (nlp) applications, including knowledge base enrichment @xcite , biomedical knowledge discovery @xcite , and question answering @xcite . conventional neural methods @xcite train a deep network through a large amount of labeled data with extensive relations, so that the model can recognize these relations during the test phase. although impressive performance has been achieved, these methods are difficult to adapt to novel relations that have never been seen in the training process. in contrast, humans can identify new relations with very few examples. it is thus of great interest to enable the model to generalize to new relations with a handful of labeled instances. figure 1 : an example of easy few-shot task (top) and hard few-shot task (bottom). this is a 3-way-1-shot setup -each task involves three relations, and each relation has one supporting instance. blue and red colors indicate head and tail entities respectively. for the easy task, the relations are very different, and it is easy to classify the query instance. however, due to the subtle differences among the relations in the hard tasks, it is challenging to correctly predict the true relation. inspired by the success of few-shot learning in the computer vision (cv) community @xcite , @xcite first introduce the task of few-shot relation extraction (fsre). fsre requires models to be capable of handling classification of novel relations with scarce labeled instances. a popular framework for few-shot learning is meta-learning @xcite , which optimizes the model through collections of few-shot tasks sampled from the external data containing disjoint relations with novel relations, so that the model can learn cross-task knowledge and use the knowledge to acclimate rapidly to new tasks. a simple yet effective algorithm based on meta-learning is prototypical network @xcite , aiming to learn a metric space in which a query instance is classified according to its distance to class prototypes. recently, many works @xcite @xcite for fsre are in line with prototypical networks, which achieve remarkable performance. nonetheless, the difficulty of distinguishing relations varies in different tasks @xcite , depending on the similarity between relations. as illustrated in figure 1 , there are easy few-shot tasks whose relations are quite different, so that they can be consistently wellclassified, and also hard few-shot tasks with subtle inter-relation variations which are prone to misclassification. current fsre methods struggle with handling the hard tasks given limited labeled instances due to two main reasons. first, most works mainly focus on general tasks to learn generalized representations, and ignore modeling subtle and local differences of relations effectively, which may hinder these models from dealing with hard tasks well. second, current meta-learning methods treat training tasks equally, which are randomly sampled and have different degrees of difficulty. the generated easy tasks can overwhelm the training process training and lead to a degenerate model. to fill this gap, this paper proposes a hybrid contrastive relation-prototype (hcrp) approach, which focuses on improving the performance on hard fsre tasks. concretely, we first propose a hybrid prototypical network, capable of capturing global and local features to generate the informative class prototypes. next, we present a novel relation-prototype contrastive learning method, which leverages relation descriptions as anchors, and pulls the prototype of the same class closer in representation space and pushes those of different classes away. in this way, the model gains diverse and discriminative prototype representations, which could be beneficial to distinguish the subtle difference of confusing relations in hard few-shot tasks. furthermore, we design a task-adaptive training strategy based on focal loss @xcite to learn more from hard tasks, which allocates dynamic weights to different tasks according to task difficulty. extensive experiments on two large-scale benchmarks show that our model significantly outperforms the baselines. ablation and case studies demonstrate the effectiveness of the proposed modules. our code is available at https://github.com/hanjiale/hcrp . the contributions of this paper are summarized as follows: ## task definition we follow a typical few-shot task setting, namely the n -way-k-shot setup, which contains a support set s and a query set q. the support set s includes n novel classes, each with k labeled instances. the query set q contains the same n classes as s. and the task is evaluated on query set q, trying to predict the relations of instances in q. what's more, an auxiliary dataset d base is given, which contains abundant base classes, each with a large number of labeled examples. note the base classes and novel classes are disjoint with each other. the few-shot learner aims to acquire knowledge from base classes and use the knowledge to recognize novel classes. one popular approach is the meta-learning paradigm @xcite , which mimics the few-shot learning settings at training stage. specifically, in each training iteration, we randomly select n classes from base classes, each with k instances to form a support set @xmath0; @xmath1, . . . , n, @xmath2, . . . , k}. meanwhile, r instances are sampled from the remaining data of the n classes to construct a query set @xmath3; @xmath4, . . . , r}. the model is optimized by collections of few-shot tasks sampled from base classes, so that it can rapidly adapt to new tasks. for an fsre task, each instance consists of a set of samples (x, e, y), where x denotes a natural language sentence, @xmath5, e t ) indicates a pair of head entity and tail entity, and y is the relation label. the name and description for each relation are also provided as auxiliary support evidence for relation extraction. for example, for a relation with its relation id "p726" in a dataset that we use, we can obtain its name "candidate" and description "person or party that is an option for an office in an election". ## approach in this section, we present the details of our proposed hcrp approach. the overall learning framework is illustrated in figure 2 . the inputs are n -way-k-shot tasks (sampled from the auxiliary dataset d base ), where each task contains a support set s and a query set q. meanwhile, we take the names and descriptions of these n classes (i.e., relations) as inputs as well. hcrp consists of three components. the hybrid prototype learning module generates informative prototypes by capturing global and local features, which can better capture the subtle differences of relations. the relationprototype contrastive learning component is then used to leverage the relation label information to further enhance the discriminative power of the prototype representations. finally, a task adaptive focal loss is introduced to encourage the model to focus training on hard tasks. ## conclusion this paper focuses on hard few-shot relation extraction tasks and proposes a hybrid contrastive relation-prototype approach. the method proposes a hybrid prototype learning method that generates informative prototypes to model small interrelation variations. a relation-prototype contrastive learning approach is proposed. using relation information as anchors, it pulls instances of the same relation class closer in the representation space while pushing dis-similar ones apart. this process further enables the model to acquire more discriminative representations. in addition, we introduce two easy-to-confuse relations ("mother" and "child") with 100 samples are adopted. best viewed in color. a task adaptive focal loss to focus more on hard tasks during training to achieve better performance. experiments have demonstrated the effectiveness of our proposed model. there are multiple avenues for future work. one possible direction is to design a better mechanism for selecting tasks in the training process rather than using random sampling.