Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention (2020)

Abstract: Wrong labeling problem and long-tail relations are two main challenges caused by distant supervision in relation extraction. Recent works alleviate the wrong labeling by selective attention via multi-instance learning, but cannot well handle long-tail relations even if hierarchies of the relations are introduced to share knowledge. In this work, we propose a novel neural network, Collaborating Relation-augmented Attention (CoRA), to handle both the wrong labeling and long-tail relations. Particularly, we first propose relation-augmented attention network as base model. It operates on sentence bag with a sentence-to-relation attention to minimize the effect of wrong labeling. Then, facilitated by the proposed base model, we introduce collaborating relation features shared among relations in the hierarchies to promote the relation-augmenting process and balance the training data for long-tail relations. Besides the main training objective to predict the relation of a sentence bag, an auxiliary objective is utilized to guide the relation-augmenting process for a more accurate bag-level representation. In the experiments on the popular benchmark dataset NYT, the proposed CoRA improves the prior state-of-the-art performance by a large margin in terms of Precision@N, AUC and Hits@K. Further analyses verify its superior capability in handling long-tail relations in contrast to the competitors.

## introduction relation extraction, as a fundamental task in natural language processing (nlp), aims to discriminate the relation between two given entities in a plain text. as recent data-driven algorithms, e.g., deep neural networks @xcite @xcite , have shown their capabilities in tackling nlp tasks, labor-intensive annotation and scarce training data become the major obstacles of achieving promising performance on relation extraction. instead of costly manual labeling, distant supervision method @xcite is proposed to autolabel the training data for relation extraction. it labels a sentence containing a pair of entities with the relation between them in a knowledge graph, e.g. freebase @xcite . a strong assumption behind this is that a sentence containing two entities only expresses the relation existing in the knowledge graph, but this assumption will not always hold @xcite . hence, it leads to two main problems regrading the training data. first, when the assumption is invalid, wrong labeling problem appears and degrades algorithms by introducing noisy supervision signals. this problem has been well-studied by recent works @xcite @xcite operating at bag level for multi-instance learning @xcite , where the "bag" denotes a set of sentences with the same entity pair. most of them use selective attention to avoid wrongly-labeled sentences. second, the long-tail problem is caused by using a knowledge graph as distant supervision to autolabel a domain-specific corpus, where the knowledge graph usually suffers from long-tail relations. for example, to build nyt dataset, applying freebase to a news corpus, new york times, leads to ∼70% of relations long-tail, as illustrated in figure 1 (left). this problem seriously disrupts the data balance and thus becomes one of the main barriers of improvement. to alleviate the long-tail problem, two approaches @xcite naturally share the knowledge from data-rich relations to the long-tail ones when those relations have semantic overlap. this semantic overlap or relatedness is usually stored in the relation hierarchies of a knowledge graph, e.g., freebase in figure 1 (middle). specifically, these approaches extend selective attention @xcite intuitively, selective attention and its hierarchical extensions learn relation label embeddings to score each sentence in a bag. in contrast, the proposed relation-augmented attention network achieves the same goal via a memory network-like structure: sentences equipped with relation features are passed into an attention-pooling (i.e., a kind of self-attention @xcite ) for bag-level representations. our method is especially effective when extended to multi-granular relations -the features are enriched by cross-relation sharing, which hence benefits long-tail relations. as shown in figure 1 (right), our proposed approach achieves consistent outstanding performance on both data-rich and long-tail relations. we use two objectives to jointly train the cora. the first is predicting the relation label at bag level, which is the goal of relation extraction. as auxiliary objective, the second is guiding the model to equip each sentence with correct multi-granular relation embeddings during the augmenting process. it 14 ). aims to boost downstream attention-pooling and is fulfilled by applying the multi-granular labels to the sentence-to-relation attention during training. our main contributions are summarized as: ## proposed approach this section begins with a definition of distantly supervised relation extraction with multi-granular relation labels. then an embedding method is introduced to represent sentences. lastly, our base model and its hierarchical extension are presented to handle wrong labeling and long-tail relations. an illustration of the model is shown in figure 2 . ## experiments we evaluate our proposed network on a popular benchmark dataset and conduct several analyses for insights into our proposed model. dataset and evaluation metrics. by following previous works @xcite @xcite , we employ the only popular distantly supervised relation extraction dataset, new york times (nyt) dataset @xcite . it contains 53 distinct relations which includes a na class denoting the relation between the entity pair is unavailable. and it consists of 570k and 172k sentences in training and test sets respectively. two metrics, 1) area under precision-recall curve (auc) and 2) top-n precision (p@n) are usually used to measure the effectiveness. we also use hits@k for long-tail relations by following @xcite . ## conclusion in this paper, we propose a novel multi-instance learning framework, relation-augmented attention, as our base model for distantly supervised relation extraction. it operates at bag level to minimize the effect of the wrong labeling and leverages a sent2rel attention to alleviate the long-tail problem. by fully exploiting hierarchical knowledge of relations, we extend the base model to cora for boosting the performance on long-tail relations. the experiments conducted on the nyt dataset show that cora delivers new state-of-the-art performance in terms of both standard metrics (i.e., top-n precision, pr curve, auc) and long-tail metrics (i.e., hits@k). and extensive analyses provide comprehensive insights into the proposed model, and verify its capability in handling both wrong labeling and long-tail problems.