Document-Level Relation Extraction via Pair-Aware and Entity-Enhanced Representation Learning (2022)

Abstract: Document-level relation extraction aims to recognize relations among multiple entity pairs from a whole piece of article. Recent methods achieve considerable performance but still suffer from two challenges: a) the relational entity pairs are sparse, b) the representation of entity pairs is insufficient. In this paper, we propose Pair-Aware and Entity-Enhanced(PAEE) model to solve the aforementioned two challenges. For the first challenge, we design a Pair-Aware Representation module to predict potential relational entity pairs, which constrains the relation extraction to the predicted entity pairs subset rather than all pairs; For the second, we introduce a Entity-Enhanced Representation module to assemble directional entity pairs and obtain a holistic understanding of the entire document. Experimental results show that our approach can obtain state-of-the-art performance on four benchmark datasets DocRED, DWIE, CDR and GDA.

## introduction relation extraction (re) is a primary task in the field of information extraction, which aims to identify the relationships between two entities in a document. previous works mainly focus on sentencelevel relation extraction, i.e, recognizing the relationships between entities in a sentence. however, large amounts of relationships are expressed over multiple sentences in real-world applications. according to docred @xcite , above 40.7% of the relational facts can only be extracted from multiple sentences. therefore, it requires the model to capture complex interactions among entities in the whole document. previous work commonly referred to this problem as document-level relation extraction which has attracted much attention recently @xcite @xcite . although the considerable performance of these methods, there are still two critical challenges in document-level re to be addressed. the first challenge is how to identify relational entity pairs that are sparse in a document. specifically, given a document with n entities, there will be n(n -1) combinations of entities to classify. however, only a few entity pairs have predefined relationships. for example, as shown in figure 1 , this document contains 21 entities with 420 potential entity pairs. however, the number of relational entity pairs is only 11, accounting for 2.62 % of the total entity pairs. according to statistics, for docred @xcite dataset, the proportion are 3.18% and 3.11% in the train set and dev set, respectively. to further explore the impact of sparsity on performance bottlenecks, we conduct a diagnostic experiment on docred dataset. utilizing previous sota model atlop @xcite , we divulge the information of whether existing a predefined relationship between the entities to the model. specifically, we just concatenate a 0-1 variable on the original representation of entity pairs, where "1" represents the entity pair exists a predefined relationship. experimental results show that the f1 score reaches 93.50% in dev set which is 32.20% higher than normal setting. this demonstrates that the importance of identifying the relational entity pairs when facing the sparsity problem @xcite . the second challenge is how to effectively model the representations of entity pairs. there are commonly two characteristics for entity pairs. firstly, the entities-scattering, which means the entities of an entity pair may scatter across multiple sentences. figure 1 illustrates an example from the docred dataset. for pair_a, the subject ali abdullah ahmed and object y asir al-salami are distributed in different sentences ( @xcite and @xcite ), which requires model to capture the long-distance dependency among entities across sentences. secondly, the directivity of entity pairs, which means that the relationships of entity pairs are directional. for example, the pair_b and pair_b ′ in the figure 1 , their subject and object are opposite, and the relations of them are different. therefore, this challenge requires the model to assemble directional entity pairs and obtain a holistic understanding of the cross-sentence context. to model the representation of entity pairs, most current approaches include graph-based methods and transformer-based methods. specifically, some methods @xcite @xcite ) construct a document graph with structured attention, dependency structures or heuristics. meanwhile, considering the transformer can capture long-distance information, some studies @xcite @xcite directly apply pre-trained language models without introducing graph structures. however, they directly concatenate two entities together to obtain the representation of entity pair, without considering the directivity of entity pairs and modeling the representations of entity pairs adequately. in this paper, we propose a pair-aware and entity-enhanced (paee) model for document-level re. to deal with the sparsity of entity pairs, we propose the p air-aware representation(par) module to identify potential relational entity pairs, which constrains the relation extraction to the predicted pairs subset rather than all pairs. furthermore, to capture the global features of triples, par utilizes tnet @xcite to model the relation between entity pairs, unlike previous methods, par designs a sliding window filling strategy for filling relation matrix, which enhances the interaction between entity pairs. to effectively model the representation of entity pairs, we focus on the global interactions among sentences and entities. specifically, we propose a entity-enhanced representation(eer) module. the eer first introduces a representation-enhanced encoder to facilitate the interaction between all sentences and entities. in this way, eer obtains a holistic understanding of the entire document. then, considering that the characteristics of entities as subjects and objects are different, especially in different relationship categories, eer utilizes a cross matching method to assemble directional entity pairs. experiments on four document-level relation extraction datasets, docred @xcite , dwie @xcite , cdr @xcite and gda @xcite , demonstrate that our paee model significantly outperforms the state-of-the-art methods. to our best knowledge, we are first to consider the sparsity and the directivity of relational entity pairs for the task. we summarize our contributions as follows: ## methodology before introducing our proposed approach for paee in this section, we first introduce the problem definition. given a document d and a set of entities {e i } n @xmath0, and there are n(n -1) entity ## conclusion in this paper, we propose the pair-aware and entity-enhanced (paee) model. specifically, paee introduces pair-aware representation(par) module to alleviate the negative impact of sparsity, which constrains the following relation extraction to the predicted entity pairs subset rather than all pairs. in addition, paee also designs entity-enhanced representation(eer) module to assemble directional entity pairs and obtain holistic understanding of document. experiments on four benchmark datasets docred, dwie, cda and gda, show that paee outperforms the previous methods and obtains new state-of-the-art results.