Double Graph Based Reasoning for Document-level Relation Extraction (2020)

Abstract: Document-level relation extraction aims to extract relations among entities within a document. Different from sentence-level relation extraction, it requires reasoning over multiple sentences across paragraphs. In this paper, we propose Graph Aggregation-and-Inference Network (GAIN), a method to recognize such relations for long paragraphs. GAIN constructs two graphs, a heterogeneous mention-level graph (MG) and an entity-level graph (EG). The former captures complex interaction among different mentions and the latter aggregates mentions underlying for the same entities. Based on the graphs we propose a novel path reasoning mechanism to infer relations between entities. Experiments on the public dataset, DocRED, show GAIN achieves a significant performance improvement (2.85 on F1) over the previous state-of-the-art. Our code is available at https://github.com/PKUnlp-icler/GAIN .

## introduction the task of identifying semantic relations between entities from text, namely relation extraction (re), plays a crucial role in a variety of knowledge-based applications, such as question answering @xcite and large-scale knowledge graph construction. previous methods @xcite @xcite @xcite focus on sentence-level re, which predicts relations among entities in a single sentence. however, sentence-level re models suffer from an inevitable limitation -they fail to recognize relations between entities across sentences. hence, extracting relations at the document-level is necessary for a holistic understanding of knowledge in text. figure 1 : an example document and its desired relations from docred @xcite . entity mentions and relations involved in these relation instances are colored. other mentions are underlined for clarity. there are several major challenges in effective relation extraction at the document-level. firstly, the subject and object entities involved in a relation may appear in different sentences. therefore a relation cannot be identified based solely on a single sentence. secondly, the same entity may be mentioned multiple times in different sentences. crosssentence context information has to be aggregated to represent the entity better. thirdly, the identification of many relations requires techniques of logical reasoning. this means these relations can only be successfully extracted when other entities and relations, usually spread across sentences, are identified implicitly or explicitly. as figure 1 shows, it is easy to recognize the intra-sentence relations (maryland, country, u.s.), (baltimore, located in the administrative territorial entity, maryland), and (eldersburg, located in the administrative territorial entity, maryland), since the subject and object appear in the same sentence. however, it is non-trivial to predict the inter-sentence relations between baltimore and u.s., as well as eldersburg and u.s., error type count intra-sentence 535 inter-sentence 615 logical reasoning 242 previous work in document-level re do not consider reasoning @xcite @xcite , or only use graph-based or hierarchical neural network to conduct reasoning in an implicit way @xcite @xcite . in this paper, we propose a graph aggregation-and-inference network (gain) for document-level relation extraction. it is designed to tackle the challenges mentioned above directly. gain constructs a heterogeneous mention-level graph (mg) with two types of nodes, namely mention node and document node, and three different types of edges, i.e., intra-entity edge, inter-entity edge and document edge, to capture the context information of entities in the document. then, we apply graph convolutional network @xcite on mg to get a document-aware representation for each mention. entity-level graph (eg) is then constructed by merging mentions that refer to the same entity in mg, on top of which we propose a novel path reasoning mechanism. this reasoning mechanism allows our model to infer multi-hop relations between entities. in summary, our main contributions are as follows: we evaluate gain on the public docred dataset. it significantly outperforms the previous state-of-the-art model by 2.85 f1 score. further analysis demonstrates the capability of gain to aggregate document-aware context information and to infer logical relations over documents. ## task formulation we formulate the document-level relation extraction task as follows. given a document comprised of n sentences @xmath0, where s @xmath1, e @xmath2, the task aims to extract the relations between different entities in e, namely {(e i , r ij , e j )|e i , e @xmath3, r @xmath4, where r is a pre-defined relation type set. in our paper, a relation r ij between entity e i and e j is defined as inter-sentential, if and only if s e i ∩ s e @xmath5, where s e i denotes those sentences containing mentions of e i . instead, a relation r ij is defined as intra-sentential, if and only if s e i ∩s e @xmath6. we also define k-hop relational reasoning as predicting relation r ij based on a k-length chain of existing relations, with e i and e j being the head and tail of the reasoning chain, i.e., e i r 1 ## graph aggregation and inference network (gain) gain mainly consists of 4 modules: encoding module (sec. 3.1), mention-level graph aggrega- figure 2 : the overall architecture of gain. first, a context encoder consumes the input document to get a contextualized representation for each word. then, the mention-level graph is constructed with mention nodes and a document node. after applying gcn, the graph is transformed into entity-level graph, where the paths between entities are identified for reasoning. finally, the classification module predicts target relations based on the above information. different entities are in different colors. the number i in the mention node denotes that it belongs to the i-th sentence. ## related work previous approaches focus on sentence-level relation extraction @xcite @xcite @xcite @xcite . but sentence-level re models face an inevitable restriction in practice, where many realworld relation facts can only be extracted across sentences. therefore, many researchers gradually shift their attention into document-level relation extraction. several approaches @xcite @xcite @xcite leverage dependency graph to better capture document-specific features, but they ignore ubiquitous relational inference in document. recently, many models are proposed to address this problem. @xcite proposed a hierarchical inference network by considering information from entity-level, sentence-level, and documentlevel. however, it conducts relational inference implicitly based on a hierarchical network while we adopt the path reasoning mechanism, which is a more explicit way. @xcite is one of the most powerful systems on document-level re tasks recently. compared to @xcite and other graph-based approaches to relation extraction, our architecture features many different designs with different motivations behind them. first, the ways of graph construction are different. we create two separate graphs of different levels to capture long-distance document-aware interactions and entity path inference information, respectively. while @xcite put mentions and entities in the same graph. moreover, they do not conduct graph node representation learning like gcn to aggregate interactive information on the constructed graph, only using the features from bilstms to represent nodes. second, the processes of path inference are different. @xcite use a walkbased method to iteratively generate a path for every entity pair, which requires the extra overhead of hyper-parameter tuning to control the process of inference. instead, we use an attention mechanism to selectively fuse all possible path information for the entity pair while without extra overhead. when we were writing this paper, @xcite ) make their work public as preprints, which adopt the dependency tree to capture the semantic information in the document. they put mention and entity nodes in the same graph and conduct inference implicitly by using gcn. unlike their work, our gain presents mention node and entity node in different graphs to better conduct inter-sentence information aggregation and infer relations more explicitly. some other attempts @xcite @xcite study document-level re in a specific domain like biomedical re. however, the datasets they use usually contain very limited relation types and entity types. for instance, cdr @xcite only has one type of relation and two types of entities, which may not be the ideal testbed for relational reasoning.