A Novel Table-to-Graph Generation Approach for Document-Level Joint Entity and Relation Extraction (2023)

Abstract: Document-level relation extraction (DocRE) aims to extract relations among entities within a document, which is crucial for applications like knowledge graph construction. Existing methods usually assume that entities and their mentions are identified beforehand, which falls short of real-world applications. To overcome this limitation, we propose TaG, a novel table-to-graph generation model for joint extractionof entities and relations at document-level. To enhance the learning of task dependencies, TaG induces a latent graph among mentions, with different types of edges indicating different task information, which is further broadcast with a relational graph convolutional network. To alleviate the error propagation problem, we adapt the hierarchical agglomerative clustering algorithm to back-propagate task information at decoding stage. Experiments on the benchmark dataset, DocRED, demonstrate that TaG surpasses previous methods by a large margin and achieves state-of-the-art results.

## introduction relation extraction (re) is the task to extract relational facts from natural language text, which plays a crucial role in various downstream tasks, e.g. knowledge graph construction and question answering @xcite @xcite . early studies mostly focus on sentence-level re, i.e. predicting relations among entities in one single sentence. however, in realworld scenarios such as wikipedia articles or scientific papers, large amounts of relational facts are expressed across multiple sentences, which necessitate inter-sentence reasoning skills. hence, recent efforts have been moving towards the more realistic document-level re (docre) @xcite @xcite . figure 1 : an example adapted from the docred dataset. mentions refer to the same entity are in same color. we omit some relations and denote some entities with underline for clarity. despite the rapid progress, most previous docre methods solely focus on the task of relation extraction, which assumes that entities and their corresponding mentions are given beforehand. as shown by figure 1 , to extract both of entities and relations at document-level, a natural idea is to use a pipeline approach. traditionally, it first divide the whole task into subtasks of mention extraction (me), coreference resolution (coref) and relation extraction (re), then use separate models to conduct each task step by step @xcite . however, the pipeline framework ignores the underlying dependencies among subtasks, which may lead to suboptimal performance. some progress on jointly considering the subtasks has been made @xcite , yet, previous attempts still model the tasks of coref and re separately, inducing possible bias at both encoding and decoding stages. on the one hand, these methods still suffer from the problem with lack of information sharing. they either completely rely on the shared language model (e.g. bert) at representation level @xcite , or only consider one-way information flow from re to coref and neglect other cross-task dependencies @xcite . on the other hand, prior approaches mostly employ the pipelinestyle decoding, which first recognize mention spans and form entity clusters, then perform relation classification for each entity pair. such routine is not only time consuming, but faces with the error propagation problem @xcite . the results of entity extraction may affect the performance of relation extraction and lead to cascading errors. @xcite attempt to use a regularization term in coref scorer to mitigate this issue, but the problem is still not fully resolved. in this work, we propose tag, a novel tableto-graph generation model, to address these aforementioned challenges. we first unify both tasks of coref and re with the classic table filling framework @xcite . we then devise a following table filler to encode original texts and make predictions for both tasks at a coarse level. regarding mentions as nodes, we dynamically build two corresponding coreference and relation graphs, where the edges are weighted by the confidence scores of table filler. besides, to alleviate the long-term dependency problem as well as explicitly model the syntactic information, we construct a syntactic graph over mentions. given these three subgraphs, tag regards them as three different types of edges and uses a relational graph convolutional network (r-gcn, @xcite to model implicit task dependencies at a fine level. unlike previous multi-task systems that solely share span representations directly from the language model, our coarse-to-fine framework leverages rich node representations by propagating information through semantic and syntactic links. intuitively, mentions within the same entity cluster should establish similar relation links with other entities @xcite . to avoid the error propagation problem, we exploit this postulation and adapt the hierarchical agglomerative clustering (hac) algorithm to cluster mentions. the core of hac is the computation of coreference distance between each cluster pair. to back-propagate relational information, we compute the relation vectors of nodes and use the average hamming distances among different clusters as additional penalty. we evaluate tag on docred @xcite , a widely-adopted docre benchmark. experiments show that: (1) the coarse-grained table filler baseline establishes competitive results, as compared with previous methods. (2) the finegrained information propagation module and enhanced hac decoding algorithm can effectively promote cross-task interactions and better alleviate the error propagation problem. (3) our proposed tag achieves new state-of-the-art and outperforms prior approaches by a large margin. we also report the first result of joint entity and relation extraction on re-docred @xcite , a revised version of docred, for future research. our contributions can be summarized as follow: ## problem formulation given a document d comprised of l tokens, our goal is to jointly extract all entities and relations in an end-to-end manner. as an entity may occur multiple times in the document with different mentions, the joint extraction process can be naturally divided into three subtasks: ## methodology figure 2 shows the overall architecture of tag. tag first conducts me to predict mention spans ( § 3.1), after which, it jointly learns the tasks of coref and re with a table-to-graph generation model ( § 3.2). we will also detail the multi-task training process in § 3.3 and enhanced decoding algorithm in § 3.4. ## related works document-level extraction and joint extraction are two important topics in the field of ie. our work lies at the intersection of these two lines, which aims to jointly extract entities and relations, two core elements of ie, at document-level. ## conclusion in this paper, we propose tag, a novel table-tograph generation model, to jointly extract entities and relations within a document. different from prior approaches, we unify the tasks of coreference resolution and relation extraction with a table filling framework, and leverage a coarse-to-fine strategy to facilitate information sharing among these subtasks. to avoid the error propagation problem, we adapt the hac algorithm to enhance coref with re predictions at decoding stage. experimental results on the widely-adopted benchmark, docred, demonstrate that tag significantly outperforms previous methods. further analysis also confirms the effectiveness of the modules in our model.