Anaphor Assisted Document-Level Relation Extraction (2023)

Abstract: Document-level relation extraction (DocRE) involves identifying relations between entities distributed in multiple sentences within a document. Existing methods focus on building a heterogeneous document graph to model the internal structure of an entity and the external interaction between entities. However, there are two drawbacks in existing methods. On one hand, anaphor plays an important role in reasoning to identify relations between entities but is ignored by these methods. On the other hand, these methods achieve cross-sentence entity interactions implicitly by utilizing a document or sentences as intermediate nodes. Such an approach has difficulties in learning fine-grained interactions between entities across different sentences, resulting in sub-optimal performance. To address these issues, we propose an Anaphor-Assisted (AA) framework for DocRE tasks. Experimental results on the widely-used datasets demonstrate that our model achieves a new state-of-the-art performance.

## introduction document-level relation extraction (docre) has garnered increasing attention from researchers lately due to its alignment with real-world applications, where a large number of relational facts are expressed in multiple sentences @xcite . compared with its sentence-level counterpart @xcite @xcite , docre is more difficult as it requires a more sophisticated understanding of context and needs to model the interaction in mentions belonging to the same or different entities distributed in multiple sentences @xcite @xcite . to address docre, most of existing works use pre-trained language model, such as bert (devlin et al., 2019) and roberta @xcite , to capture the long-range dependency between entities in the document, which are called the transformerbased models @xcite @xcite . these models only take the word sequence as input without considering the internal structure of an entity and fail to explicitly learn the external interaction between entities, thus having difficulties in addressing complex instances that require reasoning @xcite . to enhance the reasoning ability of transformer-based models, recent works propose to transfer the document into a graph where entity mentions are introduced as nodes and operate a graph neural network @xcite on the document graph to explicitly learn information interactions between entities, which are usually called the graph-based models @xcite @xcite . though some promising results have been achieved by recent graph-based models, crosssentence entity interaction is not well learned in these models since most of them use a document node @xcite or sentence nodes @xcite as intermediate nodes to link entities in different sentences with an unclear information delivery. such an approach captures interactions between entities distributed in different sentences via a common document node or sentence node, which is not effective in learning fine-grained interactions between entities, leading to sub-optimal performance. besides, previous methods @xcite @xcite are generally missing the anaphors in a document. however, identifying cross-sentence relations between entities often requires the intervention of anaphors since anaphors often convey the interaction between entities in multiple sentences as intermediate nodes, such as pronouns or definite referents, which refer to entities within the context. as shown in figure 1 , to identify the relation "creator" between head entity "fox mulder" and tail entity "chirs carter", sentences [1], [3], and [8] should be considered simultaneously. in these sentences, the anaphors play an important role in helping the model identify the target relation. specifically, "it" in sentence [3] refers to "nisei" in sentence @xcite while "the show" in sentence @xcite refers to "the x-files" in sentence @xcite . with these anaphors as bridges, the information transport between entities across sentences can be facilitated. for instance, the connection between "fox mulder" and "the x-files" can be established via "the show". similarly, the connection between "nisei" and "chirs carter" can be established via "it". by leveraging the correspondence between these anaphors and entities, it is easier for the model to capture the semantic relations between "fox mulder" and "the x-files", "nisei" and "chirs carter", and finally promote extraction of the relation between "fox mulder" and "chirs carter". based on these observations above, we are motivated to develop a new framework to explicitly and jointly leverage the coreference and anaphora information in the document. we achieve this by introducing a new document graph which is constructed by considering all possible entity mentions and anaphors in the document. to distinguish the importance of nodes and edges in the graph, we define three types of edges to connect the nodes and further propose an attention-based graph convolutional neural network to dynamically learn the structure of the graph. with the proposed framework, the information transport on the graph is sufficiently modeled and an expressive entity presentation is extracted for final classification. following previous works @xcite @xcite , we also introduce evidence retrieval as an auxiliary task to help the model filter out irrelevant information. extensive experiments on docred and re-docred confirm the effectiveness of our proposed anaphor-assisted model. in summary, the main contributions of this paper are as follows: ## related work according to whether explicitly model the information interaction between entities, current works can be divided into two lines: transformer-based models and graph-based models. ## problem definition and each mention @xmath0. except for the mentions, there are other phrases, known as anaphors, that may refer to the entities. anaphors usually include pronouns like he, she, or it, and definite referents like the song or the show. our purpose is to use the information of anaphors to promote relation extraction. moreover, for each entity pair that has a valid relation @xmath1, a set of evidence sentences v h,@xmath2. ## conclusions in this paper, we present an approach that explicitly and jointly models coreference and anaphora information to effectively capture entities' internal structure and external interactions. moreover, we employ a dynamic algorithm for graph pruning and structural optimization, which requires minimal additional annotations. we also introduce evidence retrieval as an auxiliary task to enhance the encoder. empirical studies conducted on wellestablished benchmarks confirm the effectiveness of our proposed model.