Global Context-enhanced Graph Convolutional Networks for Document-level Relation Extraction (2020)

Abstract: Document-level Relation Extraction (RE) is particularly challenging due to complex semantic interactions among multiple entities in a document. Among exiting approaches, Graph Convolutional Networks (GCN) is one of the most effective approaches for document-level RE. However, traditional GCN simply takes word nodes and adjacency matrix to represent graphs, which is difficult to establish direct connections between distant entity pairs. In this paper, we propose Global Context-enhanced Graph Convolutional Networks (GCGCN), a novel model which is composed of entities as nodes and context of entity pairs as edges between nodes to capture rich global context information of entities in a document. Two hierarchical blocks, Context-aware Attention Guided Graph Convolution (CAGGC) for partially connected graphs and Multi-head Attention Guided Graph Convolution (MAGGC) for fully connected graphs, could take progressively more global context into account. Meantime, we leverage a large-scale distantly supervised dataset to pre-train a GCGCN model with curriculum learning, which is then fine-tuned on the human-annotated dataset for further improving document-level RE performance. The experimental results on DocRED show that our model could effectively capture rich global context information in the document, leading to a state-of-the-art result. Our code is available at https://github.com/Huiweizhou/GCGCN .

## introduction the task of relation extraction (re) aims to detect semantic relations among entities in text, which plays an important role in many natural language processing applications such as knowledge discovery @xcite , and question answering @xcite . previous research on relation extraction mainly focuses on sentence-level, i.e., predicting relations between entity pairs in a given sentence. however, in real-world scenarios, many relations are expressed across sentences. the task of identifying these relations is named inter-sentence re. typically, intersentence relations occur in textual snippets with several sentences, such as documents. in a document, multiple mentions of the target entities in different sentences should be used for inter-sentence relation extraction, since their relations are expressed through the interactions of these mentions in the whole document. @xcite introduce a dataset called docred to accelerate the research on document-level re. take a document from docred as an example in figure 1 .there are many simple intra-sentence relations, such as ("ikwilalles met je delen", entry song, eurovision song contest 1990) in sentence 1, ("ikwilalles met je delen", performed after, "quand je terêve") and (céline carzo, performer, "quand je terêve") in sentence 5. only one evidence sentence is needed to predict these relation facts. however, the inter-sentence relation @xcite ) is supported by two evidence sentences (sentence 1 and sentence 5), and should be inferred based on the above three intra-sentence relations. since inter-sentence relations are inferred based on multiple relations, they are also called multi-hop relations. the prediction of inter-sentence relations is much more difficult than that of intra-sentence relations. previous work in document-level re employ hierarchical inference networks or graph convolutional networks (gcn) @xcite to extract features from local level to global level for multihop relational reasoning @xcite @xcite . how to construct a hierarchical inference network with gcn to encode rich global context information is crucial for document-level re. in this paper, we propose novel global context-enhanced graph convolutional networks (gcgcn) with entities as nodes and context of entity pairs as edges between nodes for document-level re. gcgcn is composed of two hierarchical inference blocks. the first block context-aware attention guided graph convolution (caggc) connects two entities if they co-occur in at least one sentence. all sentences where a pair of entities co-occur are represented as an edge between the two entity nodes. thus, caggc can learn an entity node representation based on context representations of all its mentions in the document and its neighbour nodes to encode local and global information. the second block multi-head attention guided graph convolution (maggc) applies multi-head attention @xcite to generate multiple fully connected edge-weighted graphs. maggc aims to enhance global context representations by connecting entity pairs in different sentences for multi-hop relational reasoning. furthermore, to reliably estimate the parameters of gcgcn model, we introduce a large-scale distantly supervised dataset in docred with curriculum learning to pre-train our model, and then fine-tune it on the human annotated dataset for improving the performance. in summary, we mainly make the following contributions: in recent years, graph convolutional networks (gcn) @xcite have attracted much attention in natural language processing @xcite @xcite , and have been approved effective for modelling sentence-level and document-level re. the most existing sentence-level gcn for relation extraction are built on dependency structures over input sentences. these methods construct graphs with words as nodes and dependency relations as edges between nodes @xcite . instead of dependency structures, @xcite construct a fully connected graph on unstructured texts with entities as nodes, and propagate relational information among nodes for multi-hop relational reasoning. as for document-level re, @xcite use attention guided gcn to transform the original dependency tree into a fully connected edge-weighted graph for encoding relations across sentences. besides syntactic dependency edges, @xcite introduce inter-sentence dependencies, such as coreference edges, adjacent sentence edges etc., into a document-level graph for inter-sentence relation extraction. @xcite construct a document-level graph with mention, entity and sentence as nodes, and dependencies between these nodes as edges to infer entity-to-entity relations. ## graph construction in this section, we will introduce how to construct a graph for each document as an input of graph convolutional networks. ## global context-enhanced graph convolutional networks global context-enhanced graph convolutional networks (gcgcn) has four main components: an encoder layer, a context-aware attention guided graph convolution (caggc) block, a multi-head attention guided graph convolution (maggc) block and a classification layer, as shown in figure 2 .