Graph Convolution over Multiple Dependency Sub-graphs for Relation Extraction (2020)

Abstract: We propose a contextualised graph convolution network over multiple dependency-based sub-graphs for relation extraction. A novel method to construct multiple sub-graphs using words in shortest dependency path and words linked to entities in the dependency parse is proposed. Graph convolution operation is performed over the resulting multiple sub-graphs to obtain more informative features useful for relation extraction. Our experimental results show that the proposed method achieves superior performance over the existing GCN-based models achieving state-of-the-art performance on cross-sentence n-ary relation extraction dataset and SemEval 2010 Task 8 sentence-level relation extraction dataset. Our model also achieves a comparable performance to the SoTA on the TACRED dataset.

## introduction in recent times, graph convolutional network (gcn) @xcite based models such as contextualised gcn (c-gcn) @xcite and attention guided gcn (aggcn) @xcite are shown to be useful for relation extraction. the c-gcn model employs pruned dependency trees obtained using a path-centric pruning distance to filter irrelevant nodes from sentence graph to aid in relation extraction. experiments with c-gcn model shows that the performance of the c-gcn model significantly decreases with the inclusion of all nodes in the sentence graph obtained from the dependency graph @xcite , thus making it important to use a right pruning distance to achieve optimum performance. the aggcn model @xcite on the other hand, instead of using pruned dependency trees, considers the full dependency tree with a self-attention mechanism to learn node representation useful for relation extraction. in contrast to these gcn-based models using a single graph, we propose in this paper to use multiple sub-graphs constructed from dependency tree to represent a sentence for relation extraction. given the decrease in performance of c-gcn with an increase in the size of graph @xcite , we hypothesise that using multiple sub-graphs in place of a single graph can boost performance of gcn-based models for relation extraction. in comparison to the aggcn model @xcite , our proposed model simply considers smaller graphs associated with entities to learn representation for important nodes in the sentence. further, while the pruning strategy used by the c-gcn model @xcite is applicable for sentence-level relation extraction, it is not suitable for cross-sentence n-ary relation extraction as there could be more than two entities across multiple sentences. the challenges arise in obtaining a single dependency path in the lca sub-tree connecting n-ary entities across sentences for cross-sentence relation extraction. previous work on cross-sentence relation extraction task @xcite @xcite have largely made use of full dependency graph with additional features such as adjacent words and co-reference links @xcite . the usage of full dependency tree significantly increases network parameters, requiring high computational power. in contrast to these works on n-ary relation extraction, we propose in this paper a novel method to construct sub-graphs using connections between entities and root terms in dependency tree across sentences. our approach described in this paper significantly reduces the number of nodes in a graph and also at the same time helps in achieving higher performance for cross-sentence n-ary relation extraction. more specifically, the key contributions of this paper are: @xcite and semeval 2010 task 8 dataset @xcite to outperform c-gcn and aggcn models. we also show that c-gcn-mg achieves comparable performance against c-gcn and aggcn models on tacred dataset @xcite . ## related work gcns have been successfully applied for various information retrieval tasks in different areas such as bioinformatics @xcite , chemoinformatics @xcite , social network analysis @xcite , urban computing @xcite and natural language processing @xcite . with specific application to relation extraction, several studies have examined the use of graph-based mechanisms to improve relation extraction. for example, a combination of dependency graphs, propbank and framenet based features and surface-level lexical features were successfully used with support vector machines to improve relation extraction @xcite . sequence-based neural network models such as recurrent neural networks (rnns), lstms, bilstms have been used with graph-based features for relation extraction @xcite . shortest dependency path (sdp) between entities combined with rnns are also shown to be useful for relation extraction @xcite . further, word sequence and dependency tree substructures are combined to jointly model entity and relation extraction @xcite . more recently, graph-based lstm networks have been shown particularly useful in the context of n-ary cross-sentence relation extraction @xcite . gcn-based models such c-gcn @xcite and aggcn @xcite are shown to achieve sota performance on sentence-level and cross-sentence relation extraction. a combined neural network model that brings together the usefulness of lstm networks in learning from longer sequences and cnns to capture salient features is also proposed cross-sentence n-ary relation extraction @xcite . the main focus of this paper is to examine the use of multiple sub-graphs along with graph convolution models for relation extraction. in contrast to previous gcn-based models for relation extraction such as c-gcn @xcite and aggcn @xcite which use a single graph to learn node representations, we propose in this paper a c-gcn model that uses multiple sub-graphs to learn a richer node representation that helps in relation extraction. for this purpose, we propose a novel method to obtain multiple sub-graphs from dependency parse trees of a given sentence. to the best knowledge of the authors, this is the first study which effectively combines gcns with multiple sub-graphs for relation extraction and shows that such a strategy is useful for achieving sota performance for relation extraction. accordingly, multiple sub-graphs (as shown in figure 1 ) are constructed to represent a sentence, which includes (a) graph using nodes in sdp between two entities "knowledge" and "recruits" (figure 1 (ii)); and (b) graph using nodes linked to entity mentions "knowledge" (e 1 ) (figure 1 (iii)) and entity mention "recruits" (e 2 ) (figure 1 (iv)). the sdp is defined as the dependency path in the lca sub-tree, connecting the two entities. figure 1 (i) shows partial the dependency tree for example sentence 1. ## c-gcn over multiple sub-graphs in this section, we formally describe the problem and explain the architecture of c-gcn-mg model.