Dependency-driven Relation Extraction with Attentive Graph Convolutional Networks (2021)

Abstract: Syntactic information, especially dependency trees, has been widely used by existing studies to improve relation extraction with better semantic guidance for analyzing the context information associated with the given entities. However, most existing studies suffer from the noise in the dependency trees, especially when they are automatically generated, so that intensively leveraging dependency information may introduce confusions to relation classification and necessary pruning is of great importance in this task. In this paper, we propose a dependency-driven approach for relation extraction with attentive graph convolutional networks (A-GCN). In this approach, an attention mechanism upon graph convolutional networks is applied to different contextual words in the dependency tree obtained from an off-the-shelf dependency parser, to distinguish the importance of different word dependencies. Consider that dependency types among words also contain important contextual guidance, which is potentially helpful for relation extraction, we also include the type information in A-GCN modeling. Experimental results on two English benchmark datasets demonstrate the effectiveness of our A-GCN, which outperforms previous studies and achieves state-of-the-art performance on both datasets.

## introduction relation extraction (re), which aims to detect the relationship between entity mentions from raw text, is one of the most important tasks in information extraction and retrieval, and plays a crucial role in supporting many downstream natural language processing (nlp) applications such as text mining @xcite , sentiment analysis (sun figure 1 : an illustration of noises in the dependency tree that can hurt relation extraction, where the word dependency connected in between "pumpkin mixture" and "bowl" (whose relation is content-container) may introduce confusion to the system when the object is to predict the relation between "milk" and "pumpkin mixture" (whose relation is entity-destination). recently, neural re methods @xcite @xcite @xcite @xcite with powerful encoders (such as cnn, rnn, and transformers) have significantly improved model performance for re without requiring any elaborately designed systems or manually constructed features. these methods are superior in capturing contextual information and thus enable re systems to better understand the text and identify relations between entities in the given text. adopting neural models to help re is not only straightforward and effective, but is also expected to incorporate more diverse and informative knowledge into re systems. among all different knowledge sources, syntactic information, especially the dependency trees, have been demonstrated to be beneficial in many studies @xcite @xcite because they provide long-distance word connections between useful words and thus accordingly guide the system to better extract relations between entity pairs. however, intensively leveraging dependency information may not always lead to good re performance, because the noise in the dependency tree can potentially introduce confusions to relation classification @xcite , especially when those trees are automatically generated. for example, figure 1 shows an example sentence with its dependency tree, where the dependency connection between "pumpkin mixture" and "bowl" may introduce noise when the object is to predict the relation between "milk" and "pumpkin mixture". therefore, previous studies have always required necessary pruning strategies before encoding the dependency information through a particular model such as lstm @xcite or graph convolutional networks (gcn) @xcite . because fixed pruning strategies are not guaranteed to result in a sub-tree with all important contextual information included and with all noise filtered out, it is necessary to design an appropriate way for distinguishing the noise in the dependency tree and modelling them accordingly. in this paper, we propose a dependency-driven neural approach for re, where attentive graph neural network (a-gcn) is proposed to distinguish the important contextual information for this task. furthermore, given that the dependency types (e.g., nominal subject) that associate with dependency connections are also potentially useful for re since they contain the syntactic instruction among connected words, we further improve a-gcn by introducing type information into it. specifically, we first obtain the dependency tree of an input sentence from an off-the-shelf toolkit, then build the graph over the dependency tree, and assign different weights to different labeled dependency connections between any two words, with the weights computed based on the connections and their dependency types, lastly predict relations by the a-gcn according to the learned weights. in doing so, not only is a-gcn able to distinguish important contextual information from dependency trees and leverage them accordingly, such that reliance on pruning strategies is unnecessary, but a-gcn can also leverage the dependency type information that is omitted by most previous studies (in particular, the studies that also use attention mechanism @xcite ). experimental results on two english benchmark datasets, i.e., ace2005en and se-meval 2010 task 8, demonstrate the effectiveness of our approach to re through a-gcn equipped with dependency type information. state-of-the-art performance is observed on both datasets. ## the proposed approach re is conventionally performed as a typical classification task. our approach follows this paradigm by using a-gcn and incorporates dependency information to improve model performance, where the overall architecture of our model is illustrated in figure 2 . specifically, given an unstructured input sentence @xmath0, • • • , x n with n words and let e 1 and e 2 denote two entities in x , our approach predicts the relation b r between e 1 and ## related work recently, neural networks with integrating external knowledge or resources play important roles in re because of their superiority in better capturing contextual information @xcite . particularly, as one kind of such knowledge, dependency parses show their effectiveness in supporting re for its ability in capturing long-distance word relations @xcite . however, intensively leveraging dependency information could introduce confusions to re @xcite so that necessary pruning is required to alleviate this problem. e.g., @xcite proposed to use the connections along the shortest dependency path between the two entities and apply lstm to model them; @xcite proposed to prune the original dependency tree into the lowest common ancestor subtree. however, these pruning strategies are either too aggressive or modest, so that the resulted graph might lose some important contexts or filled with more noise. @xcite adopted gcn to model the dependencies and proposed a trade-off pruning strategy in between @xcite and @xcite . besides, there are other graphbased models for re that utilize layers of multihead attentions @xcite , dynamic pruning @xcite , and additional attention layers @xcite to encode dependency trees. compared with the aforementioned methods, especially the graph-based ones, our approach offers an alternative to enhance re with a-gcn by using attention mechanism and dependency type, which are effective and efficient improvement to standard gcn without requiring complicated model design. ## conclusion in this paper, we propose a-gcn to leverage dependency information for relation extraction, where an attention mechanism is applied to dependency connections to applying weighting on both connections and types so as to better distinguish the important dependency information and leverage them accordingly. in doing so, a-gcn is able to dynamically learn from different depen-dency connections so that less-informative dependencies are smartly pruned. experimental results and analyses on two english benchmark datasets for relation extraction demonstrate the effectiveness of our approach, especially for entities with long word-sequence distances, where state-of-theart performance is obtained on both datasets.