Relation Extraction with Word Graphs from N-grams (2021)

Abstract: Most recent studies for relation extraction (RE) leverage the dependency tree of the input sentence to incorporate syntax-driven contextual information to improve model performance, with little attention paid to the limitation where high-quality dependency parsers in most cases unavailable, especially for in-domain scenarios. To address this limitation, in this paper, we propose attentive graph convolutional networks (A-GCN) to improve neural RE methods with an unsupervised manner to build the context graph, without relying on the existence of a dependency parser. Specifically, we construct the graph from n-grams extracted from a lexicon built from pointwise mutual information (PMI) and apply attention over the graph. Therefore, different word pairs from the contexts within and across n-grams are weighted in the model and facilitate RE accordingly. Experimental results with further analyses on two English benchmark datasets for RE demonstrate the effectiveness of our approach, where state-of-the-art performance is observed on both datasets.

## introduction recently, neural models @xcite @xcite @xcite @xcite @xcite with powerful encoders (e.g., transformers) have achieved promising performance for relation extraction (re), for the reason that the encoders are superior in capturing contextual information and thus allow re systems to better understand the text and correctly identify the relations between entities in the given text. to further improve the ability of re models to understand the context, many studies @xcite @xcite @xcite @xcite leverage extra resources, such as auto-parsed word dependency, through graph-based approaches, e.g., graph convolutional networks (gcn). in doing so, such studies learn the long-distance connections among useful words from the dependency tree and extract relations between entity pairs accordingly. however, in doing so, dependency parsers required by their approaches are not always available. in this dilemma, one needs another way to extract useful word connections to build the graph for gcn-based models, whereas limited attentions from previous studies have been paid to this alternative. in this paper, we propose attentive gcn (a-gcn) for relation extraction, where its input graph is built based on n-grams extracted with unsupervised methods i.e., pointwise mutual information (pmi), rather than an existing dependency parser. specifically, two types of edges in the graph are introduced to model word connections within and across n-grams and an attention mechanism is applied to gcn to weight these edges. in doing so, different contextual information are discriminatively learned to facilitate re without requiring any external resources. we evaluate our approach on two english benchmark datasets, i.e., ace2005en and semeval 2010 task 8, where the results demonstrate the effectiveness of our approach with stateof-the-art performance observed on both datasets. ## the approach re is often treated as a classification task, where the input is a sentence @xmath0, • • • , x n with two given entities (denoted by e 1 and e 2 ) in it. our approach follows this paradigm and uses a variant of graph neural model, i.e., attentive gcn (a-gcn), to incorporate word pair information and predicts the relation r between e 1 and where g x is the graph built based on n-grams in x , r is the relation type set; p computes the probability of a particular relation type @xmath1.e., x , g x , e 1 , and e 2 ), and r is the prediction of our a-gcn model. in the following text, we firstly elaborate how we construct the graph based on n-grams, and then illustrate the architecture of the a-gcn model for re. ## conclusion in this paper, we propose a-gcn to leverage information of word pairs for re, where the graph for a-gcn is built from n-grams without relying models ace05 semeval xu et al. (2015) -83.7 wang et al. (2016) -88.0 zhou et al. (2016) -84.0 zhang et al. (2018) -84.8 christopoulou et al. (2018) 64.2 -ye et al. (2019) 68.9 -wu and he (2019) (bert) -89.2 soares et al. (2019) (bert) -89.5 mandya et al. (2020) -85.9 yu et al. (2020) -86.4 wang et al. (2020) 66.7 -wang and lu (2020) 67.6 -ours (bert + a-gcn (n-gram)) 77.72 89.67