Modeling Multi-Granularity Hierarchical Features for Relation Extraction (2022)

Abstract: Relation extraction is a key task in Natural Language Processing (NLP), which aims to extract relations between entity pairs from given texts. Recently, relation extraction (RE) has achieved remarkable progress with the development of deep neural networks. Most existing research focuses on constructing explicit structured features using external knowledge such as knowledge graph and dependency tree. In this paper, we propose a novel method to extract multi-granularity features based solely on the original input sentences. We show that effective structured features can be attained even without external knowledge. Three kinds of features based on the input sentences are fully exploited, which are in entity mention level, segment level, and sentence level. All the three are jointly and hierarchically modeled. We evaluate our method on three public benchmarks: SemEval 2010 Task 8, Tacred, and Tacred Revisited. To verify the effectiveness, we apply our method to different encoders such as LSTM and BERT. Experimental results show that our method significantly outperforms existing state-of-the-art models that even use external knowledge. Extensive analyses demonstrate that the performance of our model is contributed by the capture of multi-granularity features and the model of their hierarchical structure.

## introduction relation extraction (re) is a fundamental task in natural language processing (nlp), which aims to extract relations between entity pairs from given plain texts. re is the cornerstone of many downstream nlp tasks, such as knowledge base construction @xcite ), question answering @xcite , and information extraction @xcite . * contribution during internship at tencent inc. ## methodology the structure of our model and details of each component is shown in figure 2 . we can see the overall architecture in the middle. it is divided into three components from bottom to top: 1) a text encoder which is employed to obtain text vector representations; 2) a multi-granularity hierarchical feature extractor which can exploit effective structured features from text representations; 3) a feature aggregation layer which aggregate previous multi-granularity features for relation prediction. in this section, we will introduce details of three components. firstly, we formalize the relation extraction task. let @xmath0, x 2 , ..., x n } be a sequence of input tokens, where x 0 = [cls] and x @xmath1. let s 1 = (i, j) and s 2 = (k, l) be pairs of entity indices. the indices in s 1 and s 2 delimit entities in x: [x i , . . . , x j-1 ] and [x k , . . . , x l-1 ]. our goal is to learn a function p (r) = f θ (x, s 1 , s 2 ), where @xmath2, which is marked by s1 and s2. r is a pre-defined relation set. ## conclusion and limitations in this paper, we analyze previous typical works and empirically focus on three granularity features: entity mention level, segment level and sentence level. based on the hierarchical structure between entity mention level and segment level feature, we propose a multi-granularity hierarchical feature extractor for relation extraction, which does not need any external knowledge or tools. we evaluate our method with different encoders and results on three public benchmarks show that our method can bring outstanding improvement for them. the structure of our model make it not easy to apply on multi-relation extraction tasks. in the future, we will focus on how to extend our method to longer input tasks and multi-relation extraction tasks (e.g. document level relation extraction). besides, we will also investigate what makes graph structure effective in relation extraction tasks and why our method can obtain better results than them.