A Frustratingly Easy Approach for Entity and Relation Extraction (2021)

Abstract: End-to-end relation extraction aims to identify named entities and extract relations between them. Most recent work models these two subtasks jointly, either by casting them in one structured prediction framework, or performing multi-task learning through shared representations. In this work, we present a simple pipelined approach for entity and relation extraction, and establish the new state-of-the-art on standard benchmarks (ACE04, ACE05 and SciERC), obtaining a 1.7%-2.8% absolute improvement in relation F1 over previous joint models with the same pre-trained encoders. Our approach essentially builds on two independent encoders and merely uses the entity model to construct the input for the relation model. Through a series of careful examinations, we validate the importance of learning distinct contextual representations for entities and relations, fusing entity information early in the relation model, and incorporating global context. Finally, we also present an efficient approximation to our approach which requires only one pass of both entity and relation encoders at inference time, achieving an 8-16× speedup with a slight reduction in accuracy.

## introduction extracting entities and their relations from unstructured text is a fundamental problem in information extraction. this problem can be decomposed into two subtasks: named entity recognition @xcite and relation extraction @xcite . early work employed a pipelined approach, training one model to extract entities @xcite , and another model to classify relations between them @xcite @xcite . more recently, however, end-toend evaluations have been dominated by systems 1 our code and models are publicly available at https: //github.com/princeton-nlp/pure . in this work, we re-examine this problem and present a simple approach which learns two encoders built on top of deep pre-trained language models @xcite @xcite . the two models -which we refer them as to the entity model and relation model throughout the paper -are trained independently and the relation model only relies on the entity model to provide input features. our entity model builds on span-level representations and our relation model builds on contextual representations specific to a given pair of spans. despite its simplicity, we find this pipelined approach to be extremely effective: using the same pre-trained encoders, our model outperforms all previous joint models on three standard benchmarks: ace04, ace05 and scierc, advancing the previous state-of-the-art by 1.7%-2.8% absolute in relation f1. to better understand the effectiveness of this approach, we carry out a series of careful analyses. we observe that, (1) the contextual representations for the entity and relation models essentially capture distinct information, so sharing their representations hurts performance; (2) it is crucial to fuse the entity information (both boundary and type) at the input layer of the relation model; finally, one possible shortcoming of our approach is that we need to run our relation model ## related work traditionally, extracting relations between entities in text has been studied as two separate tasks: named entity recognition and relation extraction. in the last several years, there has been a surge of interest in developing models for joint extraction of entities and relations @xcite @xcite . we group existing joint models into two categories: structured prediction and multi-task learning: structured prediction structured prediction approaches cast the two tasks into one unified framework, although it can be formulated in various ways. @xcite propose an action-based system which identifies new entities as well as links to previous entities, @xcite ; wang and lu (2020) adopt a table-filling approach proposed in @xcite ; @xcite and @xcite employ sequence tagging-based approaches; @xcite and @xcite propose graph-based approaches to jointly predict entity and relation types; and, @xcite convert the task into a multi-turn question answering problem. all of these approaches need to tackle a global optimization problem and perform joint decoding at inference time, using beam search or reinforcement learning. multi-task learning this family of models essentially builds two separate models for entity recognition and relation extraction and optimizes them together through parameter sharing. @xcite propose to use a sequence tagging model for entity prediction and a tree-based lstm model for relation extraction. the two models share one lstm layer for contextualized word representations and they find sharing parameters improves performance (slightly) for both models. the approach of @xcite is similar except that they model relation classification as a multi-label head selection problem. note that these approaches still perform pipelined decoding: entities are first extracted and the relation model is applied on the predicted entities. the closest work to ours is dygie and dy-gie++ @xcite , which builds on recent span-based models for coreference resolution @xcite and semantic role labeling @xcite . the key idea of their approaches is to learn shared span representations between the two tasks and update span representations through dynamic graph propagation layers. a more recent work @xcite further extends dygie++ by incorporating global features based on cross-substask and cross-instance constraints. 2 our approach is much simpler and we will detail the differences in section 3.2 and explain why our model performs better. ## method in this section, we first formally define the problem of end-to-end relation extraction in section 3.1 and then detail our approach in section 3.2. finally, we present our approximation solution in section 3.3, which considerably improves the efficiency of our approach during inference. ## analysis despite its simple design and training paradigm, we have shown that our approach outperforms all previous joint models. in this section, we aim to take a deeper look and understand what contributes to its final performance. ## conclusion in this paper, we present a simple and effective approach for end-to-end relation extraction. our model learns two encoders for entity recognition and relation extraction independently and our experiments show that it outperforms previous stateof-the-art on three standard benchmarks considerably. we conduct extensive analyses to undertand the superior performance of our approach and validate the importance of learning distinct contextual representations for entities and relations and using entity information as input features for the relation model. we also propose an efficient approximation, obtaining a large speedup at inference time with a small reduction in accuracy. we hope that this simple model will serve as a very strong baseline and make us rethink the value of joint training in end-to-end relation extraction.