An Empirical Study of Pipeline vs. Joint approaches to Entity and Relation Extraction (2022)

Abstract: The Entity and Relation Extraction (ERE) task includes two basic sub-tasks: Named Entity Recognition and Relation Extraction. In the last several years, much work focused on joint approaches for the common perception that the pipeline approach suffers from the error propagation problem. Recent work reconsiders the pipeline scheme and shows that it can produce comparable results. To systematically study the pros and cons of these two schemes. We design and test eight pipeline and joint approaches to the ERE task. We find that with the same span representation methods, the best joint approach still outperforms the best pipeline model, but improperly designed joint approaches may have poor performance. We hope our work could shed some light on the pipeline-vs-joint debate of the ERE task and inspire further research.

## introduction the entity and relation extraction (ere) task aims to extract entities and their relations from unstructured text and is a fundamental task in the area of information extraction. there are two typical approaches to the ere task: one is the pipeline approach (chan and roth, 2011) consisting of two models for the two sub-tasks, named entity recognition (ner) and relation extraction (re), respectively. another is the joint approach that models the two sub-tasks jointly @xcite @xcite . pipeline approaches do not share any parameters between sub-tasks and decode sequentially. for joint approaches, one typical method is to share encoders across sub-tasks and performs pipelined decoding @xcite . another method uses joint inference in addition to shared encoders, for example, @xcite cast the ere task into a table-filling problem. among these joint approaches, some span-based joint approaches @xcite have different task-dividing strategy. span-based models embed each span for an input sentence and there are o(n 4 ) possible span pairs. to reduce the high complexity, previous span-based joint approaches pre-identify entity spans and then use a cross-task module for the entity and relation type deduction. to clarify the definitions, we define a purely joint approach as a method with only a cross-task module for sub-tasks, a purely pipelined approach has no cross-task module. it is generally believed that the pipeline approach suffers from the problem of error propagation, while the joint approach could leverage interactions between sub-tasks. however, recent research from @xcite shows that the feature confusion problem of the joint model may negate its benefits.there is also some work @xcite that disagree with their conclusion and propose a new state-of-the-art approach. however, these studies are based on different settings and hence cannot be directly compared. the debate on pipeline vs. joint approaches motivates us to perform a systematically empirical study. for a fair comparison, we design pipeline and joint approaches with similar settings. the pipeline works recently @xcite use span-based models to better leverage span-level features, we also adopt this setting. for previous span-based joint approaches @xcite divide the ner task into two sub-tasks, this leads to a second level of pipeline vs. joint dilemma which we also wish to investigate. specifically, we consider four sub-tasks for ere: entity identification (eid), entity classification (ecls), relation identification (rid) and relation classification (rcls). we design ten modules for these sub-tasks and connect them to build eight 1 . following the recent work @xcite , we use high-order inference to better exploit the correlation between sub-tasks. to experiment with the full joint spanbased approach, we use span pruner in addition to an entity pre-identifier for our joint approaches. with the high recall of the span pruner, our last approach a8 can be viewed as a full joint approach. our empirical study shows that with the same embedding method, the pipeline approach could achieve competitive results even compared to some joint approaches, but the full joint approach still outperforms all the pipeline approaches. ## our methods as mentioned above, there are ten modules solving different sub-task combinations and eight approaches. the former four approaches are fully pipelined, the final one is the fully joint approach and the others are approaches with cross-task modules. we first introduce all the modules and then describe the training and decoding processes of the approaches. we denote an input sequence with n tokens as @xmath0, x 2 , ..., x n }. m candidate spans of these tokens can be denoted as @xmath1, start(i) and end(i) represent the head and tail token indices of s i . the gold entity label set and the gold relation label set are represented as e and r respectively. ## conclusion in this paper, we empirically study several pipeline and joint approaches of the ere task. we find that pipeline approaches could achieve quite competitive results with some joint approaches, but with span pruning and high-order inference, the full joint model could still outperforms the pipeline approaches. we observe that if the tasks have strong correlations, a properly designed joint approach tends to have higher performance.