R ex UIE : A Recursive Method with Explicit Schema Instructor for Universal Information Extraction (2023)

Abstract: Universal Information Extraction (UIE) is an area of interest due to the challenges posed by varying targets, heterogeneous structures, and demand-specific schemas. Previous works have achieved success by unifying a few tasks, such as Named Entity Recognition (NER) and Relation Extraction (RE), while they fall short of being true UIE models particularly when extracting other general schemas such as quadruples and quintuples. Additionally, these models used an implicit structural schema instructor, which could lead to incorrect links between types, hindering the model’s generalization and performance in low-resource scenarios. In this paper, we redefine the true UIE with a formal formulation that covers almost all extraction schemas. To the best of our knowledge, we are the first to introduce UIE for any kind of schemas. In addition, we propose RexUIE, which is a Recursive Method with Explicit Schema Instructor for UIE. To avoid interference between different types, we reset the position ids and attention mask matrices. RexUIE shows strong performance under both full-shot and few-shot settings and achieves state-of-the-art results on the tasks of extracting complex schemas.

## introduction as a fundamental task of natural language understanding, information extraction (ie) has been widely studied, such as named entity recognition (ner), relation extraction (re), event extraction (ee), aspect-based sentiment analysis (absa), etc. however, the task-specific model structures hinder the sharing of knowledge and structure within the ie community. some recent studies attempt to model ner, re, and ee together to take advantage of the dependencies between subtasks. @xcite et al. (2021) modeled the cross-task dependency by graph neural networks. another successful attempt is universal information extraction (uie). lu et al. (2022) designed novel structural schema instructor (ssi) as inputs and structured extraction language (sel) as outputs, and proposed a unified text-to-structure generation framework based on t5-large. while lou et al. (2023) introduced three unified token linking operations and uniformly extracted substructures in parallel, which achieves new sotas on ie tasks. however, they have only achieved limited success by unifying a few tasks, such as named entity recognition (ner) and relation extraction (re), while ignoring extraction with more than 2 spans, such as quadruples and quintuples, thus fall short of being true uie models. as illustrated in @xcite , previous uie can only extract a pair of spans along with the relation between them, while ignoring other qualifying spans (such as location, time, etc.) that contain information related to the two entities and their relation. moreover, previous uie models are short of explicitly utilizing extraction schema to restrict outcomes. the relation work for provides a case wherein the subject and object are the person and organization entities, respectively. omitting an explicit schema can lead to spurious results, hindering the model's generalization and performance in resource-limited scenarios. in this paper, we redefine universal information extraction (uie) via a comprehensive formal framework that covers almost all extraction schemas. to the best of our knowledge, we are the first to introduce uie for any kind of schema. additionally, we introduce rexuie, which is a recursive method with explicit schema instructor for uie. rexuie recursively runs queries for all schema types and utilizes three unified tokenlinking operations to compute the results of each query. we construct an explicit schema instructor (esi), providing rich label semantic information to rexuie, and assuring the extraction results meet the constraints of the schema. esi and the text are concatenated to form the query. take figure 1 (b) as an example, given the extraction schema, rexuie firstly extracts "leonard parker" classified as a "person", then extracts "harvard university" classified as "university" coupled with the relation "educated at" according to the schema. thirdly, based on the extracted tuples ( "leonard parker", "person" ) and ( "harvard university", "educated at (university)" ), rexuie derives the span "phd" classified as an "academic degree". rexuie extracts spans recursively based on the schema, allowing extracting more than two spans such as quadruples and quintuples, rather than exclusively limited to pairs of spans and their relation. we pre-trained rexuie on a combination of supervised ner and re datasets, machine reading comprehension (mrc) datasets, as well as 3 million joint entity and relation extraction (jere) instances constructed via distant supervision. extensive experiments demonstrate that rexuie surpasses the state-of-the-art performance in various tasks and outperforms previous uie models in fewshot experiments. additionally, rexuie exhibits remarkable superiority in extracting quadruples and quintuples. the contributions of this paper can be summarized as follows: 1. we redefine true universal information extraction (uie) through a formal framework that covers almost all extraction schemas, rather than only extracting spans and pairs. extensive experiments demonstrate its remarkable effectiveness, as rexuie surpasses not only previous uie models and task-specific sotas in extracting entities, relations, quadruples and quintuples, but also outperforms large language models (such as chatgpt) under zero-shot setting. ## related work task-specific models for ie have been extensively studied, including named entity recognition @xcite @xcite , relation extraction @xcite @xcite , event extraction @xcite , and aspect-based sentiment analysis @xcite . some recent works attempted to jointly extract the entities, relations and events @xcite . oneie @xcite firstly extracted the globally optimal ie result as a graph from an input sentence, and incorporated global features to capture the cross-subtask and cross-instance interactions. fourie @xcite introduced an interaction graph between instances of the four tasks. @xcite proposed using consistent tagging schemes to model the extraction of entities and relationships. @xcite extended the idea to a unified matrix representation. tplinker formulates joint extraction as a token pair linking problem and introduces a novel handshaking tagging scheme that aligns the boundary tokens of entity pairs under each relation type. another approach that has been used to address joint information extraction with great success is the text-to-text language generation model. @xcite generated the linearized sequence of trigger words and argument in a text-to-text manner. @xcite purposed to jointly extract information by adding some general or task-specific prompts in front of the text. ## redefine universal information extraction while @xcite and @xcite proposed universal information extraction as methods of addressing ner, re, ee, and absa with a single unified model, their approaches were limited to only a few tasks and ignored schemas that contain more than two spans, such as quadruples and quintuples. hence, we redefine uie to cover the extraction of more general schemas. in our view, genuine uie extracts a collection of structured information from the text, with each item consisting of n spans @xmath0, s 2 , . . . , s n ] and n corresponding types @xmath1, t 2 , . . . , t n ]. the spans are extracted from the text, while the types are defined by a given schema. each pair of (s i , t i ) is the target to be extracted. formally, we propose to maximize the probabil-ity in equation 1. in equation 1 is represented by the prefix in the i-th query, and (s, t) i is calculated by the linking operations. ## experiments we conduct extensive experiments in this section under both supervised settings and few-shot settings. for implementation, we adopt debertav3-large @xcite as our text encoder, which also incorporates relative position information via disentangled attention, similar to our rotary module. we set the maximum token length to 512, and the maximum length of esi to 256. we split a query into sub-queries when the length of the esi is beyond the limit. detailed hyper-parameters are available in appendix b. due to space limitation, we have included the implementation details of some experiments in appendix c. ## conclusion in this paper, we introduce rexuie, a uie model using multiple prompts to recursively link types and spans based on an extraction schema. we redefine uie with the ability to extract schemas with any number of spans and types. through extensive experiments under both full-shot and fewshot settings, we demonstrate that rexuie outperforms state-of-the-art methods on a wide range of datasets, including quadruples and quintuples extraction. our empirical evaluation highlights the significance of explicit schemas and emphasizes that the ability to extract complex schemas cannot be substituted.