U ni EX : An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive Perspective (2023)

Abstract: We propose a new paradigm for universal information extraction (IE) that is compatible with any schema format and applicable to a list of IE tasks, such as named entity recognition, relation extraction, event extraction and sentiment analysis. Our approach converts the text-based IE tasks as the token-pair problem, which uniformly disassembles all extraction targets into joint span detection, classification and association problems with a unified extractive framework, namely UniEX. UniEX can synchronously encode schema-based prompt and textual information, and collaboratively learn the generalized knowledge from pre-defined information using the auto-encoder language models. We develop a traffine attention mechanism to integrate heterogeneous factors including tasks, labels and inside tokens, and obtain the extraction target via a scoring matrix. Experiment results show that UniEX can outperform generative universal IE models in terms of performance and inference-speed on 14 benchmarks IE datasets with the supervised setting. The state-of-the-art performance in low-resource scenarios also verifies the transferability and effectiveness of UniEX.

## introduction information extraction (ie) aims at automatically extracting structured information from unstructured textual sources, covering a wide range of subtasks such as named entity recognition, relation extraction, semantic role labeling, and sentiment analysis @xcite . however, the variety of subtasks build the isolation zones between each other and form their own dedicated models. fig 1 @xcite presents that the popular ie approaches handle structured extraction by the addition of task-specific layers on top of pretrained language models (lms) and a subsequent fine-tuning of the conjoined model @xcite @xcite . the isolated architectures and chaotic situation prevents enhancements from one task from being applied to another, which hinders the effective latent semantics sharing such as label names, and suffer from inductive bias in transfer learning @xcite . with powerful capabilities in knowledge sharing and semantic generalization, large-scale lms bring the opportunity to handle multiple ie tasks using a single framework. as shown in fig 1 (b), by developing sophisticated schema-based prompt and structural generation specification, the ie tasks can be transformed into text-to-text and text-to-structure formats via large-scale generative lms @xcite @xcite such as t5 @xcite . moreover, the universal ie frameworks can learn general knowledge from multi-source prompts, which is beneficial for perceiving unseen content in lowresource scenarios. despite their success, these generative frameworks suffer from their inherent problems, which limit their potential and performance in universal modeling. firstly, the schemabased prompt and contextual information are synthetically encoded for generating the target structure, which is not conducive to directly leveraging the position information among different tokens. secondly, the generative architecture utilizes the token-wise decoder to obtain the target structure, which is extremely time-consuming. the aforementioned issues prompt us to rethink the foundation of ie tasks. fundamentally, we discover that the extraction targets of different ie tasks involve the determination of semantic roles and semantic types, both of which can be converted into span formats by the correlation of the inside tokens in the passage. for instance, an entity type is the boundary detection and label classification of a semantic role, while a relation type can be regarded as the semantic association between specific semantic roles. from this perspective, the ie tasks can be decoded using a span-extractive framework, which can be uniformly decomposed as several atomic operations: i) span detection, which locates the boundaries of the mentioned semantic roles; ii) span classification, which recognizes the semantic types of the semantic roles; iii) span association, which establishes and measures the correlation between semantic roles to determine semantic types. according to the above observation, we propose a new paradigm for universal ie, called unified extraction model (uniex) as figure 1 (c ). specifically, we first introduce a rule-based transformation to bridge various extraction targets and unified input formats, which leverages task-specific labels with identifiers as the schema-based prompt to learn general ie knowledge. then, recent works @xcite state that the auto-encoder lms with bidirectional context representations are more suitable for natural language understanding. therefore, we employ bert-like lms to construct an extractive architecture for underlying semantic encoding. finally, inspired by the successful application of span-decoder and biaffine network to decode entity and relation with a scoring matrix @xcite @xcite , we introduce a triaffine attention mechanism for structural decoding, which jointly considers high-order interactions among multiple factors, including tasks, labels and inside tokens. each triaffine scoring matrix is assigned to a demand-specific prompt for obtaining span-extractive objectives. through extensive experiments on several challenging benchmarks of 4 main ie tasks (entity/relation/event/sentiment extraction), we demonstrate that compared with the state-of-the-art universal ie models and task-specific low-resource approaches, our uniex achieves a substantial improvement in performance and efficiency with supervised, few-shot and zero-shot settings. our main contributions are summarized as: label information label semantics is an important information source, which carries out the related meaning induced from the data @xcite @xcite . the l-tapnet @xcite introduces the collapsed dependency transfer mechanism to leverage the semantics of label names for few-shot tagging tasks. lsap @xcite improves the generalization and data efficiency of few-shot text classification by incorporating label semantics into the pre-training and fine-tuning phases of generative lms. together, these successful employments of label knowledge in low-resource setting motivates us to introduce label semantics into our unified inputs to handle few-shot and zero-shot scenarios. ## approaches generally, there are two main challenges in universally modeling different ie tasks via the extractive architecture. firstly, ie tasks are usually demand-driven, indicating that each pre-defined schema should correspond to the extraction of specific structural information. secondly, due to the diversity of ie tasks, we need to resolve appropriate structural formats from the output sequence to accommodate different target structures, such as entity, relation and event. in this section, we outline how the uniex exploits a shared underlying semantic encoder to learn the prompt and text knowledge jointly, and conduct various ie tasks in a unified text-to-structure architecture via the triaffine attention mechanism. ## conclusion in this paper, we introduce a new paradigm for universal ie by converting all ie tasks into joint span detection, classification and association problems with a unified extractive framework. uniex collaboratively learns the generalized knowledge from schema-based prompts and controls the correspondence between schema and extraction targets via the triaffine attention mechanism. experiments on both supervised setting and low-resource scenarios verify the transferability and effectiveness of our approaches.