U ni R el: Unified Representation and Interaction for Joint Relational Triple Extraction (2022)

Abstract: Relational triple extraction is challenging for its difficulty in capturing rich correlations between entities and relations. Existing works suffer from 1) heterogeneous representations of entities and relations, and 2) heterogeneous modeling of entity-entity interactions and entity-relation interactions. Therefore, the rich correlations are not fully exploited by existing works. In this paper, we propose UniRel to address these challenges. Specifically, we unify the representations of entities and relations by jointly encoding them within a concatenated natural language sequence, and unify the modeling of interactions with a proposed Interaction Map, which is built upon the off-the-shelf self-attention mechanism within any Transformer block. With comprehensive experiments on two popular relational triple extraction datasets, we demonstrate that UniRel is more effective and computationally efficient. The source code is available at https://github.com/wtangdev/UniRel.

## introduction relational triple extraction (rte) aims to identify entities and their semantic relations jointly. it extracts structured triples in the form @xmath0, and is a crucial task towards automatically constructing large-scale knowledge bases @xcite . early works try to solve rte tasks in a pipelined fashion involving two sub-tasks @xcite , where entities are firstly recognized, and relations are then assigned for each extracted entity pair. therefore, such methods fail to capture the implicit correlation between these two isolated sub-tasks and are thus prone to propagated errors @xcite . instead, many researchers @xcite @xcite seek * corresponding author. ## related work early works @xcite apply pipeline approaches that divide relational triple extraction into two isolated sub-tasks: first do named entity recognition to extract all entities, and then apply relation extraction to identify relations for each entity pair. however, these methods always suffer from error propagation for failing to capture the implicit correlation between these two isolated sub-tasks. to tackle these issues, recent researches focus on jointly extracting entities and relations. previous feature-based joint models @xcite @xcite require complex feature engineering and heavily depend on nlp tools. researchers propose neural network-based joint models to eliminate hand-craft features. @xcite propose a model to jointly learn entities and relations through parameter sharing. @xcite transform rte into a sequence tagging problem, which unifies the annotation role of entities and relations. despite their success, most models cannot deal with complex scenarios where one sentence con-sists of multiple overlapping relational triples sharing a single entity (singleentityoverlap, seo) or an entity pair (entitypairoverlap, epo). to handle the problem, researchers @xcite @xcite propose generative models that view triple as a token sequence. some works @xcite @xcite introduce methods to extract triples in one-stage but suffer from huge prediction space. other researchers @xcite @xcite @xcite decompose rte into different subtasks, but learn the interaction between sub-tasks only by input sharing, or falling into the cascade error. pfn @xcite proposes a partition filter network to fuse the task representation of ner and re, but still models entity-entity interactions and entity-relation interactions with separate modules. in this work, unirel unifies the modeling of the two kinds of interactions in one single interaction map to fully capture their interdependencies and is superior in computational efficiency. more recently, @xcite propose emrel that explicitly introduce relation representation to leverage the rich interactions across relations, entities, and context. however, it still suffers from heterogeneity between entities and the newly initialized embeddings of relations. some approaches @xcite @xcite introduce prompt-tuning to extract relation with semantic information. they transform the relation extraction task into a masked language modeling problem. however, such methods focus on the simple scenario of sentence-level relation classification without capturing the correlations between entities and relations. in terms of technical designs, ssan @xcite also delves into the selfattention layer within the transformer to model structural interaction, but it makes extra adaptions for standard self-attention mechanism and focuses on document-level re tasks. compared to these approaches, our work aims to extract relational triples in complex scenarios where rich intrinsic correlations exist between entities and relations. in this paper, we unify the representations and interactions to fully exploit the correlations to extract entities and relations jointly. ## methodology in this section, we present our model in detail. we first introduce the problem formulation in section 3.1. then, we introduce the unified representation and the unified interaction in section 3.2 and section 3.3, respectively. finally, we present the details of training and decoding in section 3.4. ## conclusion in this work, we propose unirel to fully leverage the rich correlations between entities and relations by resolving the heterogeneity. unified representation eliminates the representation's heterogeneity by encoding both entity and relation into meaningful sequence embeddings. unified interaction eliminates the interaction's heterogeneity by simultaneously modeling entity-entity interactions and entity-relation interactions in one single interaction map. unirel produces significant improvements over competitive baselines. we give a comprehensive analysis to further justify our design.