U ni RE : A Unified Label Space for Entity Relation Extraction (2021)

Abstract: Many joint entity relation extraction models setup two separated label spaces for the two sub-tasks (i.e., entity detection and relation classification). We argue that this setting may hinder the information interaction between entities and relations. In this work, we propose to eliminate the different treatment on the two sub-tasks’ label spaces. The input of our model is a table containing all word pairs from a sentence. Entities and relations are represented by squares and rectangles in the table. We apply a unified classifier to predict each cell’s label, which unifies the learning of two sub-tasks. For testing, an effective (yet fast) approximate decoder is proposed for finding squares and rectangles from tables. Experiments on three benchmarks (ACE04, ACE05, SciERC) show that, using only half the number of parameters, our model achieves competitive accuracy with the best extractor, and is faster.

## introduction extracting structured information from plain texts is a long-lasting research topic in nlp. typically, it aims to recognize specific entities and relations for profiling the semantic of sentences. an example is shown in figure 1 , where a person entity "david perkins" and a geography entity "california" have a physical location relation phys. methods for detecting entities and relations can be categorized into pipeline models or joint models. in the pipeline setting, entity models and relation models are independent with disentangled feature spaces and output label spaces. in the joint setting, on the other hand, some parameter sharing of feature spaces @xcite expresses overlapped relations, e.g., the person entity "david perkins" participates in two relations, ("david perkins", "wife", per-soc) and ("david perkins", "california", phys). for every cell, a same biaffine model predicts its label. the joint decoder is set to find the best squares and rectangles. ## task definition given an input sentence @xmath0, x 2 , . . . , x |s| (x i is a word), this task is to extract a set of entities e and a set of relations r. an entity e is a span (e.span) with a pre-defined type e.@xmath1.g., per, gpe). the span is a continuous sequence of words. a relation r is a triplet (e 1 , e 2 , l), where e 1 , e 2 are two entities and @xmath2.g., the phys relation between per and gpe mentioned before). here y e , y r denote the set of possible entity types and relation types respectively. we formulate the joint entity relation extraction as a table filling task (multi-class classification between each word pair in sentence s), as shown in figure 1 . for the sentence s, we maintain a table t |s|×|s| . for each cell (i, j) in table t , we assign a label y i,@xmath3, where @xmath4. for each entity e, the label of corresponding cells y i,j (x @xmath5.span, x @xmath6.span) should be filled in e.type. for each relation @xmath7, e 2 , l), the label of corresponding cells y i,j (x @xmath8.span, x @xmath9.span) should be filled in l. 2 while others should be filled in ⊥. in the test phase, decoding entities and relations becomes a rectangle finding problem. note that solving this problem is not trivial, and we propose a simple but effective joint decoding algorithm to tackle this challenge. ## approach in this section, we first introduce our biaffine model for table filling task based on pre-trained language models (section 3.1). then we detail the main objective function of the table filling task (section 3.2) and some constraints which are imposed on the table in training stage (section 3.3). finally we present the joint decoding algorithm to extract entities and relations (section 3.4). figure 2 shows an overview of our model architecture. 3 ## related work entity relation extraction has been extensively studied over the decades. existing methods can be roughly divided into two categories according to the adopted label space. separate label spaces this category study this task as two separate sub-tasks: entity recognition and relation classification, which are defined in two separate label spaces. one early paradigm is the pipeline method @xcite that uses two independent models for two sub-tasks respectively. then joint method handles this task with an end-to-end model to explore more interaction between entities and relations. the most basic joint paradigm, parameter sharing @xcite , adopts two independent decoders based on a shared encoder. recent span-based models @xcite also use this paradigm. to enhance the connection of two decoders, many joint decoding algorithms are proposed, such as ilp-based joint decoder @xcite , joint mrt @xcite , gcn-based joint inference @xcite . actually, table filling method @xcite @xcite ) is a special case of parameter sharing in table structure. these joint models all focus on various joint algorithms but ignore the fact that they are essentially based on separate label spaces. unified label space this family of methods aims to unify two sub-tasks and tackle this task in a unified label space. entity relation extraction has been converted into a tagging problem @xcite , a transition-based parsing problem @xcite , and a generation problem with gold table intermediate table decoded table table" presents the gold label. "intermediate table" presents the biaffine model's prediction (choosing the label with the highest probability for each cell). "decoded table" presents the final results after decoding. seq2seq framework @xcite . we follow this trend and propose a new unified label space. we introduce a 2d table to tackle the overlapping relation problem in @xcite . also, our model is more versatile as not relying on complex expertise like @xcite , which requires external expert knowledge to design a complex transition system. ## conclusion in this work, we extract entities and relations in a unified label space to better mine the interaction between both sub-tasks. we propose a novel table that presents entities and relations as squares and rectangles. then this task can be performed in two simple steps: filling the table with our biaffine model and decoding entities and relations with our joint decoding algorithm. experiments on three benchmarks show the proposed method achieves not only state-of-the-art performance but also promising efficiency. algorithm 1 decoding algorithm input: probability tensor @xmath10: a set of entities e and a set of relations r 1: e @xmath11, @xmath12, @xmath13: p row ← p.view(n, n * |y|) 3: p col ← p.transpose(0, 1).view(n, n * |y|) 4: i ← 1 5: while @xmath14: d ← ||p row i -p row i+1 || 2 2 +||p col i -p col i+1 || 2 2 2 7: if @xmath15: e split .append(i) 9: end if 10: i ← i + 1 11: end while 12: e split .append(|s|) 13: i ← 1 14: for @xmath16: @xmath17:j,i:j,t) 16: if @xmath18: new e: e.@xmath19, j) and e.@xmath20: e.add(e) 19: