A Partition Filter Network for Joint Entity and Relation Extraction (2021)

Abstract: In joint entity and relation extraction, existing work either sequentially encode task-specific features, leading to an imbalance in inter-task feature interaction where features extracted later have no direct contact with those that come first. Or they encode entity features and relation features in a parallel manner, meaning that feature representation learning for each task is largely independent of each other except for input sharing. We propose a partition filter network to model two-way interaction between tasks properly, where feature encoding is decomposed into two steps: partition and filter. In our encoder, we leverage two gates: entity and relation gate, to segment neurons into two task partitions and one shared partition. The shared partition represents inter-task information valuable to both tasks and is evenly shared across two tasks to ensure proper two-way interaction. The task partitions represent intra-task information and are formed through concerted efforts of both gates, making sure that encoding of task-specific features is dependent upon each other. Experiment results on six public datasets show that our model performs significantly better than previous approaches. In addition, contrary to what previous work has claimed, our auxiliary experiments suggest that relation prediction is contributory to named entity prediction in a non-negligible way. The source code can be found at https://github.com/Coopercoppers/PFN .

## introduction joint entity and relation extraction intend to simultaneously extract entity and relation facts in the given text to form relational triples as @xcite . the extracted information provides a supplement to many studies, such as knowledge graph construction @xcite , question * * corresponding author. according to their differences in encoding taskspecific features, most of the existing methods can be divided into two categories: sequential encoding and parallel encoding. in sequential encoding, task-specific features are generated sequentially, which means features extracted first are not affected by those that are extracted later. @xcite and @xcite are typical examples of this category. their methods extract features for different tasks in a predefined order. in parallel encoding, task-specific features are generated independently using shared input. compared with sequential encoding, models build on this scheme do not need to worry about the implication of encoding order. for example, @xcite encodes entity and relation information separately using common features derived from their gcn encoder. since both taskspecific features are extracted through isolated submodules, this approach falls into the category of parallel encoding. however, both encoding designs above fail to model two-way interaction between ner and re tasks properly. in sequential encoding, interaction is only unidirectional with a specified order, resulting in different amount of information exposed to ner and re task. in parallel encoding, although encoding order is no longer a concern, interaction is only present in input sharing. considering adding two-way interaction in feature encoding, we adopt an alternative encoding design: joint encoding. this design encodes task-specific features jointly with a single encoder where there should exist some mutual section for inter-task communication. in this work, we instantiate joint encoding with a partition filter encoder. our encoder first sorts and partitions each neuron according to its contribution to individual tasks with entity and relation gates. during this process, two task partitions and one shared partition are formed (see figure 1 ). then individual task partitions and shared partition are combined to generate task-specific features, filtering out irrelevant information stored in the opposite task partition. task interaction in our encoder is achieved in two ways: first, the partitions, especially the taskspecific ones, are formed through concerted efforts of entity and relation gates, allowing for interaction between the formation of entity and relation features determined by these partitions. second, the shared partition, which represents information useful to both task, is equally accessible to the formation of both task-specific features, ensuring balanced two-way interaction. the contributions of our work are summarized below: this method is capable of encoding taskspecific features and guarantees proper twoway interaction between ner and re. in recent years, joint entity and relation extraction approaches have been focusing on tackling triple overlapping problem and modelling task interaction. solutions to these issues have been explored in recent works @xcite @xcite @xcite . the triple overlapping problem refers to triples sharing the same entity (seo, i.e. singleentityoverlap) or entities (epo, i.e. entitypairoverlap). for example, in "adam and joe were born in the usa", since triples (adam, birthplace, usa) and (joe, birthplace, usa) share only one entity "usa", they should be categorized as seo triples; or in "adam was born in the usa and lived there ever since", triples (adam, birthplace, usa) and (adam, residence, usa) share both entities at the same time, thus should be categorized as epo triples. generally, there are two ways in tackling the problem. one is through generative methods like seq2seq @xcite where entity and relation mentions can be decoded multiple times in output sequence, another is by modeling each relation separately with sequences @xcite , graphs @xcite or tables @xcite . our method uses relation-specific tables @xcite to handle each relation separately. task interaction modeling, however, has not been well handled by most of the previous work. in some of the previous approaches, task interaction is achieved with entity and relation prediction sharing the same features @xcite . this could be problematic as information about entity and relation could sometimes be contradictory. also, as models that use sequential encoding @xcite @xcite or parallel encoding @xcite lack proper two-way interaction in feature extraction, predictions made on these features suffer the problem of improper interaction. in our work, the partition filter encoder is built on joint encoding and is capable of handling communication of inter-task information more appropriately to avoid the problem of sequential and parallel encoding (exposure bias and insufficient interaction), while keeping intra-task information away from the opposite task to mitigate the problem of negative transfer between the tasks. ## problem formulation our framework split up joint entity and relation extraction into two sub-tasks: ner and re. formally, given an input sequence @xmath0, . . . , w l } with l tokens, w i denotes the i-th token in sequence s. for ner, we aim to extract all typed entities whose set is denoted as s, where ⟨w i , e, w j ⟩ ∈ s signifies that token w i and w j are the start and end token of an entity typed @xmath1. e represents the set of entity types. concerning re, the goal is to identify all head-only triples whose set is denoted as t , each triple ⟨w i , r, w j ⟩ ∈ t indicates that tokens w i and w j are the corresponding start token of subject and object entity with relation @xmath2. r represents the set of relation types. combining the results from both ner and re, we should be able to extract relational triples with complete entity spans. ## model we describe our model design in this section. our model consists of a partition filter encoder and two task units, namely ner unit and re unit. the partition filter encoder is used to generate taskspecific features, which will be sent to task units as input for entity and relation prediction. we will discuss each component in detail in the following three sub-sections. ## effects of relation signal on entity recognition it is a widely accepted fact that entity recognition helps in predicting relations, but the effect of relation signals on entity prediction remains divergent among researchers. through two auxiliary experiments, we find that the absence of relation signals has a considerable bearing on entity recognition.