S et GNER : General Named Entity Recognition as Entity Set Generation (2022)

Abstract: Recently, joint recognition of flat, nested and discontinuous entities has received increasing attention. Motivated by the observation that the target output of NER is essentially a set of sequences, we propose a novel entity set generation framework for general NER scenes in this paper. Different from sequence-to-sequence NER methods, our method does not force the entities to be generated in a predefined order and can get rid of the problem of error propagation and inefficient decoding. Distinguished from the set-prediction NER framework, our method treats each entity as a sequence and is capable of recognizing discontinuous mentions. Given an input sentence, the model first encodes the sentence in word-level and detects potential entity mentions based on the encoder’s output, then reconstructs entity mentions from the detected entity heads in parallel. To let the encoder of our model capture better right-to-left semantic structure, we also propose an auxiliary Inverse Generation Training task. Extensive experiments show that our model (w/o. Inverse Generation Training) outperforms state-of-the-art generative NER models by a large margin on two discontinuous NER datasets, two nested NER datasets and one flat NER dataset. Besides, the auxiliary Inverse Generation Training task is found to further improve the model’s performance on the five datasets.

## introduction named entity recognition (ner) is a fundamental task in the field of information extraction and has played an important role in the development of natural language processing. there exist three wellstudied subtasks of ner, i.e. flat ner, nested ner and discontinuous ner, as illustrated in figure 1 . recently, researchers have grown more interest in tackling the three subtasks jointly, which we refer to as general ner @xcite figure 1 : examples of the discontinuous / nested / flat ner. it can be observed that entities may overlap with each other in general ner scenes and these overlapped entities are inherently unordered. hence, instead of generating entities sequentially in a predefined order, it is more suitable to generate the set of entities concurrently. @xcite . existing frameworks for general ner fall into three categories: in this paper, we abandon the linear design of target entity sequence adopted by previous generative ner methods, and come up with a novel entity set generation framework, setgner. given an input sentence, the framework first encodes the sentence and detects potential entity mentions based on the encoder's output, then utilizes detected entity heads as a set of initial sequences to reconstruct the set of target entities. note that, this procedure is similar to the way human beings perform ner. while reading, a human attends to potential mentions of entities subconsciously. when encountering a plausible start of entities, the mind will activate a cluster of neurons for it @xcite , which consistently collects words related to it until all entities starting with it are recollected. these clusters of neurons just function as a distributed decoder that generates a set of entities in parallel. the distributed nature of our decoding scheme also brings remarkable inference speed-up. with a decoding time complexity of o(n l), where n is the sentence length and l is the average entity length, our model is efficient enough for both offline and online applications. to correctly recognize the boundary of an entity, right-to-left semantic structure is as important as left-to-right semantic structure, which is ignored by previous generative ner research. inspired by this, we additionally propose an auxiliary learning task -inverse generation training. guided by the inverse generation loss, the encoder can get more familiar with the right-to-left semantic structure of entity mentions, making it easier for the decoder to generate complete entity mentions. note that, our work is different from previous work proposed to address the problems faced with generative ner models. in @xcite , a sequence-to-set network is proposed to predict the set of target entities in a sentence. however, it assumes that each entity is a span and cannot handle the recognition of discontinuous mentions. in contrast, our model treats each entity as a sequence and can naturally tackle the discontinuous mentions. @xcite analyze two kinds of incorrect bias in the seq2seq ner models, i.e. pre-context confounder (when generating a word of an entity, a model can be affected by pre-generated words that has no causal relation with the word to be generated), entity-order confounder, and propose two data augmentation methods to address them. however, their model is still trained to generate entities sequentially after all. to sum up, our main contributions include: ## problem formulation we uniformly formulate the task of recognizing flat / nested / discontinuous entities as a pointer-based entity set generation problem. a pointer is used for copying a source word or a class tag or a special tag into target sequence. there are three special tags: ⟨∅⟩, indicating no-entityfound; ⟨ ⟩, indicating fragment of entity is found; ⟨/s⟩, indicating the end of the generated sequence. suppose the number of entity classes is c. we first define the pointer to class tag t i (0 ≤ @xmath0, the pointers to special tags ⟨∅⟩, ⟨ ⟩, ⟨/s⟩ as ptr(⟨∅⟩) = c, ptr(⟨ ⟩) = c + 1, ptr(⟨/s⟩) = c + 2. given a sentence with n words [w 0 , ..., w n -1 ], the pointer to word w j is then defined as: ptr(w j ) = c + 3 + j to simplify annotation, we generalize the ptr(•) operation to sequences of words / tags, i.e. ptr( [•] ) = [ ptr(•) ]. the target output is the set of entity pointer sequences ptr(e i ) m @xmath1, where m is the number of entities in the input sentence, e i is the i-th entity consisting of l words w e i 0 , w e i 1 , ..., w e i l-1 . and the pointer sequence for entity e i is defined as: ptr(e i ) = ptr([w e i 0 , w e i 1 , ..., w e i l-1 , t (e i ), ⟨/s⟩]) where t (e i ) is the class tag of e i , ⟨/s⟩ indicates the end of generated sequence. under this definition, each head word of entity is associated with a target sequence or multiple target sequences (when there are multiple entities starting with the head word). we additionally appoint a target sequence ptr([w j , ⟨∅⟩, ⟨/s⟩]) to word w j , if w j does not belong to any entity; or ptr([w j , ⟨ ⟩, ⟨/s⟩]) if w j is not an head word but a fragment of entity. ## method as shown in figure 2 , the proposed model consists of a word-level encoder, a mention detector and a parallel generator based on copying mechanism. we train it with a combination of generation loss, occurrence detection loss and entity part classification loss, and leverage an adaptive beam-search scheme during inference. ## conclusion we observe that existing generative ner models suffer from the entity order confounder and faces the problems of error propagation and slow inference speed. to address this, a novel entity set generation framework for general ner is proposed in this paper. we also propose to train our model with an auxiliary inverse generation task that helps the encoder learn the right-to-left semantic structure. experiments on five datasets prove the effectiveness of our methods.