NAG - NER : a Unified Non-Autoregressive Generation Framework for Various NER Tasks (2023)

Abstract: Recently, the recognition of flat, nested, and discontinuous entities by a unified generative model framework has received increasing attention both in the research field and industry. However, the current generative NER methods force the entities to be generated in a predefined order, suffering from error propagation and inefficient decoding. In this work, we propose a unified non-autoregressive generation (NAG) framework for general NER tasks, referred to as NAG-NER. First, we propose to generate entities as a set instead of a sequence, avoiding error propagation. Second, we propose incorporating NAG in NER tasks for efficient decoding by treating each entity as a target sequence. Third, to enhance the generation performances of the NAG decoder, we employ the NAG encoder to detect potential entity mentions. Extensive experiments show that our NAG-NER model outperforms the state-of-the-art generative NER models on three benchmark NER datasets of different types and two of our proprietary NER tasks.\footnote{Code will be publicly available to the research community upon acceptance.}

## introduction named entity recognition (ner) is a fundamental task in the field of information extraction. it is the basic task for many natural language processing applications like dialogue systems, document analysis, and search engines. currently, ner tasks can be divided into three subtasks @xcite , i.e., flat ner, nested ner, and discontinuous ner, as illustrated in figure 1 . recently, researchers have grown interested in tackling the three subtasks via a unified model architecture, which we refer to as general ner models @xcite @xcite . existing literature for general ner models fall into the following three categories: (1) span-based models @xcite figure 1 : examples of the discontinuous / nested / flat ner. @xcite ; (2) models based on carefully designed data structures like hyper-graphs and shift-reduce parsers @xcite ; (3) sequence-to-sequence (seq2seq) models @xcite . among the three branches of literature, the seq2seq models @xcite have achieved sota performances. however, they organize target entities into a single sequence according to a predetermined order. this setting is against the intuition that the target entities are essentially an unordered set and results in an incorrect bias (entity-order confounder) to the model @xcite . in addition, sequentially generating target entities suffers from two disadvantages: (1) low inference speed due to autoregressive decoding; (2) error propagation, i.e., errors generated by the previous steps could misguide the current and future generation steps. in this paper, we propose a non-autoregressive generation (nag) framework for named entity recognition, nag-ner (as depicted in figure 2 ). given an input sentence, the framework first encodes the sentence and detects where and how many entities will start at each token of the input sentence. then, it asks the decoder to generate the set of targeted entities accordingly. we conducted extensive experiments on three benchmark datasets (cadec, ace2004, conll03) and two proprietary datasets we developed (referred to as cme lower right: during inference, the encoder first propose the entity starting words and #entities starting from these words, and then the decoder will generate corresponding entity results. and qer). the results validate that our method can perform comparably to or outperform the previous generative ner models while achieving significant speedups. to summarize, our main contributions include the following: ## related work due to limited length, we include the related works of nag in appendix a. we also include a preliminary introduction of nag in appendix b. ## methods in detail, we formally introduce our framework (figure 2 ). we uniformly formulate the task of recognizing flat, nested, and discontinuous entities as nag-based entity sequence generation problems. we will take a pre-trained nag model consisting of an encoder and a decoder as the model backbone. denote the tokenized @xcite input sentence with length l as @xmath0, ..., w l-1 ]. and the target output is the set of entity sequences {(es i , t i )} m @xmath1, where m is the number of entities, es i is the i-th entity consisting of l i tokens, and t i is the type label of entity i. ## conclusion in this work, we propose nag-ner, a unified generative model for various ner tasks based on non-autoregressive generation (nag). in nag-ner, different ner tasks are formulated as entity set generation tasks. we employ the nag encoder to detect potential entity starts and the nag decoder to efficiently decode entity sequences. experiments on three benchmark ner tasks and two proprietary ner tasks demonstrate that our method can outperform baseline generative ner methods while achieving higher inference speed. we also conduct ablation studies to demonstrate the necessity of each module in our nag-ner method. shuai zhang, yongliang shen, zeqi tan, yiquan wu, and weiming lu. 2022. de-bias for generative extraction in unified ner task. in annual meeting of the association for computational linguistics.