De-Bias for Generative Extraction in Unified NER Task (2022)

Abstract: Named entity recognition (NER) is a fundamental task to recognize specific types of entities from a given sentence. Depending on how the entities appear in the sentence, it can be divided into three subtasks, namely, Flat NER, Nested NER, and Discontinuous NER. Among the existing approaches, only the generative model can be uniformly adapted to these three subtasks. However, when the generative model is applied to NER, its optimization objective is not consistent with the task, which makes the model vulnerable to the incorrect biases. In this paper, we analyze the incorrect biases in the generation process from a causality perspective and attribute them to two confounders: pre-context confounder and entity-order confounder. Furthermore, we design Intra- and Inter-entity Deconfounding Data Augmentation methods to eliminate the above confounders according to the theory of backdoor adjustment. Experiments show that our method can improve the performance of the generative NER model in various datasets.

## introduction named entity recognition (ner) is a task aimed at identifying distinct and independent entities from a given text while classifying them into predefined types. as a fundamental work in natural language processing (nlp), its research facilitates the application of many downstream tasks @xcite @xcite . in previous work @xcite @xcite @xcite , three kinds of different ner subtasks were raised (as shown in figure 1 ), which are flat ner, nested ner and discontinuos ner. the existing ner methods can be divided into three main categories, including labeling-based . @xcite , spanbased @xcite and generative-based @xcite @xcite methods. nongenerative methods have different problems when applied to all three different subtasks: the labelingbased methods need to design different tagging schema for various types @xcite @xcite while the span-based methods suffers from ambiguity of boundary when applied to discontinuous task. although generativebased methods are able to model all ner subtasks uniformly @xcite , the training objective differ significantly from ner task due to the autoregressive generation mannner, resulting in some incorrect biases learned by the model during the training process. from a causal perspective, the incorrect biases stem from two confounders: pre-context confounder and entity-order confounder. pre-context confounder means that the model is affected by pre-context words that may be extra-entity words when generating a particular entity word. for example, in s3 of figure 1 , the autoregressive generation mannner causes the model to generate the word "fatigue" of the entity "muscle fatigue" conditioned on the extra-entity words "muscle" and "pain". this causes the model to mistakenly establish dependencies between the intra-entity word "fatigue" and the extra-entity words "muscle" and "pain", while ignoring the dependency between the intra-entity words "muscle" and "fatigue". therefore, when only the entity "muscle fatigue" is in the input sentence, the model cannot predict the entity accurately and completely due to the learned incorrect dependency bias. entity-order confounder refers to the fact that the model is affected by a predetermined order of entities when generating an entity sequence. the entities in a sentence are essentially a set structure without decoding order among them. in contrast, the generative ner model pre-specifies the decoding order of entities, which introduces incorrect bias and ignores the bidirectional dependency between entities. as in s1 of figure 1 , after fixing the set of entities as "stallone" → "rocky" → "rambo", the model only models the unidirectional dependency of "rambo" on "stallone" and "rocky", without considering the reverse dependency of "stallone" on "rocky" and "rambo". in this case, if "rambo" is decoded first, it is difficult for the model to decode the other two entities "stallone" and "rocky" due to the lack of the reverse dependency. we can formulate the causalities in the process of entity sequence generation with a structural causal model (scm). as illustrated in figure 2 , the direct links denote the causality between the two nodes: @xmath0. @xmath1, which can be divided into two cases according to the location of the generated words: intra-entity generation and inter-entity generation. in the former case, n denotes the pre-context words, which can affect the generation of the next word (@xmath2. while in the latter case, n denotes the entity decoding order, and can affect the generation of the next entity (@xmath3. in both cases, the representation of input x is contaminated by the backdoor path x ← @xmath4. therefore, n is a confounder for the @xmath5. in order to eliminate the bias caused by confounders n in both cases, we designed the intraand inter-entity deconfounding data augmentation method from the theory of backdoor adjustment. our contributions are as follows: ## prerequisite for subsequent analysis, in this section we first illustrate how the ner task is modeled as a generative task, after which we illustrate the training and inference process of the generative model. ## experiments in this section, we first describe the dataset we used, then we present related implementation details and experimental results, after which we make an analysis based on the experimental results. ## conclusion in this paper, we analyze two kinds of confounder that generative models arised when applied to ner and use backdoor adjustment methods in causal inference to perform deconfounding. specifically, for pre-context confounder and entity-order confounder, we respectively design intra-entity and inter-entity de-confounding data augmentation methods. experiments show that the performance of the model improves on all datasets after deconfounding. in the future, we will continue to explore the application of causal inference to other tasks.