Debiasing Generative Named Entity Recognition by Calibrating Sequence Likelihood (2023)

Abstract: Recognizing flat, overlapped and discontinuous entities uniformly has been paid increasing attention. Among these works, Seq2Seq formulation prevails for its flexibility and effectiveness. It arranges the output entities into a specific target sequence. However, it introduces bias by assigning all the probability mass to the observed sequence. To alleviate the bias, previous works either augment the data with possible sequences or resort to other formulations. In this paper, we stick to the Seq2Seq formulation and propose a reranking-based approach. It redistributes the likelihood among candidate sequences depending on their performance via a contrastive loss. Extensive experiments show that our simple yet effective method consistently boosts the baseline, and yields competitive or better results compared with the state-of-the-art methods on 8 widely-used datasets for Named Entity Recognition.

## introduction recently, recognizing flat, overlapped and discontinuous entities in a unified manner has been paid increasing attention. among the existing works for unified named entity recognition (ner), seq2seq formulation prevails for its flexibility and effectiveness in unified modeling @xcite @xcite . typically, it arranges the output entities into a fixed order to form a target sequence, and trains the generative model by maximum likelihood estimation (mle). however, this estimation introduces bias by assuming a deterministic target distribution, where the model learns to assign all the probability mass to the observed target sequence. the biased estimation hurts the performance during decoding where predicted sequence likelihoods often do not accurately rank the performance of the generated sequences. to alleviate the bias, @xcite propose two data augmentation methods that sample possible sequences from the target space. others resort to other formulations, e.g., w 2 ner @xcite reformulates ner as a word-word relation classification. in this study, we stick to the seq2seq formulation and explore how to mitigate the bias from another perspective orthogonal to @xcite . beam search decoding algorithms maintain b candidates in descending likelihoods and output the highest one. however, the rest candidates could contain predictions with better performance. we measure this phenomenon with oracle scores. as shown in recently, reranking-based methods proposed for the abstractive summarization task offer a potential technique @xcite . they train a discriminator on the candidates to predict a score for picking out the best candidate. for example, simcls @xcite regards the cosine similarity between the input and candidate representations as the score. however, when applying reranking-based methods to our task, we find a challenge originating from the nature of information extraction. candidates of the same input share most of the words and the discriminators trained from scratch have difficulty differentiating them people can communicate with international friends without the hefty phone bills. people international friends per per 84.0% 1.0 people international per loc 92.8% 0.8 international friends per people international per 74.5% 0.8 per international friends per people international per 51.2% 0.8 gpe international friends per people international per 30.0% 0.7 international friends per 99.7% candidates likelihood f1 score calibrated likelihood 60.5% 43.8% 37.7% 33.5% ground truth per per figure 1: illustration of sequence likelihood calibration. after guiding the estimated sequence likelihood by f1 score, the likelihood is more consistent with the f1 score. more cases can be found in appendix 8. our contributions are summarized as follows: to the best of our knowledge, we are the first to explore reranking-based methods in the field of generative information extraction (ye et al., 2022). 2. we propose a method for generative ner tackling the bias problem. 3. experimental results show that our method consistently boosts the baseline, and yields competitive results compared with the stateof-the-art methods on 8 widely-used datasets for ner. ## conclusion through pilot experiments, we find the decoded candidates provide potential supervision. based on this finding, we propose rerankner to debias generative ner based on a reranking framework adapted for the task. it consistently boosts the baseline and achieves competitive results with state-ofthe-art generative methods on eight ner datasets, which verifies the effectiveness of candidate order supervision. future work could consider extending this method to other generative ie tasks. another meaningful direction is to consider incorporating large language models into the reranking process.