Type Enhanced BERT for Correcting NER Errors (2023)

Abstract: We introduce the task of correcting named entity recognition (NER) errors without re-training model. After an NER model is trained and deployed in production,it makes prediction errors, which usually need to be fixed quickly. To address this problem, we firstly construct a gazetteer containing named entities and corresponding possible entity types. And then, we propose type enhanced BERT (TyBERT),a method that integrates the named entity’s type information into BERT by an adapter layer. When errors are identified, we can repair the model by updating the gazetteer. In other words, the gazetteer becomes a trigger to control NER model’s output. The experiment results in multiple corpus show the effectiveness of our method, which outperforms strong baselines.x

## introduction named entity recognition (ner) is the task of identifying spans that belong to particular categories, such as person, location, organization, etc. the ner task is important in the information extraction area and ner models are widely deployed in real production systems @xcite . in recent years, many neural-based methods were proposed to push ner accuracy by designing novel network architectures @xcite @xcite or incorporating external knowledge @xcite . unfortunately, all approaches are still far from perfect. when the model is served in production, we may still encounter recognition errors (e.g., bad cases). typically, to fix those bad cases, model developers need to (1) annotate the input sentences causing errors with correct labels, (2) combine newly annotated sentences with existing training data, (3) train and tune a new model with the new training data * equal contribution. ## related work our work is influenced by existing methods which combine both neural networks and lexicons or gazetteers for ner. for example, @xcite proposed a lattice-structured lstm encoding both a sequence of input characters and potential words that match a pre-gathered lexicon. @xcite presented collaborative graph network to solve the challenges of self-matched lexical words and the nearest contextual lexical words. @xcite aimed to alleviate the word ambigu-ity issue by a lexicon-based graph neural network with global semantics. @xcite designed an attentive neural network to explicitly model the mention-context association and gazetteer network to effectively encode name regularity of mentions only using gazetteers. @xcite introduced a flat-lattice transformer to incorporate lexicon information for chinese ner. @xcite invented gemnet to include a contextual gazetteer representation encoder, combined with a novel mixture-of-expert gating network to conditionally utilize this information alongside any word-level model. @xcite invented an approach of using a token-level gating layer to augment pretrained multilingual transformers with gazetteers from a target domain. finally, @xcite proposed lexicon enhanced bert (lebert) for chinese sequence labeling, which integrates external lexicon knowledge into bert layers directly by a lexicon adapter layer. it is worth noting that none of the previous works can be directly applied for correcting ner models without re-training. for example, lebert requires learning lexicon embeddings in the adapter layer. if we want to add a new span in the lexicon to fix a bad case, the model has to be re-trained to learn the new span's embedding. ## conclusions we introduced a new task of correcting ner errors without re-training models. we propose ty-bert which extended standard bert model with an adapter layer to incorporate span's type infor-mation stored in a gazetteer. we further introduce a noise rate parameter to balance the strength of the gazetteer and model's generalization ability. extensive results justified the effectiveness of the proposed method. we hope our work will inspire future studies towards ner error correction without model re-training.