Learning “ O ” Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER (2023)

Abstract: As the categories of named entities rapidly increase, the deployed NER models are required to keep updating toward recognizing more entity types, creating a demand for class-incremental learning for NER. Considering the privacy concerns and storage constraints, the standard paradigm for class-incremental NER updates the models with training data only annotated with the new classes, yet the entities from other entity classes are regarded as “Non-entity” (or “O”). In this work, we conduct an empirical study on the “Unlabeled Entity Problem” and find that it leads to severe confusion between “O” and entities, decreasing class discrimination of old classes and declining the model’s ability to learn new classes. To solve the Unlabeled Entity Problem, we propose a novel representation learning method to learn discriminative representations for the entity classes and “O”. Specifically, we propose an entity-aware contrastive learning method that adaptively detects entity clusters in “O”. Furthermore, we propose two effective distance-based relabeling strategies for better learning the old classes. We introduce a more realistic and challenging benchmark for class-incremental NER, and the proposed method achieves up to 10.62% improvement over the baseline methods.

## introduction existing named entity recognition systems are typically trained on a large-scale dataset with predefined entity classes, then deployed for entity recognition on the test data without further adaptation or refinement @xcite @xcite . in practice, the newly-arriving test data may include new entity classes, and the user's required entity class set might keep expanding. therefore, it is in demand that the ner model can be incrementally * equal contribution. † corresponding authors. "o" "o" "o" "o" ... ... ## handling the unlabeled entity problem in order to learn discriminative representations for unlabeled entity classes and the true "o" (connected to observations 1, 2, 3), we propose entityaware contrastive learning, which adaptively detects entity clusters in "o" during contrastive learning. to further maintain the class discrimination of old classes (connected to observation 2), we propose two distance-based relabeling strategies to relabel the unlabeled entities from old classes in "o". additionally, we propose the use of the nearest class mean classifier based on learnt representations in order to avoid the prediction bias of linear classifier. rehearsal-based task formulation to better learn representations for entities and "o", in this work, we follow the memory replay (rehearsal) setting adopted by most of the previous works @xcite @xcite . formally, we retain a set of exemplars for each class c, where x i c refers to one token x labeled as class c and x is the context of x labeled as "o". in all our experiments, we set @xmath0. 2 we set a small k because the class number can be large. ## experiment previous works @xcite @xcite on class-incremental ner conducted experiments on the conll 2003 @xcite and ontonotes 5.0 @xcite datasets. however, due to the limited class number of these datasets, the class number introduced in each step and the total number of incremental steps in these datasets are limited. for instance, there are only four classes in the conll03 dataset, thus only one class is introduced in each step and there are only four incremental tasks to repeat. in more realistic situations, multiple classes can be introduced in each step (e.g., a set of product types) and there can be a larger number of incremental steps. in this work, we provide a more realistic and challenging benchmark for class-incremental ner based on the few-nerd dataset 3 @xcite , which contains 66 fine-grained entity types. following the experimental settings of previous works @xcite @xcite , we randomly split the 66 classes in few-nerd into 11 tasks, corresponding to 11 steps, each of which contains 6 entity classes and an "o" class. the training set and development set of each task t t contains sentences only labeled with classes of the current task. the test set contains sentences labeled with all learnt classes in task {0 . . . t}. the statistics and class information of each task order can be found in appendix a.6. ## conclusion in this work, we first conduct an empirical study to demonstrate the significance of the unlabeld entity problem in class-incremental ner. based on our observations, we propose a novel and effective representation learning method for learning discriminative representations for "o" and unlabeled entities. to better evaluate class-incremental ner, we introduce a more realistic and challenging benchmark. intensive experiments demonstrate the effectiveness and show the superior of the proposed method over the baselines.