Learning "O" Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER
Year: 2023

Abstract:
As the categories of named entities rapidly increase, the deployed NER models are required to keep updating toward recognizing more entity types, creating a demand for classincremental learning for NER. Considering the privacy concerns and storage constraints, the standard paradigm for class-incremental NER updates the models with training data only annotated with the new classes, yet the entities from other entity classes are unlabeled, regarded as "Non-entity" (or "O"). In this work, we conduct an empirical study on the "Unlabeled Entity Problem" and find that it leads to severe confusion between "O" and entities, decreasing class discrimination of old classes and declining the model’s ability to learn new classes. To solve the Unlabeled Entity Problem, we propose a novel representation learning method to learn discriminative representations for the entity classes and "O". Specifically, we propose an entity-aware contrastive learning method that adaptively detects entity clusters in "O". Furthermore, we propose two effective distance-based relabeling strategies for better learning the old classes. We introduce a more realistic and challenging benchmark for classincremental NER, and the proposed method achieves up to 10.62% improvement over the baseline methods.

heading: 1 Introduction
text: Existing Named Entity Recognition systems are typically trained on a large-scale dataset with predefined entity classes, then deployed for entity recognition on the test data without further adaptation or refinement (Li et al., 2020; Wang et al., 2022; Liu et al., 2021; Ma et al., 2022a). In practice, the newly-arriving test data may include new entity classes, and the user’s required entity class set might keep expanding. Therefore, it is in demand that the NER model can be incrementally
∗Equal contribution. †Corresponding authors.
"O"
... ... LOC
Step 1
Step 2
Interstellar , directed by Nolan, premiered on in 2014 in California. PER
Interstellar , directed by Nolan, premiered on in 2014 in California.
"O""O"
"O" "O" ... ...
Step K "O"
Interstellar , directed by Nolan, premiered on in 2014 in California.
FILM "O"
DATE
"O"
updated for recognizing new entity classes. However, one challenge is that the training data of old entity classes may not be available due to privacy concerns or memory limitations (Li and Hoiem, 2017; Zhang et al., 2020). Also, it is expensive and time-consuming to re-annotate all the old entity classes whenever we update the model (Delange et al., 2021; Bang et al., 2021). To solve the problem, Monaikul et al. (2021) proposes to incrementally update the model with new datasets only covering the new entity classes, adopted by following studies as standard class-incremental NER paradigm.
However, as NER is a sequence labeling task, annotating only the new classes means entities from other entity classes are regarded as "Non-entity" (or "O") in the dataset. For example, in step 2 in Fig.1, the training data for model updating is only annotated with "LOC" and "DATE", while the entities from "PER" and "FILM" are unlabeled and regarded as "O" during training. We refer to this problem as the "Unlabeled Entity Problem" in class-incremental NER, which includes two types of unlabeled entities: (1) old entity classes (e.g., "PER" in step 2) that the model learned in previous
5959
steps are unlabeled in the current step, causing the model catastrophically forgetting these old classes. (Lopez-Paz and Ranzato, 2017; Castro et al., 2018) (2) potential entity classes that are not annotated till the current step, yet might be required in a future step. For example, the "FILM" class is not annotated till step 2, yet is required in step K.
In this work, we conduct an empirical study to demonstrate the significance of the "Unlabeled Entity Problem" on class-incremental NER. We observe that: (1) The majority of prediction errors come from the confusion between entities and "O". (2) Mislabeled as "O" leads to the reduction of class discrimination of old entities during incremental learning. (3) The model’s ability to learn new classes also declines as the potential classes are unlabeled during incremental training. These problems attribute to the serious performance drop of incremental learning with the steps increasing.
To tackle the Unlabeled Entity Problem, we propose a novel representation learning method for learning discriminative representations for the unlabeled entity classes and "O". Specifically, we propose an entity-aware contrastive learning approach, which adaptively detects entity clusters from "O" and learns discriminative representations for these entity clusters. To further maintain the class discrimination of old classes, we propose two distance-based relabeling strategies. By relabeling the entities from old classes with high accuracy, this practice not only keeps the performance of old classes, but also benefits the model’s ability to separate new classes from "O".
We also argue that the experimental setting of previous works Monaikul et al. (2021) is less realistic. Specifically, they introduce only one or two entity classes in each incremental step, and the number of total steps is limited. In real-world applications, it is more common that a set of new categories is introduced in each step (e.g., a set of product types), and the incremental learning steps can keep increasing. In this work, we provide a more realistic and challenging benchmark based on the Few-NERD dataset (Ding et al., 2021), following the settings of previous studies (Rebuffi et al., 2017; Li and Hoiem, 2017). We conduct intensive experiments on the proposed methods and other comparable baselines, verifying the effectiveness of the proposed method 1.
1Our code is publicly available at https://github. com/rtmaww/O_CILNER.
To summarize the contribution of this work:
• We conduct an empirical study to demonstrate the significance of the "Unlabeled Entity Problem" in class-incremental NER.
• Based on our observations, we propose a novel representation learning approach for better learning the unlabeled entities and "O", and verify the effectiveness of our method with extensive experiments.
• We provide a more realistic and challenging benchmark for class-incremental NER.

heading: 2 Class-incremental NER
text: In this work, we focus on class-incremental learning on NER. Formally, there are N incremental steps, corresponding to a series of tasks {T1, T2, . . . , TN}. Here, Tt = (Dtrt ,Ddevt ,Dtestt , Ct,new, Ct,old) is the task at the tth step. Ct,new is the label set of the current task, containing only the new classes introduced in the current step (e.g., {"LOC", "DATE"} in Fig.1, step 2). Ct,old = t−1⋃ i=1
Ci,new ∪ {“O”} is the label set of old classes, containing all classes in previous tasks and the class "O" (e.g., {"PER", "O"} in Fig.1, step 2). Dtrt = {Xjt , Y jt }nj=1 is the training set of task t, where each sentence Xjt = {xj,1t , . . . , xj,lt } and Y jt = {yj,1t , . . . , yj,lt }, yj,kt ∈ Ct,new is annotated with only the new classes. In each step t, the model At−1 from the last step needs to be updated with only the data Dtrt from the current step, and is expected to perform well on the test set covering all learnt entity types Callt = Ct,new ∪ Ct,old.

heading: 3 The Importance of Unlabeled Entity Problem in Class-incremental NER
text: In this section, we demonstrate the importance of the Unlabeled Entity Problem in Class-incremental NER with empirical studies. We conduct experiments on a challenging dataset, the FewNERD dataset, to investigate the problems in classincremental NER. We conduct experiments with two existing methods: (1) iCaRL (Rebuffi et al., 2017), a typical and well-performed method in class-incremental image classification. (2) Continual NER (Monaikul et al., 2021), the previous state-of-the-art method in class-incremental NER. More details of the dataset and the baseline methods can be found in Section 5.
Observation 1: The majority of prediction errors come from the confusion between entities and "O". In Fig.2, we show the distributions of prediction errors of different models in step 6, where the y-axis denotes samples belonging to "O" or the classes of different tasks. The x-axis denotes the samples are wrongly predicted as "O" or as classes from different tasks. Each number in a grid denotes the number of error predictions. From the results, we can see that the majority of error predictions are samples belonging to "O" wrongly predicted as entities (the first row of each model), indicating serious confusion between "O" and entity classes, especially the old entity classes. As explained in Section 1, the training data of each new task is only annotated with the new entity classes and the entities from old classes are labeled as "O". As the training proceeds, the class variance between the true "O" and old entity classes will decrease, leading to serious confusion of their representations.
Observation 2: Old entity classes become less discriminative during incremental learning.
We further investigate the representation variation of old classes during incremental learning. As shown in Fig.3, we select similar classes from step 0 and step 1, and visualize their representations after step 2 and step 5. The results show that the representations of these classes are discriminative enough in step 2. However, after a series of incremental steps, the representations of these old classes become less discriminative, leading to decreasing performance of old classes. This phenomenon also indicates the influence of the unlabeled entity problem on the unlabeled old classes.
Observation 3: The model’s ability to learn new classes declines during incremental learning. Finally, we conduct an experiment to investigate the model’s ability to learn new classes. In Table 1, we test the results of new classes in each step on dev sets that only contain these new classes. Here, Full Data is a baseline that trains on datasets that both old and new classes are annotated. Surprisingly, we find that the performance of the new classes of iCaRL and Continual NER keeps decreasing during incremental learning, compared to the stable performance of Full Data. This phenomenon is also related to the Unlabeled Entity Problem. As explained in the introduction, the potential entity classes (i.e., the entity classes that might be needed in a future step) are also unlabeled and regarded as "O" during incremental learning. As a result, the representations of these classes become less separable from similar old classes (also labeled as "O"), thus hindering the model’s ability to learn new classes.
Conclusion to the Observations: Based on above observations, we propose that appropriate representation learning are required to tackle the Unlabeled Entity Problems. The representations of entity and "O" are expected to meet the following requirements: (1) The "O" representations are expected to be distinct from the entity representations, so as to decline the confusion between "O" and en-
tities (Observation 1). (2) The representations of old entity classes are expected to keep discriminative in spite of being labeled as "O" (Observation 2). (3) The potential entity class are expected to be detected and separated from "O", and also be discriminative to other entity classes (Observation 3). These observations and conclusions contribute to the motivation of the proposed method.

heading: 4 Handling the Unlabeled Entity Problem
text: In order to learn discriminative representations for unlabeled entity classes and the true "O" (connected to Observations 1, 2, 3), we propose entityaware contrastive learning, which adaptively detects entity clusters in "O" during contrastive learning. To further maintain the class discrimination of old classes (connected to Observation 2), we propose two distance-based relabeling strategies to relabel the unlabeled entities from old classes in "O". Additionally, we propose the use of the Nearest Class Mean classifier based on learnt representations in order to avoid the prediction bias of linear classifier. Rehearsal-based task formulation To better learn representations for entities and "O", in this work, we follow the memory replay (rehearsal) setting adopted by most of the previous works (Rebuffi et al., 2017; Mai et al., 2021; Verwimp et al., 2021). Formally, we retain a set of exemplars Mc = {xic, yic, X i c}Ki=1 for each class c, where xic refers to one token x labeled as class c and X is the context of x labeled as "O". In all our experiments, we set K = 5 2.
2We set a small K because the class number can be large.

heading: 4.1 Entity-aware Contrastive Learning
text: In this section, we introduce the entity-aware contrastive learning, which dynamically learns entity clusters in "O". To this aim, we first learn an entityoriented feature space, where the representations of entities are distinctive from "O". This entityoriented feature space is learnt through contrastive learning on the labeled entity classes in the first M epochs of each step. Based on the entity-oriented feature space, we further conduct contrastive learning on "O", with the anchors and positive samples dynamically selected based on an entity threshold.

heading: Learning an Entity-oriented Feature Space.
text: Firstly, we are to learn an entity-oriented feature space, where the distance between representations reflects entity semantic similarity, i.e., representations from the same entity class have higher similarity while keeping the distance from other classes. This feature space is realized by learning a nonlinear mapping F (·) on the output representations h of PLM. We adopt cosine similarity as the similarity metric and train with the Supervised Contrastive Loss (Khosla et al., 2020):
LSCL = ∑
i∈I
−1 |P (i)|
∑
p∈P (i) log es(zi,zp)/τ∑ a∈A(i) e s(zi,za)/τ
(1) where z = F (h) denotes the representation after the mapping and s(·) is the cosine similarity.
Here, we apply contrastive learning only on the entity classes, thus we define:
I = {i | i ∈ Index(Dtrt ), yi ̸= “O”} A(i) = {j | j ∈ Index(Dtrt ), j ̸= i} P (i) = {p | p ∈ A(i), yp = yi} (2)
where the anchor set I only includes entity tokens. We train with LSCL in the first K epochs, improving the representations of entities and obtaining an entity-oriented feature space.
Calculating an entity threshold for anchors and positive samples selection. Based on the entityoriented feature space, we are to dynamically select possible entity clusters in "O" and further optimize their representations via contrastive learning. This selection is realized by a dynamically adjusted entity threshold.
Specifically, we first define the class similarity Sc as the average of exemplar similarities inside each class:
Sc = 1 |Mc| ∑
xi,xj∈Mc, xi ̸=xj
s(F (h(xi)), F (h(xj))) (3)
Then, we sort the class similarity of all classes and choose the median as the entity threshold Tent (here we simply choose the median for a modest threshold):
Tent = Sorted({S1, . . . , S|Callt |})[i], i = |Callt | 2 (4)
During contrastive learning for "O", we recalculate Tent before each epoch to dynamically adjust the threshold based on convergence degree.
Contrastive Learning for "O" with the entity threshold Based on entity threshold Tent, we then apply the entity-aware contrastive learning for "O" with auto-selected anchors and positive samples. Specifically, we re-define Eq.2 as:
IO = {i | ∃j ̸= i, yj = yi = “O”, s(zi, zj) > Tent} PO(i) = {p | p ̸= i, yp = “O”, s(zi, zp) > Tent} AO(i) = PO(i) ∪ {n | yn ∈ Ct,new} (5)
Then, we define the entity-aware contrastive loss of "O" by adopting Eq.1 with the definition in Eq.5:
LSCL,O = LSCL(IO, PO, AO) (6)
In the last N −K epochs, we jointly optimize the representations of entities and "O" by:
L = LSCL,O + LSCL (7)

heading: 4.2 Relabeling Old Entity Classes
text: In order to further retain the class discrimination of old classes, we propose two distance-based relabeling strategies to recognize and relabel the unlabeled old-class entities in "O". These two strategies are designed to make use of the previous model At−1 and the exemplar set M.
Relabeling with Prototypes. This strategy relabels "O" samples based on their distance to the class prototypes. Specifically, we first calculate the prototype of each class based on the representations of exemplars from the old model At−1.
pc = 1 |Mc| ∑
x∈Mc ht−1(x) (8)
Then, we define a relabeling threshold, denoted as the prototype relabeling threshold, by calculating the lowest similarity of all exemplars with their prototypes:
T hproto = β · min (x,y)∈Mc c∈Ct,old {s(ht−1(x),py)} (9) where β is a hyper-parameter to control the relabeling degree. Next, for each "O" sample xi in Dtrt , we relabel it only if its highest similarity to prototypes is larger than T hproto:
S ={s(ht−1(xi),pc) | c ∈ Ct,old} yi =argmax
c S, if maxS > T hproto (10)
Relabeling with Nearest Neighbors. In this approach, we relabel "O" samples based on their distance to the exemplars of each class. Similarly, we define the NN relabeling threshold T hNN as:
T hNN = β · min (xi,xj)∈Mc
c∈Ct,old
{s(ht−1(xi), ht−1(xj))} (11)
For each "O" sample xi, we then relabel it with T hNN by:
S ={s(ht−1(xi), ht−1(xc)) | xc ∈ Mc, c ∈ Ct,old} yi =argmax
c S, if maxS > T hNN (12)
Since the class discrimination of old entity classes keep declining during incremental learning, the older task needs a lower threshold for relabeling sufficient samples. Therefore, we set βi = 0.98 − 0.05 ∗ (t − i) for each old task i, where t is the current step.

heading: 4.3 Classifying with NCM Classifier
text: To make full use of the learnt representations, we adopt the Nearest Class Mean (NCM) classifier used in (Rebuffi et al., 2017) for classification, which is also widely applied in few-shot learning (Snell et al., 2017). For each sample x, the class prediction is calculated by:
y∗ = argmax c∈Callt s(ht(x),pc) (13)
where pc is the prototype of class c calculated with the exemplars as the same in Eq.8.

heading: 5 Experiment
text: Previous works (Monaikul et al., 2021; Xia et al., 2022; Wang et al., 2022) on class-incremental NER conducted experiments on the CoNLL 2003 (Sang and De Meulder, 2003) and OntoNotes 5.0 (Weischedel et al., 2013) datasets. However, due to the limited class number of these datasets, the class number introduced in each step and the total number of incremental steps in these datasets are limited. For instance, there are only four classes in the CoNLL03 dataset, thus only one class is introduced in each step and there are only four incremental tasks to repeat. In more realistic situations, multiple classes can be introduced in each step (e.g., a set of product types) and there can be a larger number of incremental steps.
In this work, we provide a more realistic and challenging benchmark for class-incremental NER based on the Few-NERD dataset3 (Ding et al., 2021), which contains 66 fine-grained entity types. Following the experimental settings of previous works (Rebuffi et al., 2017; Wu et al., 2019; PourKeshavarzi et al., 2021; Madaan et al., 2021), we randomly split the 66 classes in Few-NERD into 11 tasks, corresponding to 11 steps, each of which contains 6 entity classes and an "O" class. The training set and development set of each task Tt contains sentences only labeled with classes of the current task. The test set contains sentences labeled with all learnt classes in task {0 . . . t}. The statistics and class information of each task order can be found in Appendix A.6.

heading: 5.1 Experimental Settings
text: The main experiments in this work are conducted on the Few-NERD datasets. Specifically, for each
3https://ningding97.github.io/fewnerd/
model, we repeat incremental experiments on three different task orders and report the averages of the micro-f1 score. To further illustrate the proposed method on different datasets, we also conduct experiments on the OntoNotes 5.0 dataset (by splitting 18 classes into 6 tasks) in the same way.
We compare our method with 7 comparable baselines. Full Data denotes Bert-tagger (Devlin et al., 2019) trained with datasets annotated with both old and new classes, which can be regarded as an upper bound. LwF (Li and Hoiem, 2017) is a regularization-based incremental learning method. iCaRL (Rebuffi et al., 2017) is a typical rehearsalbased representation learning method. SCR (Mai et al., 2021) is also an effective rehearsal-based contrastive learning method with an NCM classifier. Con. NER or Continual NER (Monaikul et al., 2021) is the previous SOTA method on classincremental NER. Con. NER* is Continual NER trained with exemplars and tested with NCM classifier. For our method, Ours (NN) and Ours (Proto) denote our method using NN-based and prototypebased strategies, respectively.
The implementation details of baselines and our method, the dataset details, and the detailed macrof1 and micro-f1 results of different task orders can be found in Appendix A.1, A.4, A.5 and A.6.

heading: 5.2 Main Results
text: Table 2 show the results of the proposed method and baselines on the Few-NERD dataset. From the results, we can observe that: (1) The results of Full Data, which leverages all class annotations for training, is relatively consistent. (2) Although Continual NER has shown good performance on CoNLL03 or OntoNotes 5.0 datasets, its performance is limited on this more challenging benchmark, when encountering multiple classes

heading: Methods Step0 Step1 Step2 Step3 Step4 Step5
text: and more incremental steps. (3) The proposed method shows up to 10.62% improvement over baselines, and consistently exceeded the baselines by about 10% even in the later steps, verifying the advantages of the learnt representations. (4) The prototype-based relabeling strategy is more stable than the NN-based strategy especially in the later steps. A possible reason is that using the mean vector of exemplars for relabeling is more reliable than using each of the exemplars.
We also conduct experiments on the OntoNotes dataset to further illustrate our method. As shown in Table.3, the results of all methods improve on the less challenging setting, yet the proposed method still significantly outperforms all the baselines.

heading: 5.3 Ablation Studies
text: To further illustrate the effect of each component on our method, we carry out ablation studies on Few-NERD task order 1 and show the micro-f1 and macro-f1 results in Figure 6. Here, Normal SCL means applying the normal SupCon Loss on both
entity classes and "O" without the entity-aware contrastive learning. Similarly, Normal SCL w/o "O" means applying the normal SupCon Loss only on entity classes. Normal SCL w/o relabeling means applying the normal SupCon Loss without relabel (not using any of our methods). (Both Normal SCL and Normal SCL w/o "O" adopt prototype-based relabeling) w/o relabel denotes using the entity-aware contrastive learning without relabeling.
From the result, we can see that: (1) Both the relabeling strategy and entity-aware contrastive learning contributes to high performance. (2) The performance of normal SCL without the entity-aware contrastive learning and the relabeling strategy is even worse than iCaRL, indicating that inappropriately learning "O" representations can harm performance. (3) Comparing the micro-f1 and macro-f1 results, we find that the relabeling strategy contributes less to the micro-f1 results. As the micro-f1 results are dominated by head classes with a larger amount of data, we deduce that entity-aware contrastive learning is more useful for head classes (which also appears more in "O"). Also, as the relabeling strategy is based on the distance between representations, the results indicate its effectiveness for both head classes and long-tailed classes.

heading: 5.4 Effect of Threshold Selection
text: Fig.5 shows the results of different hyperparameter choices for threshold calculation. The upper figure refers to the relabeling threshold T hproto, which we set βi = 0.98 − 0.05 ∗ (t − i) for each task t in step i. In this experiment, we tried different
strategies for setting the threshold (bata=0.9 means β = 0.9, (0.95,-0.05) means βi = 0.95 − 0.05 ∗ (t− i)). We find that the performance is relatively stable w.r.t different choices, and a lower threshold seems more helpful. 4
In the bottom figure, we also tested for different Tentity choices, which we simply set as the median (0.5) of class similarities. As seen, the performance is also robust to different choices.

heading: 5.5 Mitigating the Unlabeled Entity Problem
text: To demonstrate the effectiveness of the proposed method on mitigating the Unlabeled Entity Problem, we conduct the same experiments as in Section 3. Comparing Fig.7 to Fig.2, we can see that the proposed method largely reduce the confusion between "O" and entities, contributing to much fewer error predictions. Comparing Fig.8 to Fig.3 (b), we find that the proposed method learns discriminative representations for the old classes despite the impact of incremental learning.

heading: 6 Related Works

heading: 6.1 Class-incremental Learning
text: There are two main research lines of classincremental learning: (1) Rehearsal-based methods are the most popular and effective methods, which keeps a set of exemplars from the old classes. Typical researches include regularization-based methods that reduces the impact of new classes on old classes (Chaudhry et al., 2019; Riemer et al., 2019); methods that aim to alleviate the biased prediction problem in incremental learning (Zhao et al., 2020; Hou et al., 2019); methods that replay with generative exemplars (Kamra et al., 2017; Ostapenko et al., 2019; Ramapuram
4We further test the relabeling accuracy in Appendix A.3.

heading: The Proposed Method, step 5
text: et al., 2020). (2) Regularization-based methods aim to regularize the model learning without maintaining any memory. Typical methods include knowledge distillation-based methods (Zhang et al., 2020; Hou et al., 2019) and gradient-based methods that regularize the model parameters (Kirkpatrick et al., 2017; Schwarz et al., 2018; Aljundi et al., 2018). These methods, when directly applied to incremental-NER, do not consider the Unlabeled Entity Problem, thus show limited performance. Nonetheless, these methods are essential references for us to improve class-incremental NER.

heading: 6.2 Class-incremental Learning for NER
text: Previous works have explored the classincremental problems in NER (Monaikul et al., 2021; Wang et al., 2022; Xia et al., 2022). These methods generally care about maintaining old knowledge. Monaikul et al. (2021) propose a knowledge distillation-based method for learning old classes in "O". Wang et al. (2022) and Xia et al. (2022) propose method to generate synthetic samples for old classes. Among these studies, we are the first to comprehensively investigate the Unlabeled Entity Problem and propose solutions that benefits both the old classes and new classes. We also provide a more realistic benchmark.

heading: 6.3 Learning "O" for NER
text: Many previous works have also explored "learning ’O’" in NER (Tong et al., 2021; Li et al., 2021, 2022; Monaikul et al., 2021; Wang et al., 2022; Ma et al., 2022b). There are three typical lines of work: (1) Tong et al. (2021) solves the “O” problem for few-shot NER. It proposes a multi-step
undefined-class detection approach to explicitly classify potential entity clusters in “O”, which is similar to our core idea. Different from (Tong et al., 2021), we integrate the clustering and detection of potential entity clusters implicitly into representation learning, through a novel design for anchor and positive selection in contrastive learning. To our best knowledge, we are the first to explore the “O” problem in NER with representation learning. (2) There also exist other works that study the unlabeled entity problem (Li et al., 2021, 2022) in NER. These works focus more on avoiding false-negative samples during training and are not specifically designed for distinguishing potential entity classes. (3) The ‘O’ problem is also considered by previous works in class-incremental NER (Monaikul et al., 2021; Wang et al., 2022), yet they mainly focus on distilling old knowledge from “O”. Our work provides new insight on the “O” problem (or unlabeled entity problem) by comprehensively considers the old classes and new classes, with detailed experimental results.

heading: 7 Conclusion
text: In this work, we first conduct an empirical study to demonstrate the significance of the Unlabeld Entity Problem in class-incremental NER. Based on our observations, we propose a novel and effective representation learning method for learning discriminative representations for "O" and unlabeled entities. To better evaluate class-incremental NER, we introduce a more realistic and challenging benchmark. Intensive experiments demonstrate the effectiveness and show the superior of the proposed method over the baselines.

heading: 8 Limitations
text: The limitations of this work are: (1) In this work, we expect to consider more realistic and more applicable settings for class-incremental NER. Therefore, we consider the Unlabeled Entity Problem and provide a more realistic benchmark based on 66 fine-grained entity types. However, there remain some more serious situations unsolved in this work. First, the entity classes in each step might not be disjoint. For example, a new entity type "Director" might be included in an old entity type "Person". This problem is referred to as the coarse-to-fine problem existing in emerging types of NER. Second, the amount of data or labeled data introduced in each step can also be
limited, referring to the few-shot class-incremental problem. Therefore, the proposed method can be further improved to solve these problems. Third, the current version of the proposed method cannot handle the nested NER or contiguous NER problems. In the current version, we simply followed typical works in NER and adopted the sequence labeling scheme to model the NER task, which is not suitable for more complicated NER tasks. Nonetheless, as the proposed representation learning and re-labeling methods are agnostic to the formation of representations, we believe our method can also be adapted to a span-level version, which might be future works. (2) The proposed method is a rehearsal-based method that requires keeping exemplar sets for each class. Although the number of exemplars for each class is really small, we believe there can be more data-efficient solutions that totally avoid the need of memorizing data and also achieve good results. (3) The proposed method includes several hyper-parameters such as the entity threshold Tentity, relabeling threshold T hNN and T hproto. Although we have shown that the choice of thresholds is relatively robust (Sec.5.4), it still requires efforts to explore the most suitable thresholds when applied to other datasets or situations. There can be further work to improve this problem by formulating an automatic threshold searching strategy.

heading: Acknowledgements
text: The authors wish to thank the anonymous reviewers for their helpful comments. This work was partially funded by the National Natural Science Foundation of China (No.62076069,62206057,61976056), Shanghai Rising-Star Program (23QA1400200), and Natural Science Foundation of Shanghai (23ZR1403500).

heading: A Appendix
text: A.1 Implementation Details
We implemented the proposed method and all baselines based on the bert-base-cased pretrained model using the implementation of huggingface transformers 5. For our method, we implement the SupCon loss based on the implementation in the SupContrast library6. For LwF and iCaRL, we follow the implementations of (Masana et al., 2020) 7. For SCR, we follow the implementation of the online-continual-learning library8. There is no public source of Continual NER, so we implement based on the paper (Monaikul et al., 2021) and report the results of our implementation. At each step, we trained the model for 16 epochs and selected the best model on the dev set. For all methods, we use a learning rate of 5e-5, batch size of 16 and the max sequence length of 128. For our method, we start entity-aware contrastive learning for "O" with LSCL,O at the 10-th epoch and train it for 6 epochs at each step. We conducted all experiments on on NVIDIA GeForce RTX 3090.
Construction of the exemplar set For all rehearsal-based method, we keep 5 exemplars for each class, each of which consist of one entity word and its context. The exemplar words of each class are selected by picking the most high-frequency words of each class in the dataset. For each exemplar word, we randomly pick one sentence that contains this word as its context. We use the same exemplar set for all methods.

heading: A.2 Performance on Old and New Classes
text: In figure 9, we show the performance change of different methods on old classes and new classes. As seen, the proposed method can maintain the performance of old classes in a higher degree, which mainly attributes to the relabeling strategy. Meanwhile, the entity-aware contrastive learning method also helps to keep the discrimination of old classes in "O". Also, the proposed method is more effective on learning the new classes than baseline methods, with a highest improvement of 6.01% in the last step. These results indicate the effectiveness of entity-aware contrastive learning, which helps learn fine-grained and entity-aware represen-
5https://github.com/huggingface/transformers 6https://github.com/HobbitLong/SupContrast 7https://github.com/mmasana/FACIL 8https://github.com/RaptorMai/online-continual-learning
tations for "O", preventing the potential classes from confusing with "O" and other similar classes.

heading: Steps Precision Recall Micro-f1

heading: Prototype-based relabeling

heading: NN-based relabeling

heading: A.3 Relabeling Statistics
text: We examine the token-level micro-f1 scores of different relabeling strategies based on the gold labeled data of each step on Few-NERD task order 1. The results are shown in Table 4. We find that: (1) The proposed relabeling strategies can achieve acceptable relabeling accuracy, which greatly helps for retaining the knowledge of old classes and improving representation learning for potential classes. (2) Using a fixed β leads to higher recall and lower precision in earlier steps, as well as lower recall in later steps. This might because the convergence degree of old classes decrease in later step, thus a fixed threshold will relabel lim-
ited number of old class samples. (3) Compared to prototype-based method, the NN-based method has slightly lower recall and higher precision in earlier steps, which might correspond to its slightly higher performance in earlier steps on Few-NERD task order 1 (Table 5).

heading: A.4 Detailed Results on Few-NERD
text: The detailed results on the Few-NERD datasets are shown in Table 5 (task order 1), Table 6 (task order 2), Table 7 (task order 3). In each table, the numbers in black denote the micro-f1 scores and the numbers in green denote the macro-f1 scores. The proposed method surpass all baseline methods in all task orders.

heading: A.5 Detailed Results on OntoNotes 5.0
text: We also conduct experiments on OntoNotes 5.09 by randomly splitting the 18 entity classes into 6 tasks, each of which contains 3 entity classes and a "O" class. The detailed results on the OntoNotes datasets are shown in Table 8 (task order 1), Table 9 (task order 2), Table 10 (task order 3). In each table, the numbers in black denote the micro-f1 scores and the numbers in green denote the macrof1 scores. The proposed method surpass all baseline methods in all task orders.

heading: A.6 Dataset Details
text: The dataset details of Few-NERD are shown in Table 11 (task order 1), Table 12 (task order 2), Table 13 (task order 3). The dataset details of OntoNotes 5.0 are shown in Table 14 (task order 1), Table 15 (task order 2), Table 16 (task order 3).
9https://catalog.ldc.upenn.edu/license/ldc-non-membersagreement.pdf
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
8
3 A2. Did you discuss any potential risks of your work? 8
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? 5
3 B1. Did you cite the creators of artifacts you used? 5
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? A.5
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 5, A.5, A.6,
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? The datasets we use are publicly available. And we need to perform NER tasks that involve identifying the names of people, thus the data are usually not anonymized.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? 5, A.5, A.6,
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 5, A.5, A.6,
C 3 Did you run computational experiments? 5
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? A.1
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? A.1
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 5, A.4, A.5
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? A.1
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Nonetheless, as the proposed representation learning and re-labeling methods are agnostic to the formation of representations, we believe our method can also be adapted to a span-level version, which might be future works. (2) The proposed method is a rehearsal-based method that requires keeping exemplar sets for each class. Although the number of exemplars for each class is really small, we believe there can be more data-efficient solutions that totally avoid the need of memorizing data and also achieve good results. (3) The proposed method includes several hyper-parameters such as the entity threshold Tentity, relabeling threshold T hNN and T hproto. Although we have shown that the choice of thresholds is relatively robust (Sec. 5. 4), it still requires efforts to explore the most suitable thresholds when applied to other datasets or situations. There can be further work to improve this problem by formulating an automatic threshold searching strategy