Continual Named Entity Recognition without Catastrophic Forgetting
Authors: Duzhen Zhang
Wei Cong
Jiahua Dong
Yahan Yu
Xiuyi Chen
Yonggang Zhang
Zhen Fang
Venue: EMNLP 2023 Main (2023)
Keywords: Continual Named Entity Recognition without Catastrophic Forgetting

Abstract: Continual Named Entity Recognition (CNER) is a burgeoning area, which involves updating an existing model by incorporating new entity types sequentially. Nevertheless, continual learning approaches are often severely afflicted by catastrophic forgetting. This issue is intensified in CNER due to the consolidation of old entity types from previous steps into the non-entity type at each step, leading to what is known as the semantic shift problem of the non-entity type. In this paper, we introduce a pooled feature distillation loss that skillfully navigates the trade-off between retaining knowledge of old entity types and acquiring new ones, thereby more effectively mitigating the problem of catastrophic forgetting. Additionally, we develop a confidence-based pseudo-labeling for the non-entity type, i.e., predicting entity types using the old model to handle the semantic shift of the non-entity type. Following the pseudo-labeling process, we suggest an adaptive re-weighting type-balanced learning strategy to handle the issue of biased type distribution. We carried out comprehensive experiments on ten CNER settings using three different datasets. The results illustrate that our method significantly outperforms prior state-of-the-art approaches, registering an average improvement of 6.3% and 8.0% in Micro and Macro F1 scores, respectively.

Continual Named Entity Recognition without Catastrophic Forgetting
Duzhen Zhang! Wei Cong!,, Jiahua Dong!2), Yahan Yu², Xiuyi Chen', Yonggang Zhang and Zhen Fang2
Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China University of Chinese Academy of Sciences, Beijing, China 'Baidu Inc, Beijing, China
" Department of Computer Science, Hong Kong Baptist University, HongKong, China
Australian Artificial Intelligence Institute, University of Technology Sydney, Sydney, Australia {bladedancer congwe wwngdonnnghaio iauua1 195 piiggmall.... com
Abstract
Cuerent Task
Continual Named Entity Recognition (CNER) is burgeoning area, which involves updat- ing an existing model by incorporating new entity types sequentially. Nevertheless, con- tinual learning approaches are often severely affficted by catastrophic forgetting. This issue is intensified in CNER due to the consolida- tion of old entity types from previous steps into the non-entity type at each step, leading what is known the semantic shift prob- lem of the non-entity type. In this paper, we introduce a pooled feature distillation loss that skillfully navigates the trade-off' between re- taining knowledge of old entity types and ac- quiring new ones, thereby more effectively mit- igating the problem of catastrophic forgetting. Additionally, we develop a confidence-basee heudo-labeling for the non-entity type, i.e. predicting entity types using the old model to handle the semantic shift of the non-entity type. Following the pseudo-labeling process, we sug- gest an adaptive re- re-weighting type-balanced learning strategy handle the issue of biased type distribution. We carried out comprehen- sive experiments on ten CNER settings using three different datasets. The results illustrate that our method significantly outperforms prior state-of-the-ar approaches, registering an aver- age improvement of 6.3% and 8.0% in Micro and Macro Fl scores, respectively.'
IPERI
J01
[01
101
[ORG]
[GPE]
101 [0] [0]
CL:
[0]
[0]
[0]
[0]
[0]
[0]
[0]
[GPE]
Inputs:
Bin
will attend
the LMNLP meeting Singapore December
Figure I: A simplified CNER example, where FL and CL denote Full ground-truth Labels and Current ground- truth Labels, respectively. Old entity types (such as [ORG] (Organization), [PER] (Person)) and future entity types (such [DATE] (Date)) masked [O] (the non-entity type) at the current step 1 where [GPE] (Countries) is the current entity type to learned, causing the semantic shift problem of the non- entity type (the second row CL.).
new epoch. Conventionally, NER operates within paradigm where tokens are classified into some fixed entity types (such as Organization, Person, etc.), and the NER model undergoes a one-time learning process. However, a more realistic sce- nario calls for the NER model to continually iden- tify novel entity types as they emerge, without the need for a complete retraining. This paradigm, known as Continual Named Entity Recognition (CNER), has gained substantial research attention due to its promising practical applications (Mon- aikul et al., 2021: Zheng et al., 2022). A case in point is voice assistants like Siri and Xiao Ai, which are frequently required extract new entity types (such as Genre, Actor) to understand new user intents (for example, GetMovie).
1 Introduction
Named Entity Recognition (NER) is a essential research area in Natural Language Understanding (NLU). Its purpose is to assign each token in sequence with multiple entity types non-entity type (Ma and Hovy, 2016). Recently, the advent of Pre-training Language Models (PLMs) (Ken- ton and Toutanova, 2019) has ushered NER into a
Deep learning approaches to CNER encounter two primary challenges. The first one is common to all continual learning methods, known as catas- trophic forgetting (Robins, 1995; McCloskey and Cohen, 1989; Goodfellow et al., 2013; Kirkpatrick et al., 2017). This refers to the tendency of neural networks to forget previously acquired knowledge when learning new information. Existing CNER methods (Monaikul et al., 2021; Xia et al., 2022; Zheng et al., 2022) often resort to the widely em-
"Equal contributions.
'The corresponding author is Dr. Jiahua Dong. 'Our cnde is available at attps://gt t B1 alddeDancer957/CPF...

---

ployed technique of knowledge distillation (Hin- ton al., 2015) to tackle catastrophic forgetting. However, these methods meticulously extract the output probabilities for previous entity types from the old model and transfer them to the new model. Consequently, they tend to imbue the model with excessive stability (i.e., retention of old informa- tion) at the expense of plasticity (i.e., acquisition of new information).
duce the recognition errors from the old model, we employ entropy as a measure of uncertainty and the median entropy as a confidence threshold, retain- ing only those pseudo labels where the old model exhibits sufficient confidence. Furthermore, we introduce an adaptive re-weighting rype-balanced learning strategy to handle the biased distribution between new and old entity types that occurs post yseudo-labeling. This approach adaptively assigns different weights to the tokens of different types based on the number of tokens. Our contributions can be summarized as follows:
The second one is specific to CNER, involving the semantic shift of the non-entity type. In the conventional NER paradigm, tokens are marked as the non-entity type, indicating that they do not belong to any entity type. contrast, in the CNER paradigm, tokens marked as the non-entity type imply that they do not belong to any of the current entity types. This implies that the non-entity type may encompass: the true non-entity type, previ- ously learned old entity types, future ones not yet encountered. As depicted in Figure/ 1 (the sec- ond row CL), the old entity types Organization and Person (learned in the prior steps such as f-1, t-2, etc.) as well as the future entity type Date (will be learned in the future steps such as (+1, L+2, etc.) are all marked as the non-entity type at the current step t. Lacking mechanisms to differen- tiate tokens that pertain to previous entity types from the authentic non-entity type, this semantic shift could worsen catastrophic forgetting. To our knowledge, Zheng et al. were the first to recognize the old entity types contained within the non-entity type using the old model to distill causal effect. However, they don't sufficiently handle recogni- tion errors by the old model.
We design a pooled features distillation loss to alleviate catastrophic forgetting by retaining linguistic knowledge and establishing a suit- able balance between stability and plasticity.
We develop confidence-based pseudo- labeling strategy to better recognize previous entity types for the current non-entity type tokens and deal with the semantic shift prob- lem. To cope with the imbalanced type distri- bution, we propose an adaptive re-weighting type-balanced learning strategy for CNER.
Extensive results
CNER settings of three datasets indicate that our CPFD achieves remarkable improvements over the existing Maate-Of-Th--ae Art (SOTA) approaches with an average gain of 6.3% and 8.0% in Micro and Macro Fl scores, respectively.
Related Work
Continual Learning learns continuous tasks without reducing performance on previous tasks (Chen and Liu, 2018; Dong et al., 2022, 2023b). We divide existing continual learning methods into memory-based, dynamic architecture- based, and regularization- methods (Lopez-Paz and Ranzato, 2017; Rebuffi et al., 2017: Shin et al., 2017) learn a new task by integrating the saved or generated old samples into the current training samples. Dynamic architecture-based methods (Mallya and Lazebnik, 2018; 8; Rosenfeld and Tsotsos, 2018; Yoon et al., 2018 8) dynamically extend the model architecture to learn new tasks. Regularization- impose constraints on network weights (Aljundi et al., 2018; Kirkpatrick et al., 2017: Zenke et al., 2017), intermediary features (Hou et al., 2019a), or output probabilities (Li and Hoiem, 2017: Dong et al., 2023a) for relieving catastrophic forgetting.
In this paper, we present a novel approach named CPFD, an acronym for Confidence-basee pseudo- labeling and Pooled Features Distillation, which utilizes the old model in two significant ways to address the aforementioned challenges inherent in CNER. Firstly, we introduce pooled features dis- tillation loss that strikes a judicious trade-off be- tween stability and plasticity, thus effectively alle- viating catastrophic forgetting. These features are grounded in the attention weights learned by PLMs, capturing crucial linguistic knowledge necessary for the NER task, including coreference and syntax information (Clark al., 2019). Secondly, we de- velop a confidence-basee pseudo-labeling strategy to specifically identify previous entity types within the current non-entity type for classification, miti- gating the problem of semantic shift. To better re-
Memory-
methods

---

CNER Traditional NER focuses on the devel- opment of various deep learning models aimed at extracting entities from unstructured text (Li et al., 2020). Recently, PLMs have been widely utilized in NER and have achieved SOTA perfor- mance (Kenton and Toutanova, 2019; Liu et al., 2019). However, most existing methods are de- signed to recognize fixed set of predefined entity types. In response, CNER incorporates the contin- ual learning paradigm with traditional NER (Zhang et al., 2023b; Ma al., 2023; Zhang et al., 2023a). ExtendNER (Monaikul et al., 2021) explores the application of knowledge distillation to CNER, wherein the new model learns to fit the classifier logits of the old model. L&R (Xia et al., 2022) offers 'learm-andrreveew framework for CNER. The learning stage mirrors ExtendNER, while the reviewing stage generates synthetic samples of pre- vious entity types enhance the current dataset. CENER (Zheng et al., 2022) introduces a causal framework for CNER and distills causal effects from the non-entity type. In spite of significant advancements, these methods strictly distill the out- put probabilities for previous types from the old model into the new model, resulting in excessive stability but limited plasticity in the models. More- over, these approaches do not adequately address the semantic shift problem. CFNER employs the old model to recognize non-entity type tokens be- longing to previous entity types for distilling causal effect and utilizes a curriculum learning strategy to mitigate recognition error. However, this cur- riculum learning strategy requires the manual pre- limiting
entity types. Each step has its unique training set Dr, comprising multiple pairs (X', Y where X represents an input token sequence with a length of |X and Y represents the corresponding ground truth label sequence encoded in a one-hot format. Notably, Y only includes labels for the current entity types Et, with all other labels (for example, future entity types Ex+1:T potential old entity types g 1) collapsed into the non-entity type Co- At step t (t>1), considering the old model M as well as the current training set Dr, our objective is to update a novel model MM capable of recognizing entities from all types seen thus far, represented by U 8
4 Method
In the above formulation, we pinpoint two signifi- cant challenges in CNER. The first one is the issue of catastrophic forgetting (French, 1999), which implies that the neural network may entirely and abruptly forget previous entity types El 1 when learning the current types g This issue is further amplified by the second challenge: semantic shift. At step f, tokens labeled as the non-entity type are intrinsically ambiguous, as they could include the true non-entity type, previously learned old en- tity types, or as-yet-unseen future entity types. To address these challenges in CNER, we propose novel CPFD method, which takes advantage of the previous model in two ways, shown in Figure 2.
4.1 Pooled Features Distillation
Recent studies (Clark et al., 2019) suggest that attention weights learned by PLMs can encapsulate rich linguistic knowledge, including coreference and syntactical information, both critical to NER. In light of this, we first propose a feature distillation loss designed to encourage the transfer of linguistic knowledge ledge contained within attention weights from the previous model to the new model, formulated as follows:
definition of numerous hyper-paramette its applicability and the effect of error reduction.
In contrast, we propose a pooled features distilla- tion loss. This retains crucial linguistic knowledge embedded within attention weights and strikes a suitable balance between the stability and plasticity of the model, thus more efficiently mitigating catas- trophic forgetting issue. Moreover, we design confidence-based pseudo-labeling strategy for clas- sification. This strategy employs median entropy as the confidence threshold, climinating the need for better
K XX CTD ΣΣΣ
Aluss
1-1 Altiis
(I)
manually predefining any reducing the recognition error from the old model, and dealing with the semantic shift problem.
where A and A c [XXIXIIXXX correspond to the attention weights of layer & for M,,- and Mr respectively, 11,, l,... L. with K standing for the count of attention heads.
3 Preliminary
CNER aims to train a model across t = 1,..., steps, progressively learning an expanding set of
However, both the output probabilities distilla- tion found in prior CNER methods (Monaikul et al.,

---

Full labels: [PER] [O] [O] [GPE] [DATE]
Inputs:
Bin was in Japan yesterday
New Model
Old Model
Lx
Lx
Pooled Features Distillation
Multi-head
Multi-head
Attention
Dimension2
Head Dimension
Hea Dimenston ol
Sequence Dinien
Attention
CpFD
Attention
Attention
Weights
Weighte
S
FeedForward
FeedForward
Network
Sequence Dimension1
Sequence Dimension1
Network
Classifier
Classifier
belance-pseudi pueeddo
Confidence-based Pseudo Labeling
Direct Predict
PreditPredict
Current larget
|ER] [0] [IGN] [GPE]
[PER] [O] [X]
[ORG]
[O]
Union
Confidence Threshold Fillering (u < rg)
Retain Confident
Labels
Current ground-Iruth labels
Refined pseudo labels
[0] [0] (GPE) [0]
[PER] [O]
[IGN]
[
[GPE] Current Endly Types
[PER] (ORG] Od Endty Typea [DATE]
Enity Types
non-anity bye
[X] Operation [GN] Ignored Labal
Our CPFD method aims to learn a NER model within a continual learning paradigm, where old entity types are collapsed into the non-entity type in the current step. We constitute a suitable balance between stability and plasticity by pooled features distillation loss prevent catastrophic forgetting and generate high-quality pseudo- labels from old predictions by confidence-bart based pseudo-labeling strategy to deal with the semantic shift problem.
Figure 2:
2021; Xia al., 2022; Zheng et al., 2022) and feature distillation delineated in Equation (I) can considered rigid losses. This approach compels M, to strictly align with the output probabilities or attention weights found in Mi-1- This practice often results in an overabundance of stability (i.e., preservation of prior knowledge), consequently im- pairing its plasticity (i.e., ability to assimilate new knowledge). Contemporary studies further endorse the notion that the distillation operation ought to establish a balance between stability and plastic- ity (Douillard et al., 2020, 2021; Pelosin et al., 2022; Kurmi et al., 2021).
more permissive loss that preserves only the head dimension, formulated as follows:
K
CPFD-lax
Σ
Σ $,j=1
A
A 1-1
Ekai
i==1
(2)
Our proposed pooling-basee framework facili- tates the formulation of a more flexible feature dis- tillation loss, striking an improved balance between plasticity and stability. Based on Equation (2), we can moderately sacrifice plasticity to enhance sta- bility through less aggressive pooling, achieved by aggregating statistics across only one of the head and sequence dimensions:
To this goal, we incorporate a pooling opera- tion into our proposed loss, thus permitting a level of plasticity by consolidating the pooled dimen- sions (Douillard et al., 2020; Pelosin et al., 2022). The reasoning for this approach is that the minimal possible plasticity imposes an absolute similarity between the old and new model, whereas increased plasticity relaxes this definition similarity. Pool- ing can lessen this similarity, and more aggressive pooling can offer a greater degree of freedom.
1X XX
K
K
LPFD
A AiRAS
1-1
k
K
X
2
Σ
Al
A
2,
Σ
AL
A
A 1-1
-=1
By pooling the sequence dimensions, we derive
(3)

---

LPED achieves an appropriate balance between ex- cessive rigidity (as demonstrated by Equation (1)) and extreme leniency (as shown Equation (2)).
confidence threshold. Specifically, before embark- ing on learning step t, we initially input Dj into M for inference and calculate the entropy u of the output distribution related to each non-entity type token. Subsequently, for each non-entity type token, we can categorize these tokens according to their predicted old type (which corresponds to the index of the largest output probability predicted by calculate
4.2
Confidence-based Pseudo-labeling
As previously noted, tokens marked as the non- entity type at step might actually belong to the au- thentic non-entity type, previous entity types, or fu- ture entity types. Simply categorizing these tokens the non-entity type could intensify catastrophic forgetting. To tackle this issue of semantic shift linked the non-entity type, we design a pseudo- labeling strategy (Lee et al., 2013; Douillard et al.. 2021). Specifically, we utilize the predictions of old model for non-entity type tokens as indications of their true entity types, especially whether they are related to any of the previously acquired types. Formally, represent the cardinality of the cur- rent entity types with E card(E*). We denote the current model's 's predictions, encompassing the true non-enti non-entity type, all the old entity types, and the current ones, with Y E ([x"))(1=='+...++ We define Y E (x(E))x(++....* as the target at step L, calculated using the one-hot ground-truth label sequence Y с XXI I...! EE) in step I and pseudo-labels extracted from the predictions of the old model 1-1-1 E R xx|x(1+ +
the previous model). This allows the median entropy Te for old type E e, U U within each group. Following this, we can modify Equation (4) as follows:
if YE. =0 & argmax Y. e'Ee"
Y
if Y, = & == argmax
Pt-1 & Kuk <e
d'Ec,LEtit11
0 otherwise
(5) where denotes the uncertainty of token i as well as Te is type-specific confidence threshold. Equa- tion (5) only retains pseudo-labels where the previ- ous model is "confident" enough (u < To)-
Following the pseudo-labeling process, we find that the count of new-type tokens present in current sequences generally exceeds the count of pseudo- labeled old-type tokens. This type-imbalance issue typically skews the updated classifier towards new types. Inspired by (Zhao al., 2022), we introduce an Adaptive Re-weighting Type-balanced (ART) learning strategy that adaptively assigns varying weights to tokens of different types based on their quantity. Thus, the balanced pseudo-labeling cross- entropy loss can be expressed as follows:
The process is described as follows:
if Y t.u. =0 & argmax Vie
e'Ect
Y.e. =1 & e argmax P
(4)
d'EeLUELi
0 otherwise
1X
In other words, if a token is not marked as the non- entity type eor we replicate the ground truth label. Otherwise, we utilize the label predicted by the old model. This pseudo-labeling strategy enables the assignment of the actual semantic label to each token labeled as the non-entity type, provided the token belongs to any of the previous entity types. Nevertheless, labeling all non-entity type tokens as pseudo-labels can be unproductive, e.g., on uncer- tain tokens where the old model is likely to falter. Thus, this Vanilla Pseudo-Labeling (VPL) strategy inadvertently propagates errors from the previous model's 's incorrect predictions to the new model. Inspired by (Saporta al., 2020), we propose Confidence-based Pseudo-Labeling (CPL) strategy, designed to diminish the influence of label noises and effectively tackle the semantic shift problem. This strategy employs entropy as a measure of un- certainty, with the median entropy serving as the
Lbalance-pscudo =
1
Σ
Y log Y
(6)
|
where 7p denotes the weight of the token at the location in the sequence X computed as follow:
Noll
1/: = 0.5 + o Nnww
if E
(7)
1.0
otherwise
where Nold, Nenw and o(.) are the number of to- kens belonging to old entity types gl:-1 the num- ber of tokens belonging to the new entity types g and the sigmoid function, respectively. Finally, the total loss in CPFD is:
((O:) = bblaacce-ssudoo +A LPFD
(8)
classification
distillation
with À a hyper- for balancing losses, and θ is the set of learnable parameters for M,.

---

Table 1: The statistics for each dataset.
Performance Metries Consistent with CFNER, we utilize Micro F1 (Mi-FI) and Macro Fl score (Ma-F1) to evaluate the model's performance, tak- ing into account the issue of entity type imbalance in NER. We present the mean result across all steps, encompassing the first, as the ultimate performance. Further, to offer a more detailed analysis, we intro- duce step-wise wrsp-wisepp performance comparison line plots. To assess the statistical significance of the improve- ments, we perform paired t-test with a signifi- cance level of 0.05.
Exiie Type
Sunple
Exity Type Seguenee Alpalletiio Onder)
CoNU12000
21k
LOCADON MISC ORGANISATION PERSON
AGE, CITY. COUNTRY, HATE, DOCTOR, HOSIIDAL
INNIM MEDICA ancoon,, ORGANIZATIO,
1282
16
1411
PATHENT PKKINE, PROMISSION STAAm, STREIATT USERNAKE,
CARDINAL DATE, EVENT, BMC. LANGUAGE,
OranNieves
14
PERCENT, PERSON, PRODUCT, QUANTITY, TIME,
LAW,LOC: MONEY NORE, ORDINAL ORG,
WORK, CE_ART
5 Experiments
5.1 Experimental Setup
To ensure fair comparisons with SOTA approaches, we follow the experimental setup outlined in CENER (Zheng et al., 2022).
Baseline Methods We benchmark our CPFD against the recent CNER methods, namely Extend- NER (Monaikul et al., 2021) and CFNER (Zheng et al., 2022), with the latter serving as the pre- vious SOTA method. We also draw compar- isons with continual learning methodologies em- ployed in the field of computer vision, such Self-Training (SoSeRogg et al., 2005; Lange et al., 2019), LUCIR(Hou al., 2019b), and POD- Net (Douillard et al., 2020). Zheng et al. success- fully adapted these three methods to CNER set- tings. Additionally, we evaluate fine-tuning (FT) without the integration of any anti-forgetting prac- tices, thereby establishing a lower bound for com- parison. For more comprehensive introduction to these baselines, please consult Appendix C of CENER (Zheng et al., 2022).
Datasets We conduct the evaluation of CPFD using three widely adopted NER datasets: CoNLL2003 (Sang and De Meulder, 1837). 12B2 (Murphy et al., 2010), and OntoNotes5 (Hovy et al., 2006). The statistics of these three datasets are specifically presented in Table 1.
We divide the training set into disjoint slices, each corresponding to different continual learning steps, employing the same sampling algorithm as suggested in CFNER (Zheng et al., 2022). Within each slice, we exclusively keep the labels belong- ing to the entity types under learn, while others are masked as the non-entity type. For more compre- hensive explanations and the detailed breakdown of this sampling algorithm, refer to Appendix B of CENER (Zheng et al., 2022).
Implementation Details In alignment with prior CNER methods (Monaikul et al., 2021; Xia et al., 2022; Zheng et al., 2022), we adopt the "BIO" labeling schema for all datasets. Our NER model utilizes the bert-base-cased (Kenton and Toutanova, 2019) model as the encoder, featuring a layer depth (L) of 12 and an attention head count (K) of 12, and employs fully-connected layer as the classi- fier. We implement the model using the PyTorch framework (Paszke et al., 2019), built atop the BERT Huggingface implementation (Wolf et al., 2019). For each setting. if PG=1, we train the model for 10 epochs; otherwise, for 20 epochs. We establish the batch size, learning rate, and balanc- ing weight À as 8, 4e-4, and 2, respectively. In the total loss, we also penalize the KI. divergence be- tween the new model's classifier logits against the old model's 's classifier logits overcome forgetting better (Monaikul et al., 2021). All experiments are carried out on an NVIDIA A100 GPU with 40GB of memory, and each experiment is run times to ensure statistical robustness.
CNER Settings In terms of training, we intro- duce entity types in the same alphabetical order as CENER (Zheng et al., 2022) and learn mod- els sequentially with corresponding data slices. In particular, FG entity types are employed to train the base model, and for each continual learning step, we utilize PG entity types, represented FG- a-PG-b. For the CoNLL2003 dataset, we apply two CNER settings: FG-1-PG-1 and FG-2-PG-1. For the 12B2 and OntoNotes5 datasets, we estab- lish four CNER settings: FG-1-PG-1, FG-2-PG-2, FG-8-PG-1, and FG-8-PG-2. During evaluation, we maintain only the labels of the current entity types to learn, masking others as the non-entity type within the validation set. At each step, the model yielding the best validation performance is chosen for both testing and the next step. In the testing phase, we preserve labels for all previously learned entity types, designating the rest as the non-entity type within the test set.

---

Table 2: Comparisons with baselines on 12B2 and OntoNotes5. The red denotes the highest result, and the blue denotes the second highest result. The marker j refers to significant test p-value < 0.05 comparing with CFNER. represents results from re-implementation. Other bascline results are cited from CFNER (Zheng et al., 2022).
FG-I-PG-1
PG-2-PG-2
FG-8-PG-1
PG-8-PG-2
Dataset
Raseline
Mi-FI
Ma-F1
Mi-FI
Mc-FI
Ma-FI
Ma-F1
FT
17.4350.54
1381s1.14
28,57+0.26
21.4340.11
20.83=1.88
18.11=1.6
23,0040.15
23.54g0.33
12.31=0.35
17,14=1.03
39,26x1.38
27,23+093
26.22412.
26.08=7.42
LUCIR
43.86#248
31,31+1.62
64.3260.76
43,5340.59
57,86=0.87
33,0240.39
68. (8.5440.27
46,94=0.63
ST
31,9862.12
14.76s1.31
55.4444.78
33.3843.13
49.51a1.35
23.77a1.01
48 18.9446.78
29.00g3.04
ExtendNER
41.65110.11
23.1142.70
67.60.1.15
42.5841.99
45,1442.91
27.41:0.88
56.4842.41
38.88
ExtendNER
42.8562.86
24.05
57.011-4.14
38.2943.33
43.95.2.01
23.12::1.99
52.2555.10 36
30.93
CENER
64.7920.26
37.7950.65
52.5840.59
51.7110844
56.6643.22
36 16.13333
69.12:0.94
51,61=0.87
CENER
62.73+3.62
36.26+2.24
71 98.0005
49.09+1.38
59.79e1.70
37,30=1.15
69.07=0.89
51,09=1 1.05
CPFD (Ours)
74.19H09951
48.34=1.451
78.19=0.581
56,04=1.221
74.75+1.811
56.19=2.461
81.05+0.877
65.04:1.111
Imp. 19.40
110.55
115.61
14.33
144.66
118.89
111.93
113.43
15.2760.26
10.8511111
25 156011111
20,5540.24
17.63+0.57
12.23+1108
29:8140.12
20.05=0.11
PODNet
9.06+0.56
8.36+0.57
1900+1.00 0.0
16-93+0.85
29.00
20,54
$7.38+0.33
25. 25.85=0.29
LUCIR
28.18=1.15
21,11:084
56-4011.9
40.58+1.11
66,46:0046
46.29H0.38
76.17:0.00
55.58=0.35
50.71=0.79
33.24=1.06
68.93+1667
50.6341.66
73.5920.66
49.41=0.77
77.07=0.62
53.32=0.63
ExtendNER
51.36:0.77
33.38
63.0349.39 3939
17.6445.15
73.6520.19
50.55 56
77. 77.86&0.11 10
55.21=0.51
OntoNotes5
ExtendNER
50.53=0.86
32.84:0.84
65561+153
49 1.42664144
73.12=0.93
00.5550990
76.85.0.77
54,37=0.57
CENER'
58.44=0.71
41.75=1.51
72.1000.0
55.0240.35
78.25=0.33
58.64+0044
80.09+0.37
61.06=0.37
58.9420.57
72.5910.88
78.92:0.58
57.51aa1.32
80.6820.25
60.52a00.4
CPFD (Ours)
66, 06.73:0.701
54.12:0.30f
74.33
57.75
81.8719.471
65.52a1.001
83.38:0.18
66.2710.75
Imp.
17.79
11.99
11.74
11.79
12.95
16.88
22.70
75.21
1282 (PG-1-PGI)
1252 (PG-2-2)))))))
122 (FG-8-9G-1)
1252 (FG-8-FG-2)
60-
NIFT
15
15
10
12
Numberal Typee
Numbercl
Number Typee
Number Entiy
Destohereets (PG-1496-11
Consiicteed (PG2--96-22
Desohhinet (FG-84N--1
Cittokicest (F12-4-11-22
16.71
10 12 10
10 11
16 Number Emity Typer
12
Number Enity Types
Number Erity Types
Number of Entty Types
Figure 3: Comparison the step-wise Mi-FI on 12B2 and OntoNotes5. The result baselines is directly cited from CENER (Zheng et al., 2022).
5.2 Results and Analysis
As indicated in Tables 2 and 4, our CPFD method significantly surpasses the previous SOTA method, CFNER, yielding enhancements ranging from 1.33% to 14.96% in Mi-FI and from 0.83% to 18.89% in Ma-FI across ten CNER settings of three datasets. Figures 3, 6, and 8 further corrobo- rate that our CPFD outshines other CNER baseline methods in almost all step-wise comparisons un- der the ten settings. These results validate CPFD's s superior performance in learning a robust CNER model, demonstrating enhanced resilience against catastrophic forgetting and semantic shift problems. Additionally, we have visually represented some qualitative comparison results from the FG-8-PG- setting on the OntoNotes5 dataset in Figure 4. These results further validate the efficacy of our CPFD in learning new entity types consecutively.
Comparisons with Baselines To substantiate the efficacy of our CPFD method across various CNER settings, we conduct exhaustive experiments on the CoNLL2003, 12B2, and OntoNotes5 datasets. The results obtained from the 12B2 and OntoNotes5 datasets are presented in Table 2, and a step-wise comparison of Mi-FI scores is depicted in Figure 3. Due to the limitations in space, the step-wise Ma- Fl comparison results for I2B2 and OntoNotes5 datasets are provided in Figure 8 within Ap- pendix A. As for the outcomes obtained from the smaller dataset, CoNLL2003, these are detailed in Table 4, Figure 6 (ste Mi-Fi comparisons), and Figure (step-wise Ma-Fl comparisons), all available in our Appendix A.

---

Input Sentence
1985
the
Chinese Mentally Impaired and Blind Sports Associations
established.
ExtendNER PL
[0] [S-DATE]
[0] [B-ORG] [-WDA] [-WDA] [-WOA] [-WOA] [-WDAj [[WOA)] [-ORG]
[0] [0] [0] [0]
CENER PL
[B-DATE]
[0] [B [B-RRG)
[0]
[0] [0] [|ORG] [-ORG] [|ORG]
[0] [0] [0]
[0]
PL (Ours)
[0] [B.DATE]
[0] [BOOGG] [B NoNNORII--RG (I [RGG)] [-ORG] [-ORG] [|ORG]]
[-ORG]
[0] [0] [0]
[0]
Golden Labels
[O] [B-DATE] [O] [B-ORG] [IORG] [-ORG] [LORG] [DORG]] [-ORG] [[-RRG]
[LORG] [0] [0]
[0]
[0]
Input Sentence
Holding the
6th
Asia
Pacific Special Clympic Games Beiling will motivate society better support disabied sports
ExtendNER PL
[0] [B-WDA] [B-ORD] [-WoA][-woa] [--Oi) [MoDA] [AWDD]] [-War] [0] [B-GPE] [0]
[+]
[0] [0] [0]
[0]
[9] [0] [0]
CENER PL
[0] [0] [B-ORD] [Heve] [ANoA] [-wor] (-eve] [AWOA] [-EVE] [0] [B-OPE] [0]
[0]
[0] [0] [0] [0]
[0]
[0] [0]
CPFD PL (Ours)
[0]
[0]
[B-ORD] [-EVE] (-EvE] [HEVE] [-EVE] (HEVE) [-EVE] [0] [B-OPE] [0] [0]
[0] [0] [0] [0] [0] [0] [0]
Golden Labels
[0]
[0] [B-ORD] [B-EVE] (-EVE] (avvel [-EvE] [HEVE] [-EVE] [0] [B-GPE]
[0] [0] [0] [0] [0] [0]
[0] [0)
Figure 4: Two real cases sampled from the OntoNotes5 test set. PL denotes the predicted labels. B- and I-
distinguish begin/inside of entities. [O], [DATE], [ORG]. [WOA], [NORP), [ORD], [EVE], and [GPE] denote
the non-entity type, Date, Organization, Work of art, Nationalities, Religious, or Political groups,
Ordinals, Event, and Countries, Cities, or States, respectively. All the prediction results correspond to the final step of PG-2- These visualized cases highlight the superiority and effectiveness of our CPFD method.
Input Sentence
The Australla side will provide
China with a technical cooperation grant of
20 million Australian dollars
CL
[0] [0] [0] [O] [O]
[0]
[O]
[0]
[O]
[O] [0] [O]
[B-MON] [-MON] [-MON] [I-MON] [0]
CL + VPL
[0] [B-GPE] [0] [O] [I-GPE] [B-GPE] [B.DATE] [O]
[o]
[ILAN]
[O] [O]
[B-MON] [I-MON]
[I-ON]] [I-ONN] [0]
CL + CPL
[0] [B-GPE] [0] [O] [GN] [B-GPE][B-DATE]]
[O]
[O] [[GN]
[O] [O] [B-MON] [I-ON]] [I-ON]] [-MON] [O]
Figure 5: Visualization of pseudo labels on OntoNotes5 under FG-8-PG-2. CL denotes current ground labels.
VPL and CPL denote pseudo labels generated by vanilla and confidence-based pseudo-labeling strategies. B- and I- distinguish begin/inside of entities. [IGN] denotes ignored labels. [GPE] (Countries, Cities, or States), [LAN] (Language), and [DATE] (Date) denote the old entity types in the first step. [MON] (Money) denotes the new entity type being learned in the second step. The visualization result shows that our CPL can reduce label noises and combine with CL to form a better target for the current model, compared with VPL.
Table 3: The ablation study of our CPFD on 12B2 and OntoNotes5 under the setting FG-1-PG-1. When com- pared with Ours, all ablation variants severely degrade CNER performance. It verifies the importance of all components to address CNER collaboratively.
bility or plasticity in the CNER model. Our LPFD loss presents a judicious balance between stability and plasticity, which helps to better mitigate catas- trophic forgetting, without which poor CNER per- formance will result. CPFD w/o CPL denotes the absence of pseudo-labeling strategy. These results suggest that pseudo labels help the model recall what it has learned from the non-entity type tokens. Our CPL strategy can effectively reduce prediction errors from the old model and address semantic shift by retaining only confident pseudo labels. A visualization of these pseudo labels is shown Figure 5. The results of CPFD w/o ART under- line that ART contributes positively to addressing issues with biased type distribution.
ChinNoces5
Methoxl
Mi-FI
Ma-FI
Mi-FI
Ma-FI
CPFD (Ours)
74.19:0.95
48.34+1.45
66.73el).70
$4.12=0.30
w/ Cop
71.46=1.19
45:17=1.28
63.8
51,83=0133
w/ Cippsiss
70.22+0.90
43.89:1.10 1.10
62, 03.52+0..5
50:12:0.70
w/o CoFD
68.66 66.6610.88
42.2810.79
60, 80.800
18:9111338 33
w/o CPL
54.8615.36
37.39&3.58
59,3740.82
46.6840.45
ART
72.29:1.56
45.35x1.83
13.19
$2.94:0:66
Ablation Study This subsection examines the ef- fectiveness of individual components in our CPFD method through ablation studies, the results of which are shown in Table 3. We evaluate several al- ternatives for the PFD loss, defined in Section 4.1. Broadly, we find that substituting our LPFD loss with a more strict LED loss or more permissive LPFD-lax loss in our CPFD method tends to dimin- ish CNER performance. This occurs because these two variations either omit pooling or implement a more aggressive pooling, resulting in excessive sta-
6 Conclusion
In this paper, we lay the foundation for future re- search in CNER, an emerging field in NLU. We pinpoint two principal challenges in CNER: catas- trophic forgetting and the semantic shift problem of the non-entity type. To address these issues, we first introduce a pooled feature distillation loss that carefully establishes the balance between stabil- ity and plasticity, thereby better alleviating catas-

---

trophic forgetting. Subsequently, we present confidence-basee pscudo-labeling strategy to ex- plicitly extract old entity types contained in the current non-entity type, better reducing the impact of label noise and dealing with the semantic shift problem. We evaluate CPFD on ten CNER settings across three datasets and demonstrate that CPFD significantly outperforms the previous SOTA meth- ods across all settings.
Zhiyuan Chen and Bing Liu. 2018. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 12(3):1-207.
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. 2019. What Does BERT Look at? An Analysis of BERT's Attention, In Pro- ceedings of the 2019 ACI. Workshop BlackboxMLPP. Analyzing and Interpreting Neural Networks for NLP, pages 276-286.
Jiahua Dong, Wenqi Liang, Yang Cong, and Gan Sun. 2023a. Heterogeneous Forgetting Compensation for Class-Incremental Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1172--11751
Limitations
Our pooled features distillation loss necessitates ad- ditional computational effort to align with the inter- mediary features of the old model. Our confidence- based pseudo-labeling strategy, which employs median entropy as the confidence threshold, ne- cessitates pre-calculation for each old entity type based on the current training set and the old model, thus extending the training duration. Moreover, al- though our confidence-basee pseudo-labeling strat- egy helps reduce the prediction errors of the old model, it is not entirely foolproof, and some misla- beled instances may still persist.
Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. 2022. Federated class-incremental learning. In Proceedings of the IEFF/CVF conference on computer vision and pat- tern recognition, pages 10164-10173.
Jiahua Dong, Duzhen Zhang. Yang Cong, Wei Cong. Henghui Ding, and Dengxin Dai. 2023b. Federated Incremental Semantic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3934-3943.
Arthur Douillard, Yifu Chen. Arnaud Dapogny, and Matthieu Cord. 2021. PLOP: Learning Without For- getting for Continual Semantic Segmentation, In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 1040-4050.
Ethics Statement
In relation ethical considerations, we provide the following clarifications: (I) Our research does not engage with any sensitive data or tasks. (2) All experiments are conducted on existing datasets that have been sourced from publicly available scientific research. (3) We offer comprehensive descriptions of the dataset statistics and the hyper-parameter configurations for our method. Our analyses align with the experimental results. (4) In the interest of promoting reproducibility, plan to make our code accessible via GitHub.
Arthur Douillard, Matthieu Cord, Charles Ollion. Thomas Robert, and Eduardo Valle. 2020. PODNet: Pooled Outputs Distillation for Small-Tasks Incre- mental Learning. In Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XX, pages 86-102.
Robert M French. 1999, Catastrophic forgetting in con- nectionist networks. Trends cognitive sciences, 3(4): 128-135.
Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. 2013. An em- pirical investigation of catastrophic forgetting gradient-based neural networks. arXiv preprint
A Acknowledgement
We express our gratitude to the anonymous review- ers for their valuable and insightful comments. This research received partial support from the National Nature Science Foundation of China under Grants 32070000 and 62133005.
1rXiv:1312.6211.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. 2015. Distilling the knowledge in neural network. arXiv preprint
2(7).
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. 2019a. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 831
References
Rahaf Aljundi, Francesca Babiloni, Mohamed Elho- seiny, Marcus Rohrbach, and Tinne Tuytelaars. 2018. Memory aware synapses: Learning what (not) to for- get. In Proceedings of the European Conference on Computer Vision (ECCV), pages 139-154.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. 2019b. Learning Unified Classifier Incrementally via Rebalancing. In JEEE Conference

---

on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 831-839.
61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Longg Papers), pages 5959-5979.
Eduard Hovy, Mitch Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: the 90% solution. In Proceedings of the human lan- guage technology conference of the NAACI, Com- panion Volume: Short Papers, pages 57-60.
Xuezhe Ma and Eduard Hovy. 2016. End-to and-to--nd Se- quence Labeling via Bi-directional LSTM-CNNS- CRF. In Proceedings of the 54th Anmual Meeting of the Association for Computational Linguistics ( (Vol- ume I: Long Papers), pages 1064-1074.
Jacob Devlin Ming-Wes Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidi- rectional Transformers for Language Understanding. In Proceedings of NATCL-HLT, pages 4171-4186.
Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7765-7773.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz. Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Barbbsa-Bariinkka, et al. 2017. Over- coming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, (11113):521-3526
Michael McCloskey and Neal Cohen. 1989. Catas- trophic interference in connectionist networks: The sequential learning problem. In Psychology of learn- ing and motivation, volume 24, pages 109-165. Else- vier.
Natawut Monaikul, Giuseppe Castellucei, Simone Fil- ice, and Oleg Rokhlenko. 2021. Continual learn- ing for named entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 35, pages 13570-13577.
Vinod Kurmi, Badri N Patro, Venkatesh K Subrama- nian, and Vinay P Namboodiri. 2021. Do not forget to attend to uncertainty while mitigating catastrophic forgetting. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 736-745.
Shawn N Murphy, Griffin Weber, Michael Mendis, Vi- vian Gainer, Henry C Chueh, Susanne Churchill, and Isaac Kohane. 2010. Serving the enterprise and be- yond with informatics for integrating biology and the bedside (i2b2). Journal of the American Medical Informatics Association, 1742):1241130
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot. Xu Jia, Ales Leonardis, Gregory G. Slabaugh, and Tinne Tuytelaars. 2019. Continual learning: A comparative study on how to defy forgetting in clas- sification tasks. CoRR, abs/1909.0$$ 00393.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lever, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
Dong-Hyun Lee et al. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop challenges in representation learning, ICML, volume 3, page 896. Atlanta.
Francesco Pelosin, Saurav Jha, Andrea Torsello, Bogdan Raducanu, and Joost van de Weijer. 2022. Towards exemplar free continual learning in vision transform- ers: an account of attention, functional and weight regularization. In Proceedings of the IEFF/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3820-3829.
Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. 2020. A survey deep learning for named entity recognition. IÉEE Transactions on Knowledge and Data Engineering, 34(1):50-70.
Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEFF transactions pattern analysis and machine intelligence, 40(12):2935 2947.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarl: In- cremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001-2011
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized bert pretraining ap- proach. arXiv preprint
Anthony Robins. 1995. Catastrophic forgetting, re- hearsal and pseudorehearsal. Connection Science, 7(2):123146:
David Lopez-Paz and Marc' Aurelio Ranzato. 2017. Gradient episodic memory for continual lcarning, Ad- vances neural information processing systems, 30.
Chuck Rosenberg, Martial Hebert, and Henry Schnei- derman. 2005, Semi- Supervised Self- Training of Object Detection Models. In 7th IEEE Workshop on Applications of Computer Vision / HEEE Workshop Motion and Video Computing (WACVMOTION 2005), 5-7 January 2005, Breckenridge, CO. USA. pages 29-36.
Ruotian Ma, Xuanting Chen, Zhang Lin, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, and Yun Wen Chen. 2023. Learning "O" Helps for Leam- ing More: Handling the Unlabeled Entity Problem for Class-incremental NER. In Proceedings of the

---

Amir Rosenfeld and John Tsotsos. 2018. Incremen- tal learning through deep adaptation. IEEE transac- tions on pattern analysis and machine intelligence, 42(3):651-45 163663
Junhao Zheng. Zhanxian Liang, Haibin Chen, and Qianli Ma. 2022. Distilling Causal Effect from Mis- cellaneous Other- Class for Continual Named Entity Recognition. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.
Brik F Tjong Kim Sang and Fien De Meulder. 1837. Introduction to the CoNLL--2003 Shared Task: Language-Independemn Named Entity Recognition. Development, 932:1341.
A Supplementary Experimental Results
Table 4: Comparisons with baselines CoNLL2003. The red denotes the highest result, and the blue denotes the second highest result. The markers and refer to significant tests comparing with CENER and LUCIR, re- spectively (p-value < 0.05). represents results from re-implementation. Other baseline results are directly cited from CFNER (Zheng et al., 2022).
Antoine Saporta, Tuan- Hung Vu, Matthieu Cord, and Patrick Pérez. 2020. ESL: Entropy-guided guided Self- supervised Learning for Domain Adaptation in Se- mantic Segmentation. In Workshop on Scalability in Autonomous Driving at IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR).
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. Advances in neural information procexsing systems, 30.
FG-1-PG-1
FG-2-PG-1
Rascline
Mi-FI
Mi-FI
Mi-FI
Ma-FI
FT
50.84g0.10
40.64g0.16
57.45+0.05
43.58g0.18
PODNet
36.74=0.52
29.43=0.28
29.1240.54
58.3920.99
LUCIR
74.1550.43
70.48:00.66
80.53+0.31
77,33=0.31
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowiez, al. 2019, Huggingface's transformers: State-of- the-art natural language processing, arXiv preprint
ST
76.17=0.91
12.88
66.72=0.11
ExtendNER
76.0750.35
23.06:40.29
77.89 77.8941142
69.92e1.00 1.02
ExtendNER
76.3610.99 98
73.04s
76.6610 66
66.3660.64
CENER
80.2910.21
78.4410.24
52100.4
77,20.0.82
CENER
80.9120.29
79.1110.50
80.8340.36
75.20
CPFD (Ours)
82.24=00.31
79.94+00667
85.70=01.19
83.4940161
Imp. 11.33
10.83
[|4.18
16.16
Yu Xia, Quan Wang, Yajuan Lyu, Yong Zhu, Wenhao Wu, Sujian Li, and Dai Dai. 2022. Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Syathetic Samples. In Findings of the As- sociation for Computational Linguistics: ACL 2022,
FT - POONM + LGG
-
+ EnWNNER
CFNF
CPPD (PFD)
CoNLL2008 (FG-1-PG-1)
CoNLL2008 (FG-2.PG-1)
50-
60-
70-
pages 2291-2300.
90
50
Jaehong Yoon, Funho Yang, Jeongtae Lee, and Sung Ju Hwang. 2018. Lifelong Learning with Dynamically Expandable Networks. In International Conference on Learning Representations.
10
Number of Ertiy Types
Number Emity Types
Figure 6: Comparison of the step-wise Mi-F1 on CONLL.2003. The result of baselines is directly cited from CFNER (Zheng et al., 2022).
Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intel- ligence. In International Conference on Machine Learning, pages 3987-3995. PMLR.
+4-
POONN
+ LLCI ST + COWnNER
CMNT
CPTD (Qon))
Duzhen Zhang, Hongliu Li, Wei Cong, Rongtao Xu, Jiahua Dong, and Xiuyi Chen. 2023a. Task Rela- tion Distillation and Prototypical Pscudo Label for
(FG112203
CaNLL2008 (FG-2PP--1)
Incremental Named Entity Recognition. In Proceed- ings of the 32th ACM International Conference on Information & Knowledge Management.
62
40
30-
Number Entty Types
Number Entty Typos
Duzhen Zhang, Yahan Yu, Feilong Chen, and Xiuyi Chen. 2023b. Decomposing Logits Distillation for Incremental Named Entity Recognition. In Proceed- ings of the 46th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, pages 1919-1923.
Figure 7: Comparison of the step-wise Ma-Fl on CoNLL2003. The result of baselines is directly cited from CENER (Zheng et al., 2022).
Hanbin Zhao, Fengyu Yang, Xinghe Fu, and Xi I.. 2022. RBC: Rectifying the Biased Context in Continual Semantic Segmentation. European Conference on Computer Vision, pages 55-72. Springer.

---

POON!
LUOR
ST
EXWnSMER
OPNRR
(Duns)
1282 (F)-
(FG-2-22
(FG-8-PG-1)
1282
10
No.F
4.5
12 Number of Entty Types
16
Number Eniiyt Types
Number Emity Types
Number Emily Types
OntoNotessS (FG-1-P-11
OrioNotio5 (F3.2-PG-2)
Ontahiotes5 (FC-8-PG-1)
OnioNotee5 (FG-8-PG-2)
Mo-F1
10
Number Endly Types
Entty
Number of Endry Types
Number of Emtty Types
Figure 8: Comparison of the step-wise Ma-F1 on 12B2 and OntoNotes5. The result of baselines is directly cited from CENER (Zheng et al., 2022).


Reviews:
[{"review_id": "wWQEl5Vrz4", "reviewer": "Reviewer_CtRi", "summary": "", "questions": "Question A: Quantification analysis of the stability-plasticity trade off, similar to the Stability-Plasticity Curves in [1], is expected to verifies the claim that “our method establishes a suitable balance between stability and plasticity”.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comments. We would like to address your concerns and answer your questions in the following.\n\nQ1: This paper adopts similar attention regularization and uncertainty-aware distillation such as [1][2] to alleviate the stability-plasticity dilemma and semantic shift proposed by [3].\n\nA1: Thank you for your inquiry. Our distillation approach and the attention regularization presented in [1] both involve enhancing the degrees of freedom of the distillation loss by first performing pooling operations on the attention map with dimensions (head, height, width). This is done to alleviate the stability-plasticity dilemma. However, [1] only performs pooling on the attention map along the height and width dimensions. In addition, we conduct pooling along the head dimension. The multiple heads in self-attention can be likened to partitioning a large feature space into several distinct and non-overlapping sub-feature spaces. Subsequently, attention calculations are performed in parallel within each of these subspaces. This aids the network in capturing a more diverse set of representational information, bearing some resemblance to the concept of channels in CNNs. Pooling along the head dimension helps in disregarding unimportant noise information and distills crucial information from different subspaces. Here, we conduct supplementary ablation experiments on the I2B2 and OntoNotes5 datasets under the setting FG-1-PG-1, as shown in the table below. The results indicate that compared to performing attention map pooling along only the height and width dimensions followed by distillation ($L_{PFD\\ w/\\ height+width}$ [1]), performing attention map pooling along the head, height, and width dimensions followed by distillation ($L_{PFD\\ w/\\ head+height+width}$ (Ours)) can lead to improved CNER performance.\n\n| Distillation Loss| I2B2 dataset|  |OntoNotes5 dataset |  |\n| :-----: | :----: | :----: |:----: | :----: |\n| | Mi-F1 | Ma-F1 |  Mi-F1 | Ma-F1 |  \n|$L_{PFD\\ w/\\ head+height+width}$ (Ours) | **74.19±0.95**  | **48.34±1.45**  | **66.73±0.70**  | **54.12±0.30** |\n|$L_{PFD\\ w/\\ height+width}$ [1] |72.95±0.89| 47.01±1.11| 65.07±1.49| 52.77±0.85|\n\n[2] proposes uncertainty-aware distillation to help the model preserve useful information from the previous tasks, not specifically to alleviate the semantic shift issue. The issue refers to the scenario in the CNER paradigm where tokens labeled as the non-entity type could encompass: the actual non-entity type, previously learned old entity types, or future ones that have not been encountered yet. To address this issue, our confidence-based pseudo-labeling strategy explicitly retrieves old entity types within the current non-entity type via the old model for classification. To better reduce the recognition errors from the old model, it employs entropy as a measure of uncertainty and the median entropy as a confidence threshold, retaining only those pseudo labels where the old model exhibits sufficient confidence.\n\n\nAlthough [3] was the first to identify the semantic shift problem in CNER, it didn't effectively address this concern. There are two significant differences between their approach and ours. Firstly, [3] employs the old model to identify non-entity type tokens belonging to old entity types for distilling causal effect. In contrast, we explicitly retrieve old entity types within the current non-entity type for classification. Our experimental results in Table 1 of the paper demonstrate that the latter approach is more effective. Secondly, [3] does't sufficiently handle recognition errors by the old model. While they introduce a curriculum learning strategy to mitigate recognition errors, this strategy requires the manual pre-definition of numerous hyper-parameters, limiting its applicability and the effect of error reduction. In contrast, our confidence-based pseudo-labeling strategy employs median entropy as the confidence threshold, eliminating the need for manually predefining any hyper-parameters, better reducing the recognition error from the old model, and dealing with the semantic shift problem.\n\n\n\n\nQ2:  Quantification analysis of the stability-plasticity trade off, similar to the Stability-Plasticity Curves in [1], is expected to verifies the claim that “our method establishes a suitable balance between stability and plasticity”.\n\nA2: Thank you for your valuable suggestion. Following the definition in [1], we have plotted the task-aware Stability-Plasticity Curves for the FG-1-PG-1 settings of the I2B2 and Ontonotes5 datasets to compare the stability-plasticity dilemma for various methods, including Ours w/ $L_{FD}$, Ours w/ $L_{PFD-lax}$, and Ours (equipped with $L_{PFD}$ by default). The specific definitions of these three distillation losses can be found in Section 4.1 of our paper. $L_{FD}$ is a rigid loss, which compels the new model to strictly align with the attention weights found in the old model. This practice tends to infuse the model with excessive stability (i.e., preservation of old knowledge), thereby compromising its plasticity (i.e., ability to absorb new knowledge). This is reflected in the curves, where as the number of incremental tasks increases, the stability score remains consistently high, while the plasticity score for each task is relatively low. $L_{PFD-lax}$ is a permissive loss, which utilizes a more aggressive pooling to lessen the similarity between the new model and the old model. In contrast to Ours w/ $L_{FD}$, Ours w/ $L_{PFD-lax}$'s curve looks like this: as the number of incremental tasks increases, its stability score gradually decreases, while the plasticity score for each task is relatively high. Finally, $L_{PFD}$ achieves an appropriate balance between excessive stability and extreme plasticity via adopting a less aggressive pooling. From the curves, it is evident that as the number of incremental tasks increases, Ours consistently achieves the highest scores for both stability and plasticity. Due to the inconvenience of inserting images in the response box, we will add these task-aware Stability-Plasticity curve graphs in the final version.\n\n\nQ3: Missing References [1]. \n\nA3: Thank you for bringing this to our attention. We apologize for the oversight in not including the reference [1] you mentioned. We will include this reference [1] in the appropriate section of the final version. Your diligence in reviewing our work is greatly appreciated. \n\n\n**Reference**\n\n[1] Pelosin F, Jha S, Torsello A, et al. Towards exemplar-free continual learning in vision transformers: an account of attention, functional and weight regularization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 3820-3829.\n\n[2] Kurmi, Vinod K., et al. \"Do not forget to attend to uncertainty while mitigating catastrophic forgetting.\" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2021. \n\n[3] Zheng, Junhao, et al. \"Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition.\" Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well-written and experimental results verifies the effectiveness.", "reasons_to_reject": "This paper adopts similar attention regularization and uncertainty-aware distillation such as [1][2] to alleviate the stability-plasticity dilemma and semantic shift proposed by [3].\n\n[1] Pelosin F, Jha S, Torsello A, et al. Towards exemplar-free continual learning in vision transformers: an account of attention, functional and weight regularization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 3820-3829.\n[2] Kurmi, Vinod K., et al. \"Do not forget to attend to uncertainty while mitigating catastrophic forgetting.\" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2021.\n[3] Zheng, Junhao, et al. \"Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition.\" Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.\n", "questions_to_authors": "Question A: Quantification analysis of the stability-plasticity trade off, similar to the Stability-Plasticity Curves in [1], is expected to verifies the claim that “our method establishes a suitable balance between stability and plasticity”.", "paper_topic_and_main_contributions": "To alleviate the stability-plasticity dilemma common in continual learning, pooled features distillation is adopted.\nTo address the semantic shift problem specific to CNER, this paper filters low-confidence pseudo labels and assigns different weights to tokens of different types based on the number of tokens.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Pelosin F, Jha S, Torsello A, et al. Towards exemplar-free continual learning in vision transformers: an account of attention, functional and weight regularization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 3820-3829."}}, {"review_id": "90DOpZo6tE", "reviewer": "Reviewer_nTza", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your insightful feedback. We are committed to addressing your concerns and providing answers to your questions below.\n\nQ1: The proposed approach may not apply to all NLP tasks and datasets, and its effectiveness may depend on specific data characteristics.\n\nA1: Thank you for your thoughtful question. Our CPFD method consists of two parts, Confidence-based pseudo-labeling and Pooled Features Distillation. The former is designed to address specific issues in Continual Named Entity Recognition (CNER), namely the semantic shift of the non-entity type. The latter is employed to tackle the common challenge of continual learning, denoted as catastrophic forgetting. In other words, our Pooled Features Distillation (PFD) can be applied to almost all NLP continual learning tasks. In addition to the CNER task presented in our paper, we also provide results on continual text classification and continual relation learning tasks.\n\n**continual text classification** \n\nDue to time constraints, we conducted experiments on one dataset, THU-Seq, which is a task sequence constructed from THUCNews. The results are shown in the table below. All baseline results are directly quoted from [1]. Snapshot is the previous State-Of-The-Art (SOTA) method for continual text classification. $*$ represents results based on our re-implementation using open-source code. \n\n| THU-Seq:        | THU$_1$         | THU$_2$         | THU$_3$         | THU$_4$         |\n| --------------- | --------------- | --------------- | --------------- | --------------- |\n| MTL             | $99.0$          | $95.3$          | $95.9$          | $95.7$          |\n| FT              | $98.9$          | $61.2$          | $37.1$          | $27.4$          |\n| LwF             | $98.9$          | $76.8$          | $53.0$          | $43.8$          |\n| EWC             | $98.9$          | $82.5$          | $54.6$          | $51.8$          |\n| R-Walk          | $98.9$          | $84.9$          | $81.2$          | $73.5$          |\n| MAS             | $98.9$          | $86.4$          | $82.3$          | $60.1$          |\n| AdapterParallel | $98.7$          | $72.0$          | $71.6$          | $69.4$          |\n| AdapterStack    | $98.7$          | $55.6$          | $50.7$          | $33.3$          |\n| AdapterFusion   | $98.7$          | $52.7$          | $36.2$          | $26.5$          |\n| Snapshot        | $98.9{\\pm0.1}$ | $87.9{\\pm0.2}$ | $89.2{\\pm0.4}$ | $86.3{\\pm0.4}$ |\n| Snapshot + Our PFD*       | ${98.9{\\pm 0.1}}$ | $\\mathbf{89.3{\\pm0.1}}$ | $\\mathbf{90.3{\\pm0.3}}$ | $\\mathbf{87.6{\\pm0.2}}$ |\n\n\n**continual relation learning**\n\nWe conducted experiments on FewRel dataset. The results are shown in the table below. All baseline results are directly quoted from [2]. CRL is the previous SOTA method for continual relation learning. $\\dagger$ denotes [2]'s reproduced results with the open codebases. $*$ represents results based on our re-implementation using open-source code. \n\n| Model           | T1              | T2          | T3              | T4              | T5              | T6              | T7              | T8              | T9              | T10             |\n| --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |\n| EA-EMR          | $89.0$          | $69.0$          | $59.1$          | $54.2$          | $47.8$          | $46.1$          | $43.1$          | $40.7$          | $38.6$          | $35.2$          |\n| EMAR            | $88.5$          | $73.2$          | $66.6$          | $63.8$          | $55.8$          | $54.3$          | $52.9$          | $50.9$          | $48.8$          | $46.3$          |\n| CML             | $91.2$          | $74.8$          | $68.2$          | $58.2$          | $53.7$          | $50.4$          | $47.8$          | $44.4$          | $43.1$          | $39.7$          |\n| EMAR+BERT       | $\\mathbf{98.8}$ | $89.1$          | $89.5$          | $85.7$          | $83.6$          | $84.8$          | $79.3$          | $80.0$          | $77.1$          | $73.8$          |\n| RP-CRE          | $97.9$          | $92.7$          | $91.6$          | $89.2$          | $88.4$          | $86.8$          | $85.1$          | $84.1$          | $82.2$          | $81.5$          |\n| RP-CRE$\\dagger$ | $98.4$          | $95.2$          | $93.1$          | $91.4$          | $90.8$          | $88.8$          | $87.6$          | $86.8$          | $85.2$          | $83.9$          |\n| CRL             | $98.3$          | $\\{95.4}$ | $\\{93.4}$ | $\\{92.0}$ | $\\{91.0}$ | $\\{89.7}$ | $\\{88.3}$ | $\\{87.0}$ | $\\{85.6}$ | $\\{84.4}$ |\n| CRL   + Our PFD*          | $98.6$          | $\\mathbf{95.7}$ | $\\mathbf{93.8}$ | $\\mathbf{92.8}$ | $\\mathbf{91.3}$ | $\\mathbf{89.9}$ | $\\mathbf{89.4}$ | $\\mathbf{87.5}$ | $\\mathbf{86.4}$ | $\\mathbf{85.3}$ |\n\n\nBased on the results from the aforementioned table, our PFD consistently improves the performance of SOTA methods across various distinct NLP continual learning tasks. This is attributed to its skillful navigation of the balance between retaining old knowledge and acquiring new ones, thus more effectively addressing the challenge of catastrophic forgetting.\n\n\nQ2: The paper does not compare the proposed approach with other state-of-the-art methods outside the CNER domain, which may limit its generalizability to other NLP tasks.\n\nA2: Thank you for your insightful comment. We applied the state-of-the-art methods Snapshot[1], CRL[2], and Dynamic Knowledge Distillation[3] from domains continual text classification, continual relation learning, and continual machine translation, respectively, to the CNER domain. The experimental results on the I2B2 dataset under the setting FG-8-PG-2 are presented below. $*$ represents results based on our re-implementation using open-source code. Other baseline results are cited from CFNER[4].\n\n| Baseline | Mi-F1 | Ma-F1|\n| :------: | :------: | :------: |\n| FT | 23.60±0.15 | 23.54±0.38 | \n| PODNet |36.22±12.9 | 26.08±7.42\n| LUCIR |68.54±0.27 | 46.94±0.63|\n| ST |48.94±6.78 | 29.00±3.04|\n| Extend* | 56.48±2.41 | 38.88±1.38 |\n| Extend | 52.25±5.36  | 30.93±2.77 |\n| CFNER* |69.12±0.94 | 51.61±0.87 | \n| CFNER | 69.07±0.89 | 51.09±1.05 |\n| Snapshot* | 49.24±2.44 | 29.12±1.58 | \n| CRL* | 66.19 ± 0.92 | 49.83 ± 0.62| \n| Dynamic Knowledge Distillation* | 63.69±0.60 | 42.99±1.13 | \n|CPFD (Ours)| **81.05±0.87** | **65.04±1.13** |\n\n\nFrom the results in the above table, it is evident that the three methods, Snapshot, CRL, and Dynamic Knowledge Distillation, prioritize addressing the common issue of forgetting while overlooking the specific challenge of semantic shift in the CNER domain. As a consequence, their performance in the CNER field falls below that of our method.\n\n\nQ3: The paper does not provide a detailed discussion of the computational complexity of the proposed approach, which may be a concern for large-scale datasets or real-time applications.\n\nA3: Thank you for raising this important point. When confronted with the acquisition of a new task, the computational intricacies of our CPFD framework are contingent on two conditions: 1) In cases where the task count stands at 1 (i.e., $t=1$), the primary computational expense of CPFD lies in the updating of $\\theta_t$ (where $t=1$). To be more precise, the expenditure associated with updating $\\theta_t$ is denoted as $O(\\xi(N,n_t))$, where the function $\\xi(\\cdot)$ hinges on the single-task learner (namely, the base model), and $n_t$ signifies the quantity of training samples within task $t$, while $N$ signifies the quantity of model parameters. Consequently, upon the advent of a new task, the comprehensive computational complexity inherent in our proposed model materializes as $O(\\xi(N,n_t))$. In scenarios where the task count surpasses 1 (i.e., $t > 1$), the principal computational expense of assimilating a new task encompasses two distinct subproblems: the first optimization problem resides within Equation (3), while the second is encapsulated in Equation (6). For the problem in Eq (3), the cost of updating $\\theta_t$ is $O(\\xi(N, n_t))$. For the problem in Equation (6), the expenditure associated with calculating pseudo-labels $\\widehat{Y}_{t-1}$ stands at $O(\\zeta(N, n_t))$, with $\\zeta(\\cdot)$ denoting the inference procedure of the old model. This inference procedure encompasses a single neural network forward propagation, the computational overhead of which pales in comparison to the training process. The update of $\\theta_t$ bears a cost of $O(\\xi(N, n_t))$, whereas the computation associated with the adaptive weight $\\eta$ incurs a cost of $O(n_t)$. The overall computational complexity of our proposed method is $O(\\xi(N, n_t)+\\zeta(N, n_t)+n_t)$. \n\n\nQ4: The authors need to tell the strengths and weaknesses of their approach and provide a balanced and nuanced discussion of their findings. And the paper does not provide a detailed analysis of the limitations of the proposed approach, such as cases where it may not work well or situations where it may be less effective.\n\nA4: Thank you for your thoughtful feedback. \n\n**Strengths:** **1) Catastrophic Forgetting Mitigation:** Our method incorporates a pooled features distillation loss, effectively mitigating catastrophic forgetting. This mechanism enables the retention of linguistic knowledge while striking a harmonious balance between stability and plasticity. **2) Semantic Shift Handling:** Our method introduces a confidence-based pseudo-labeling strategy to better recognize old entity types for the current non-entity type tokens and deal with the semantic shift problem. **3) Type Imbalance Handling:** To contend with imbalanced type distributions, our method proposes an adaptive re-weighting type-balanced learning strategy tailored for continual named entity recognition. **4) State-of-the-art performance:** The outcomes of our comprehensive experiments, conducted across ten different CNER settings on three distinct datasets, substantiate the effectiveness of our method. It surpasses the previous state-of-the-art methods, showcasing an average gain of 6.3\\% in Micro F1 scores and 8.0\\% in Macro F1 scores.\n\n**Weaknesses/Limitations:**  Due to the recognition errors of the old model, the pseudo-label strategy inevitably introduces some noisy labels. When the amount of noisy labels is substantial, it can potentially impact the model's performance. Although our confidence-based pseudo-labeling strategy can mitigate the recognition errors of the old model by employing a median entropy as a confidence threshold and retaining only those pseudo labels where the old model exhibits sufficient confidence, it is not completely immune to errors, and some mislabeled instances may still persist.\n\n\n**Reference**\n\n[1] Wang J, Dong D, Shou L, et al. Effective Continual Learning for Text Classification with Lightweight Snapshots[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(8): 10122-10130.\n\n[2] Zhao K, Xu H, Yang J, et al. Consistent Representation Learning for Continual Relation Extraction[C]//Findings of the Association for Computational Linguistics: ACL 2022. 2022: 3402-3411.\n\n[3] Cao Y, Wei H R, Chen B, et al. Continual learning for neural machine translation[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021: 3964-3974.\n\n[4] Zheng, Junhao, et al. \"Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition.\" Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The strengths of this paper are its novel approach to addressing the problem of catastrophic forgetting in Continual Named Entity Recognition (CNER), the extensive experiments conducted on ten CNER settings of three datasets to demonstrate the approach's effectiveness, and the significant improvements achieved over previous state-of-the-art methods. The paper's proposed approach and experimental results could inspire further research and improve performance in CNER and other related tasks.", "reasons_to_reject": "Some potential weaknesses of this paper could include:\n1. The proposed approach may not apply to all NLP tasks and datasets, and its effectiveness may depend on specific data characteristics.\n2. The paper does not provide a detailed analysis of the limitations of the proposed approach, such as cases where it may not work well or situations where it may be less effective.\n3. The paper does not compare the proposed approach with other state-of-the-art methods outside the CNER domain, which may limit its generalizability to other NLP tasks.\n4. The paper does not provide a detailed discussion of the computational complexity of the proposed approach, which may be a concern for large-scale datasets or real-time applications.\nThe authors need to tell the strengths and weaknesses of their approach and provide a balanced and nuanced discussion of their findings.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper addresses two challenges in Chinese Named Entity Recognition (CNER): catastrophic forgetting and non-entity type semantic transfer issues. To tackle these problems, the paper adopts a phased approach and proposes corresponding strategies. It introduces an aggregated feature distillation loss to balance stability and plasticity, effectively mitigating catastrophic forgetting. Additionally, a confidence-based pseudo-labeling plan is offered to explicitly extract old entity types embedded within non-entity types, thereby reducing the impact of label noise and handling the semantic transfer problem. The paper further validates the significance of collaboration between all components in solving the CNER problem through ablation experiments.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1f9CBlxMb6", "reviewer": "Reviewer_xCyC", "summary": "", "questions": "Qestion A: Why using the entropy to measure the confidence of the old model? Can we use the maximum prediction score as the metric to select pseudo-labels?\nQestion B: Why distiling the attention score matrix instead of other model parameters, e.g., the projection matrices in self attention or MLP?\nQestion C: The paper identifies the type-imbalance issue where the classifier tends to predict new classes. Why it is important to address this issue? What are the differences and connections between the type-imbalance and the class-imbalance issue.\n\n==============================================================\n\nPost-rebuttal:\nMost of my concerns have been addressed.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful reviews. We would like to address your concerns and answer your questions in the following.\n\nQ1: This paper only conduct experiments on BERT, the performance on larger models is unclear.\n\nA1: Thank you for your valuable feedback. We have included experiments on larger models, including bert-large-cased and roberta large. The results on the I2B2 dataset under the setting FG-8-PG-2 are as follows:\n\n| Backbone | Avg. Mi-F1 | Avg. Ma-F1|\n| -------- | -------- | -------- |\n| bert-base-cased (original paper) | 81.05 $\\pm$ 0.87 | 65.04 $\\pm$ 1.13 |\n| bert-large-cased | 82.04 $\\pm$ 0.54 | 66.98 $\\pm$ 0.83|\n| roberta-large  | **82.58 $\\pm$ 0.91** | **67.61 $\\pm$ 1.08** |\n\nDue to time constraints, the results of other settings will be provided in the final version. In the future, we will also attempt to use larger and more advanced pre-trained language models as the backbone encoder.\n\n\nQ2: The performance under different hyper-parameter settings (e.g., $\\lambda$) is unclear.\n\nA2: Thank you for raising this question. We have conducted an analysis of hyperparameter $\\lambda$ on the I2B2 dataset under the FG-8-PG-2 setting. This hyperparameter is utilized for loss balancing, and the results are as follows:\n\n| $\\lambda$ | Avg. Mi-F1 | Avg. Ma-F1|\n| -------- | -------- | -------- |\n| 0.5 | 80.23 $\\pm$ 0.56 | 64.59 $\\pm$ 0.93 |\n| 1  |  80.12 $\\pm$ 0.62 | 64.41 $\\pm$ 0.78 |\n| 2  |  **81.05 $\\pm$ 0.87** | **65.04 $\\pm$ 1.13** |\n| 5  |  77.29 $\\pm$ 0.76 | 61.82 $\\pm$ 1.32 |\n\nThe best performance is achieved when $\\lambda$ is set to $2$.\n\n\nQ3: Why use entropy to measure the confidence of the old model? Can we use the maximum prediction score as the metric to select pseudo-labels? \n\nA3: Thank you for your thoughtful question. The use of entropy as a measure to assess the confidence of the old model's predictions stems from its ability to capture the uncertainty inherent in the prediction distribution. Low entropy indicates a state of less uncertainty, suggesting that the model is confident in its prediction. This aligns with our intention to select pseudo-labels where the old model is more certain. While using the maximum prediction score could be an alternative approach, it can be unproductive, e.g., on uncertain tokens where the old model is likely to fail. This Vanilla Pseudo-Labeling (VPL) approach more easily propagates errors from the old model's incorrect predictions to the new model. To better reduce the recognition errors from the old model, our Confidence-based Pseudo-Labeling (CPL) approach employs entropy as a measure of uncertainty and the median entropy as a confidence threshold, only retaining those pseudo labels where the old model exhibits sufficient confidence. Furthermore, the ablation experiment results in Table 2 of our paper also demonstrate the superior performance of CPL over VPL, indicating that using entropy is more effective than using the maximum prediction score. \n\n\nQ4: Why distiling the attention score matrix instead of other model parameters, e.g., the projection matrices in self attention or MLP? \n\nA4: Thank you for your insightful question. Recent studies [1] suggest that attention scores learned by pre-trained language models can encapsulate rich linguistic knowledge, including coreference and syntactical information, both critical to natural language understanding tasks like named entity recognition. In light of this, we distill the attention score matrix to encourage the transfer of linguistic knowledge from the old model to the new one. However, we appreciate your suggestion and will consider exploring the impact of distilling other model parameters (e.g., the projection matrices in self attention or MLP) in future research iterations. Your feedback is invaluable in refining our approach.\n\n\nQ5: The paper identifies the type-imbalance issue where the classifier tends to predict new classes. Why is it important to address this issue? What are the differences and connections between the type-imbalance and the class-imbalance issue.\n\nA5: Thank you for your insightful question. Following the pseudo-labeling process, we find that the count of new-type tokens present in current sequences generally exceeds the count of pseudo-labeled old-type tokens. This type-imbalance issue typically skews the updated classifier towards new types. Resolving the type-imbalance issue is crucial for creating models that are accurate, fair, and capable of making informed decisions across all entity types, regardless of their distribution. Here, the type-imbalance issue is essentially the class-imbalance issue. In alignment with prior CNER methods, we adopt the \"BIO\" tagging schema for all three dataset. This means that each entity type corresponds to two labels (classes): B-type and I-type, where B-/I- distinguishes the beginning/inside of an entity.  \n\n\nQ6: The caption of Figure 2 is a little bit confusing. What do the \"[X] Not Operation\" and \"[IGN] Ignored Label\" mean?\n\nA6: Thank you for bringing up this point. We apologize for any confusion caused by the caption of Figure 2. The meaning of \"[X] Not Operation\" is that if the current token has not been labeled as the non-entity type [O] (i.e., it has been labeled as the current entity type being learned), no pseudo-labeling operation is performed. In other words, the pseudo-labeling strategy is only applied to tokens currently labeled as [O]. The meaning of \"[IGN] Ignored Label\" is that our confidence-based pseudo-labeling strategy retains pseudo-labels generated by the old model only if they have a sufficient level of confidence. Pseudo-labels with lower confidence will be disregarded and will not be included in the computation of the cross-entropy loss.\n\n\n**Reference**\n\n[1] Clark K, Khandelwal U, Levy O, et al. What Does BERT Look at? An Analysis of BERT’s Attention[C]//Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. 2019: 276-286.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation is clear: this paper focuses on the semantic shift problem in Conintual Named Entity Recogntion, which is overlooked by previous methods.\n2. The comparison are extensive: this paper compares the proposed method with other baselines on different settings and datasets.", "reasons_to_reject": "1. This paper only conduct experiments on BERT, the performance on larger models is unclear.\n2. The performance under different hyper-parameter settings (e.g., \\lambda) is unclear.", "questions_to_authors": "Qestion A: Why using the entropy to measure the confidence of the old model? Can we use the maximum prediction score as the metric to select pseudo-labels?\nQestion B: Why distiling the attention score matrix instead of other model parameters, e.g., the projection matrices in self attention or MLP?\nQestion C: The paper identifies the type-imbalance issue where the classifier tends to predict new classes. Why it is important to address this issue? What are the differences and connections between the type-imbalance and the class-imbalance issue.\n\n==============================================================\n\nPost-rebuttal:\nMost of my concerns have been addressed.", "paper_topic_and_main_contributions": "This paper lays the foundation for future investigations in CNER, a novel area within NLU. The primary focus of this research is to address two key challenges in CNER: catastrophic forgetting and the semantic shift problem of the non-entity type.To tackle these challenges, this paper proposes pooled feature distillation loss and confidence-based pseudo-labeling. This paper evaluates conduct experiments on ten CNER settings across three datasets and show that the proposed method significantly outperforms the previous SOTA methods consistently.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "1. The caption of Figure 2 is a little bit confusing. What do the \"[X] Not Operation\" and \"[IGN] Ignored Label\" mean?"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary:
This paper addresses two challenges in Chinese Named Entity Recognition (CNER): catastrophic forgetting and non-entity type semantic transfer issues. To tackle these problems, it introduces a phased approach with corresponding strategies. This includes the introduction of an aggregated feature distillation loss to balance stability and plasticity, effectively mitigating catastrophic forgetting. Additionally, it proposes a confidence-based pseudo-labeling plan to explicitly extract old entity types within non-entity types, reducing label noise's impact and handling semantic transfer. The significance of component collaboration in solving CNER is validated through ablation experiments.

Reason To Accept:
1.The motivation behind this work is clear: it addresses the semantic shift problem in Continual Named Entity Recognition (CNER), a challenge overlooked by previous methods.
2.The paper's strengths lie in its innovative approach to tackling catastrophic forgetting in CNER.
3.The paper is well-written, and its experimental results validate its effectiveness. It offers a comprehensive comparison of the proposed method with other baselines across various settings and datasets. Extensive experiments conducted on ten CNER settings involving three datasets showcase the approach's effectiveness and significant improvements over previous state-of-the-art methods. The proposed approach and experimental results have the potential to inspire further research and enhance performance in CNER and related tasks.

Reason To Reject:
1.This paper adopts similar attention regularization and uncertainty-aware distillation such as [1][2] to alleviate the stability-plasticity dilemma and semantic shift proposed by [3]. ([1] Pelosin F, Jha S, Torsello A, et al. Towards exemplar-free continual learning in vision transformers: an account of attention, functional and weight regularization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 3820-3829. [2] Kurmi, Vinod K., et al. "Do not forget to attend to uncertainty while mitigating catastrophic forgetting." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2021. [3] Zheng, Junhao, et al. "Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition." Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.)
2.The experiments are inadequate. 1) The paper only conducts experiments on BERT, leaving the performance on larger models unexplored. 2) The proposed approach may not be universally applicable to all NLP tasks and datasets, with its effectiveness potentially contingent on specific data characteristics. 3) The paper lacks comparisons of the proposed approach with other state-of-the-art methods outside the CNER domain, limiting its generalizability to other NLP tasks.
3.Certain details and analyses are lacking or unclear. 1) The performance under different hyper-parameter settings (e.g., \lambda) is unclear. 2) A comprehensive analysis of the proposed approach's limitations, including scenarios where it might perform poorly or be less effective, is missing. 3) There's no detailed discussion regarding the computational complexity of the proposed approach, which could be a concern for large-scale datasets or real-time applications. The authors should provide a balanced and nuanced discussion of both the strengths and weaknesses of their approach.
Decision: Accept-Main