Flexible Weight Tuning and Weight Fusion Strategies for Continual Named Entity Recognition (2024)

Abstract: Continual Named Entity Recognition (CNER) is dedicated to sequentially learning new entity types while mitigating catastrophic forgetting of old entity types. Traditional CNER approaches commonly employ knowledge distillation to retain old knowledge within the current model. However, because only the representations of old and new models are constrained to be consistent, the reliance solely on distillation in existing methods still suffers from catastrophic forgetting. To further alleviate the forgetting issue of old entity types, this paper introduces flexible Weight Tuning (WT) and Weight Fusion (WF) strategies for CNER. The WT strategy, applied at each training step, employs a learning rate schedule on the parameters of the current model. After learning the current task, the WF strategy dynamically integrates knowledge from both the current and previous models for inference. Notably, these two strategies are model-agnostic and seamlessly integrate with existing State-Of-The-Art (SOTA) models. Extensive experiments demonstrate that the WT and WF strategies consistently enhance the performance of previous SOTA methods across ten CNER settings in three datasets.1

heading: 1 Introduction
text: As a pivotal task in information extraction, Named Entity Recognition (NER) plays a crucial role in various applications, including question answering (Li et al., 2019; Longpre et al., 2021), web search queries (Guo et al., 2009; Zhang et al., 2021). Traditional fully-supervised NER endeavors to classify tokens in a sentence into fixed entity types (Ma and Hovy, 2016). However, real-world entity types typically emerge in a streaming manner, as seen in voice assistants like Siri, which need to recognize new entity types (e.g., Band, Song) to understand
*Corresponding author. 1Our code is available at https://github.com/ku-
nlp/CNER_WT-WF.
new user intents (e.g., GetMusic) (Zhang et al., 2023c). One straightforward solution involves retraining the model on the entire dataset by incorporating both old and new entity types. Nevertheless, this strategy incurs substantial training costs. Another simplistic approach is to fine-tune the previously learned model exclusively on the newly introduced entity types, a scenario known as Continual NER (CNER) (Monaikul et al., 2021; Ma et al., 2023). However, CNER commonly faces catastrophic forgetting, wherein the knowledge acquired from previous entity types is lost after learning new ones (McCloskey and Cohen, 1989; Robins, 1995; Goodfellow et al., 2013; Kirkpatrick et al., 2017; Dong et al., 2022, 2023, 2024; Zheng et al., 2023).
To address the issue of forgetting old entity types, prior CNER methods often employ knowledge distillation to preserve previous knowledge in the current model (Hinton et al., 2015; Zhang et al., 2023b; Chen and He, 2023). These approaches maintain output logits or intermediate features to prevent substantial changes in the parameters of the current model. Specifically, ExtendNER (Monaikul et al., 2021) distills output logits from the old model to the new model, encouraging the new model to generate logits closely resembling those produced by the old model. L&R (Xia et al., 2022) utilizes a twostage learn-and-review framework. In the learning stage, it adopts a similar approach to ExtendNER; while during the reviewing stage, it integrates synthesized samples of old entity types to enhance the current dataset. Extending the ideas of ExtendNER, CFNER (Zheng et al., 2022) introduces a causal framework to distill causal effects from the nonentity type. CPFD (Zhang et al., 2023a) presents a feature distillation method to retain linguistic knowledge in attention weights, achieving StateOf-The-Art (SOTA) CNER performance. However, since only the output or internal representations of old and new models are constrained to be consistent, relying on knowledge distillation alone pro-
1351
vides limited gains in CNER performance. To address this limitation, we propose flexible Weight Tuning (WT) and Weight Fusion (WF) strategies for CNER. Shown in Figure 1, WT applied at training, utilizes a learning rate schedule for the new model’s parameters. The initial learning rate gradually decays with steps to effectively retain knowledge of historical entity types. At inference, WF incorporates knowledge from both the new and old models in the form of parameter addition with a dynamic factor. This process eventually yields a fused model for inference, striking a new balance between old and new knowledge without incurring additional computation costs, thereby further mitigating the forgetting of old entity types. Crucially, the WT and WF strategies are model-agnostic, seamlessly integrating with existing SOTA methods in a plug-and-play fashion.
Our contributions can be summarized as follows:
• We present flexible WT and WF strategies for CNER to further mitigate catastrophic forgetting. The former incorporates a learning rate decay schedule during training, while the latter effectively establishes a new balance between old and new knowledge for inference.
• We perform comprehensive experiments across ten CNER settings on three datasets. The outcomes demonstrate the efficacy of the WT and WF strategies, consistently enhancing the performance of SOTA CNER methods.

heading: 2 Preliminary
text: CNER endeavors to learn a model over t = 1, ..., T steps, progressively acquiring proficiency in an expanding array of entity types. Each step involves a distinctive training set Dt, consisting of multiple pairs (Xt,Y t), where Xt denotes an input token sequence with a length of |Xt| and Y t signifies the corresponding ground truth label sequence encoded in a one-hot format. It is noteworthy that Y t exclusively encompasses labels for the current entity types E t, while all other labels are masked as the non-entity type eo. At step t (t>1), taking into account the old modelMt−1 with parameters θt−1 and the current training set Dt, we aim to train a new modelMt with parameters θt capable of recognizing all entities types encountered thus far, as denoted by ⋃t i=1 E i.
Previous CNER methods (Monaikul et al., 2021; Xia et al., 2022; Zheng et al., 2022; Zhang et al., 2023a) commonly employ a Cross-Entropy (CE) term for acquiring knowledge about new entity types and a Knowledge Distillation (KD) term to preserve the knowledge acquired from previous entity types. This formulation is expressed as follows:
LCE(θt) = −Y t log Ŷ t
LKD(θt) = { −Ŷ t−1 log Ŷ t ||F t−1 − F t||2 , (1)
where Ŷ t−1 and Ŷ t denote the output distributions of Xt generated by the old modelMt−1 and the new modelMt, respectively. Moreover, F t−1 and F t represent the intermediate features of the old modelMt−1 and the new modelMt, respectively.

heading: 3 Method
text: To further alleviate the forgetting of old entity types, we introduce model-agnostic WT and WF strategies to augment existing CNER methods.
The WT strategy implements a learning rate schedule for each continual training step t to adjust the parameters θt of the current modelMt. Specifically, during the initial step (t=1), the learning rate is set to lr1. For subsequent steps (t>1), the learning rate is gradually reduced to better retain knowledge of historical entity types. Therefore, the learning rate lrt at step t is expressed as:
lrt = { lr1 if t = 1 e−αt · lr1 if t > 1 , (2)
Algorithm 1 Pseudo Code for WT and WF strategies in continual steps (highlighted in red font)
Require: θ0, T,Dt, initial learning rate lr1, and hyper-parameters α and β t← 1 ▷ Continual training step while t ≤ T do
θt ← θt−1 if t = 1 then
lrt← lr1 else
lrt← e−αt · lr1 end if i← 1 ▷ Iteration step while not converged do
Sample mini-batch {Xti ,Y ti } ∼ Dt θti+1 ← θti− lrt∇LCE+LKD from Equation (1) i← i+ 1 end while Initialize Ctnew,Ctold ▷ Number of new
and old entity types in the current step t λt← ( Ctnew
Ctnew+C t old
)β
θfused← λtθt + (1− λt)θt−1 θt← θfused t← t+ 1
end while
where e−t denotes the exponential decay of the learning rate, and α is a hyper-parameter.
Following each training step t (t>1), the WF strategy, instead of employing the new model directly, produces a fused model for inference. This fused model integrates knowledge from both the new and old models using a dynamic balance factor, establishing a refined equilibrium between old and new knowledge without introducing additional computation costs. The formulation is as follows:
θfused = λtθt + (1− λt)θt−1
λt =
( Ctnew
Ctnew + C t old
)β , (3)
where Ctnew and C t old represent the number of new and old entity types in the current step t, respectively, and β is a hyper-parameter.
The pseudo code for our WT and WF strategies is presented in Algorithm 1.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: To ensure a fair comparison, we adhere to the setup of CPFD (Zhang et al., 2023a) method as below:
Datasets We use three widely adopted NER datasets to assess the impact of WT&WF strategies, namely CoNLL2003 (Sang and De Meulder, 1837), I2B2 (Murphy et al., 2010), and OntoNotes5 (Hovy et al., 2006). The statistics for these datasets are shown in Table 1. We apply the greedy sampling algorithm, detailed in CFNER (Zheng et al., 2022), to split the training set into disjoint slices, each corresponding to different continual steps. Within each slice, we retain only the labels belonging to the entity types under learn, while masking others as the non-entity type.
CNER Settings During training, we sequentially learn entity types in alphabetical order and train models with corresponding slices. For CoNLL2003, we employ two CNER settings: 1-1 and 2-1. Regarding I2B2 and OntoNotes5, we establish four CNER settings: 1-1, 2-2, 8-1, and 8-2. The notation a-b indicates that we utilize a entity types to learn a base model and every b entity type to train in each continual step. During validation, we retain only the labels of the current entity types under learning, masking others as the non-entity type within the validation set. At each step, we select the model that achieves the best validation performance for both testing and the subsequent step. In the testing phase, we preserve labels for all previously learned entity types, designating the remainder as the non-entity type within the test set.
Evaluation Metrics Given the challenge of entity type imbalance in NER, we employ Micro F1 (Mi-F1) and Macro F1 (Ma-F1) to assess the performance. The reported result denotes the mean across all steps, including the first one, serving as the final performance. To assess the significance of the performance improvement, we conduct a paired t-test with a significance level of 0.05 (Koehn, 2004).
Baselines We consider the following baselines for evaluation, encompassing SOTA CNER methods: ExtendNER (Monaikul et al., 2021), CFNER (Zheng et al., 2022), and CPFD (Zhang
et al., 2023a). Moreover, we include the lower bound method, Fine-tuning, directly employing new data for fine-tuning the model without using any anti-forgetting techniques. Furthermore, we consider continual learning methods from computer vision like PODNet (Douillard et al., 2020), LUCIR (Hou et al., 2019), and SelfTraining (Lange et al., 2019). More details on these baselines can be found in Appendix A.
Implementation Details We employ the “BIO" labeling schema for all datasets. Our NER model utilizes the bert-base-cased (Kenton and Toutanova, 2019) model as the encoder and employs a fullyconnected layer as the classifier. For each CNER setting, if each continual training step (t>2) learns an entity type, we train the model for 10 epochs; otherwise, for 20 epochs. We set the batch size, initial learning rate lr1, hyper-parameters α and β to 8, 4e-4, 1e-2, and 0.5, respectively. All experiments are conducted on an NVIDIA A6000 GPU with 48GB of memory, and each experiment is run 5 times to ensure statistical significance.

heading: 4.2 Experimental Results
text: Main Results Tables 2 and 6 (Appendix B) present the performance of our WT&WF and
baselines across 10 CNER settings on the I2B2, OntoNotes5, and CoNLL2003 datasets. Our findings show substantial improvements in both Miand Ma-F1 scores for ExtendNER, CFNER, and CPFD after applying WT&WF across nearly all CNER settings. Moreover, CPFD + WT&WF achieves new SOTA performance across all settings. These outcomes highlight the broad applicability and effectiveness of our WT&WF strategies.
Ablation Study Table 3 presents the results of ablation study on our WT&WF strategies based on the CPFD method. The best performance is observed when both strategies are used simultaneously. While utilizing either strategy alone results in some improvement compared to the original
method, fixing λt to 0.5 in WF does not result in a discernible enhancement over the original method.
Sensitivity of Hyper-parameters Table 4 shows the sensitivity analysis on α. We set 7 optional values ranging from 1.0 to 1e-9. It demonstrates that if we don’t choose values that are too extreme (α = 1e-9 or 1.0), results won’t fluctuate much. We also perform a significance test, which shows the results of α = 1e-1 to α = 1e-3 don’t have statistically significant difference (not satisfy p < 0.05) compared with the results of α = 1e-2. On the contrary, when α = 1.0, the decay is too rapid, resulting in the learning rate for new tasks approaching zero. And when α = 1e-9, the results have a significant decrease (satisfy p < 0.05) compared with the results of α = 1e-2, because α = 1e-9 is almost equivalent to a fixed learning rate. Thus, our WT strategy is stable within a selection range of α = 1e-1 to 1e-3, and there is no need for intentional hyper-parameter adjustment.
Moreover, Table 5 shows the sensitivity analysis on β. We set 5 optional values ranging from 0.0 to 10. We perform a significance test, which shows the results of β = 1.0 and β = 2.0 don’t have statistically significant differences (not satisfy p < 0.05) compared with the results of β = 0.5. On the contrary, results of β = 0.0 (using the new model directly for inference, as with the previous
method) and β = 10 (approaching the scenario of using only the old model for inference) both have a significant decrease (satisfy p < 0.05) compared with the results of β = 0.5. Thus, our WF strategy is also stable within a selection range from β = 0.5 to β = 2, and there is also no need for intentional hyperparameter adjustment.

heading: 5 Conclusion
text: In this paper, we introduce model-agnostic WT and WF strategies for CNER to mitigate catastrophic forgetting. WT employs a learning rate schedule within each training step to adjust the parameters of the new model, while WF dynamically fuses the new and old models to maintain a balance between new and old knowledge. Extensive experiments conducted on 10 CNER settings across 3 datasets illustrate that our strategies further enhance the previous SOTA methods.

heading: Acknowledgments
text: We express our gratitude to the anonymous reviewers for their valuable and insightful comments. This work was supported by JSPS KAKENHI Grant Number JP23K28144.
Limitations
Existing CNER methods may suffer from instability, with the order of entity types significantly impacting performance. In real-world scenarios, the optimal entity type order is unknown beforehand, so an ideal CNER method should perform consistently regardless of this order. Previous experiments have maintained a constant entity type order (e.g., alphabetical order). Future research should explore the stability of CNER methods under varying entity type learning orders. Additionally, given the model-agnostic nature of our approach, future work can investigate combination with more CNER methods and encoders, including larger language models beyond BERT.
Ethical Considerations
In addressing ethical considerations, we wish to elucidate the following points: (1) This work does not involve the use of sensitive data or tasks; (2) We provide thorough descriptions of the dataset statistics and implementation details, and analyses are closely aligned with the experimental results; (3) All experiments use pre-existing datasets sourced from publicly available research.

heading: A Baselines
text: The detailed introductions about the baselines in our experiments are as follows.
PODNet (Douillard et al., 2020): PODNet is employed to address the challenge of catastrophic forgetting in continual learning for image classification, transferred to the CNER scenario. The overall loss of the model encompasses both classification loss and distillation loss. In the realm of classification, PODNet opts for the neighborhood component analysis loss as a substitute for the conventional cross-entropy loss. In the computation of distillation loss, PODNet imposes constraints on the output of each intermediate layer.
LUCIR (Hou et al., 2019): LUCIR establishes a framework for incrementally learning in continual image classification tasks, transferred to the CNER scenario, sharing a conceptual similarity with PODNet in addressing catastrophic forgetting. Its overall loss comprises three components: (1) the cross-entropy loss on samples with new entity types; (2) the distillation loss between features extracted by the old model and the new one; and (3) the margin-ranking loss on reserved samples for old entity types.
Self-Training (Rosenberg et al., 2005; Lange et al., 2019): Self-Training initially employs the pre-existing model to label the non-entity type tokens with their respective old entity types. Subsequently, the novel model undergoes training on fresh data, incorporating annotations for all entity types. The ultimate objective is to minimize the cross-entropy loss across all entity types, ensuring comprehensive and effective model training.
ExtendNER (Monaikul et al., 2021): ExtendNER explores the application of knowledge distillation to CNER and shares similarities with SelfTraining, but it computes cross-entropy loss for entity type tokens and KL divergence loss for nonentity type tokens. During the training process, ExtendNER minimizes the sum of cross-entropy loss and KL divergence loss.
CFNER (Zheng et al., 2022): Based on ExtendNER, CFNER proposes a causal framework for CNER, enabling the extraction of causal effects from the non-entity type. Specifically, it utilizes the old model to recognize non-entity type tokens belonging to previous entity types, extracting causal
effects. It also employs a curriculum learning strategy to mitigate recognition errors.
CPFD (Zhang et al., 2023a): CPFD addresses two challenges in CNER: catastrophic forgetting and the semantic shift problem of the non-entity type. To tackle catastrophic forgetting, CPFD introduces a pooled feature distillation loss that balances stability and plasticity. Simultaneously, CPFD proposes a confidence-based pseudo-labeling strategy to reduce the impact of label noise and address the semantic shift problem.

heading: B Supplementary Results
text: We also experiment with the effectiveness of our WT&WF strategies on the CoNLL2003 dataset and results are shown in Table 6. However, the improvement in results was not as substantial as in the I2B2 and OntoNotes5. Further analysis suggests that CoNLL2003 contains fewer (four) entity types, resulting in less noticeable forgetting of previous knowledge. CoNLL2003 is also less difficult compared to the other datasets, so the previous baseline can almost reach the performance upper-bound and leaves less room for improvements, especially for CPFD. Consequently, the enhancement on the previous SOTA methods is relatively minor.

heading: Future Work
text: Future research should explore the stability of CNER methods under varying entity type learning orders. Additionally, given the model-agnostic nature of our approach, future work can investigate combination with more CNER methods and encoders, including larger language models beyond BERT. Ethical Considerations
In addressing ethical considerations, we wish to elucidate the following points: (1) This work does not involve the use of sensitive data or tasks; (2) We provide thorough descriptions of the dataset statistics and implementation details, and analyses are closely aligned with the experimental results; (3) All experiments use pre-existing datasets sourced from publicly available research