Enhancing Continual Relation Extraction via Classifier Decomposition
Year: 2023

Abstract:
Continual relation extraction (CRE) models aim at handling emerging new relations while avoiding catastrophically forgetting old ones in the streaming data. Though improvements have been shown by previous CRE studies, most of them only adopt a vanilla strategy when models first learn representations of new relations. In this work, we point out that there exist two typical biases after training of this vanilla strategy: classifier bias and representation bias, which causes the previous knowledge that the model learned to be shaded. To alleviate those biases, we propose a simple yet effective classifier decomposition framework that splits the last FFN layer into separated previous and current classifiers, so as to maintain previous knowledge and encourage the model to learn more robust representations at this training stage. Experimental results on two standard benchmarks show that our proposed framework consistently outperforms the state-of-the-art CRE models, which indicates that the importance of the first training stage to CRE models may be underestimated. Our code is available at https://github.com/hemingkx/CDec.

heading: 1 Introduction
text: Continual relation extraction (CRE) (Wang et al., 2019) requires models to learn new relations from a class-incremental data stream while avoiding catastrophic forgetting of old relations. To address the problem of catastrophic forgetting, rehearsal-based CRE methods store a few typical instances for each relation on memory and replay the memory data in the subsequent learning process. Despite the simplicity, rehearsal-based methods have become the state-of-the-art CRE methods (Han et al., 2020; Cui et al., 2021; Hu et al., 2022; Wang et al., 2022b).
Recent rehearsal-based CRE models usually follow a two-stage training paradigm to tackle the data imbalance problem1: (1) in Stage 1, models
1If models are directly trained with the mixed data which
are trained with only new data to rapidly learn to identify new relations; (2) in Stage 2, models are trained on the updating memory data to alleviate catastrophic forgetting.
Previous CRE works mainly focus on stage 2, and propose a variety of replay strategies to better utilize the memory data, such as relation prototypes (Han et al., 2020), memory network (Cui et al., 2021) and contrastive learning (Hu et al., 2022). However, the exploration of the first training stage is still uncharted territory. In this work, we focus on stage 1 and point out that it suffers from two typical biases that harm the model’s performance:
• classifier bias: without the previous training data, the class weights of previous relations would be improperly updated, leading to a skewed output distribution as shown in Figure 1.
• representation bias: the learned representations of current relations would be easily overlapped with those of previous relations in the representation space.
To alleviate the biases mentioned above, we pro-
contains sufficient current data and few previous data, severe data imbalance will lead models to overfit on the previous data, which harms the model’s performance (Wang et al., 2022a).
10053
pose a simple yet effective classifier decomposition framework inspired by Wang et al. (2021). The framework splits the last FFN layer into the separated previous and current classifiers, and introduces two enhanced strategies: empirical initialization and adversarial tuning. Specifically, To alleviate the classifier bias, empirical initialization firstly preserves the well-learned previous classifier nodes and then reuses them after learning current relations. To ease the representation bias, adversarial tuning slows down the update of previous classifier weights. The slowly updating weights can be seen as a series of adversarial signals that induce CRE models to learn distinguishable representations.
To sum up: (1) we find that CRE models suffer from classifier and representation biases when learning the new relations. (2) we propose a simple yet effective classifier decomposition framework with empirical initialization and adversarial tuning to alleviate these two biases. (3) Extensive experiments on FewRel and TACRED verify the effectiveness of our method.

heading: 2 Task Formalization
text: In CRE, models are trained on a sequence of tasks {T1, T2, . . . , Tk}, where the k-th task has its own training set Dk = {(xi, yi)}Ni=1 with Ck new relations. Each task Tk is an independent supervised classification task to identify the instance xi (including the sentence and entity pair) into its corresponding relation label yi. The goal of CRE is to learn new tasks while avoiding catastrophic forgetting of previously seen tasks. In other words, the model is evaluated to identify an instance into
all Ckprev +C k relations, where Ckprev = ∑k−1 i=1 C i is the number of previously seen relations. To alleviate catastrophic forgetting in CRE, previous rehearsal-based work (Han et al., 2020; Cui et al., 2021; Hu et al., 2022; Wang et al., 2022a) adopts an episodic memory module to store a few representative instances for each previous relation. In the subsequent training process, instances in the memory set will be replayed to balance the model’s performance on all seen relations.

heading: 3 Methodology
text: The overall of our proposed classifier decomposition framework is illustrated in Figure 2. Following previous work (Hu et al., 2022; Wang et al., 2022a; Zhao et al., 2022), our model architecture contains two main components: (1) an encoder that generates representations of a given instance and (2) a feed-forward network (FFN) that maps the encoded representation into a probability distribution over all seen relations.

heading: 3.1 Classifier Decomposition
text: To alleviate the illustrated two typical biases (i.e., classifier bias and representation bias) in the first training stage, we propose a classifier decomposition framework that splits the FFN layer into two independent groups of FFN nodes: the previous nodes and the current ones. The framework contains two enhanced strategies to alleviate the biases in stage 1, which include:
Empirical Initialization Before the stage 1 training, we keep a frozen copy of the model’s previous FFN nodes, which represents the knowledge that
the model learned in previous incremental tasks. This frozen copy is then utilized to initialize the parameters of the previous classifier before stage 2 so as to alleviate the classifier bias and retain the previous knowledge.
Adversarial Tuning To alleviate the representation bias, we propose that the output distribution of previous relations is opposite to the optimization objective of the current task. This distribution can be viewed as adversarial signals, which helps the model learn more unique and distinguishable representations. Taking the analogous relation pair {C1, Cn−1} (e.g., the relation pair of {“Mother”, “Father”}) in Figure 2 as an example, given a training instance of Cn−1, due to the high similarity of encoder representations between C1 and Cn−1, the prediction score of C1 will also be high. Thus, the training objective of current tasks will lead to two optimization paths: reduce the scores of C1 or force the model to learn more unique representations of Cn−1. Inspired by this, we propose to slow down the update of previous classifier weights during training so as to steer the learning more toward the second optimization path. Formally,
θprev ← θprev − αprev ∂
∂θprev L (1)
θcur ← θcur − αcur ∂
∂θcur L (2)
where αprev, αcur are the learning rate of the previous and current output embeddings, respectively. We adopt a lower αprev in the training of stage 1.

heading: 3.2 Training and Inference
text: Following previous work (Han et al., 2020; Cui et al., 2021; Hu et al., 2022; Wang et al., 2022a), the training loss function at task Tk of our framework is given by:
L = D∗∑
i=1
− logP (yi|xi) (3)
where P (yi|xi) is the prediction scores calculated by the FFN layer, (xi, yi) is the sample from D∗, D∗ denotes Dk in stage 1 or the memory set in stage 2. Specifically, P (yi|xi) is given by:
P (yi|xi) = softmax(Wh) (4)
where h ∈ Rd is the encoder representation of the instance xi; W ∈ Rd×(C k prev+C k ) stands for the weights of the FFN layer. During inference, we select the relation with the max prediction score as the predicted relation.

heading: 4 Experiments

heading: 4.1 Experimental Setups
text: Datasets Following previous work (Han et al., 2020; Cui et al., 2021; Hu et al., 2022; Wang et al., 2022a), our experiments are conducted upon two standard benchmarks, FewRel (Han et al., 2018) and TACRED (Zhang et al., 2017), please refer to Appendix A.1 for more details.
Implement Details For fair comparisons, we use the same experimental settings as Cui et al. (2021); Zhao et al. (2022), which randomly divide all relations into 10 sets to simulate 10 tasks and report the average accuracy of 5 different sampling task sequences. The number of stored instances in the memory for each relation is 10 for all methods. We adopt the same random seeds to guarantee that the task sequences are exactly the same. We search the learning rate αprev ∈ [0, 1e− 6, 1e− 5, 1e− 4] for the previous classifier in adversarial tuning. More details of our experimental settings and comparison baselines are included in Appendix A.2 and A.3.

heading: 4.2 Main Results
text: The performances of our proposed classifier decomposition framework and baselines on two datasets are shown in Table 1. The results indicate that our framework is significantly superior to other baselines (p < 0.05) and achieves state-of-the-art performance in the vast majority of settings. Besides, this framework is orthogonal to the data augmentation strategy proposed by Wang et al. (2022b), which can further boost the model’s performance.

heading: 4.3 Analysis
text: Effectiveness of Empirical Initialization As shown in Figure 3, after stage 1, the model’s per-
formance on previous relations is consistently improved with our proposed empirical initialization strategy on all tasks, which indicates that the bias in previous relations is effectively alleviated.
Effectiveness of Adversarial Tuning We utilize t-SNE to visualize the representations of a analogous relation pair: P25 (“mother”) and P26 (“spouse”)2. As shown in Figure 4, with adversarial tuning, the representations of instances belonging to P25 and P26 is much more separable compared with those of the vanilla training strategy, which indicates that adversarial tuning indeed alleviates the representation bias and helps the model learn more robust representations.
Ablation Study We further conduct an ablation study of our proposed two enhanced strategies in Table 2. The experimental results show a performance degradation with the ablation of both strategies, demonstrating the effectiveness of our proposed classifier decomposition framework.

heading: 5 Conclusion
text: In this work, we found that the vanilla training strategy adopted by most previous CRE models in
2We include more cases in Appendix D.
the first training stage leads to two typical biases: classifier bias and representation bias, which are important factors causing catastrophic forgetting. To this end, we propose a simple yet effective classifier decomposition framework with two enhanced strategies to help models alleviate those biases at the first training stage. Experimental results on two benchmarks show that our framework consistently outperforms previous state-of-the-art CRE models, which indicates that the value of this training stage to CRE models may be undervalued. Further analysis shows the effectiveness of our proposed classifier decomposition framework.
Limitations
As a preliminary study, our proposed classifier decomposition framework focuses on the first training stage of CRE models with a lack of explorations on stage 2. Besides, more experiments can be conducted by combining our framework with previous leading CRE models, which we leave for future research. In addition, our work only focuses on strategies with the FFN layer. As the BERT encoder is the main component of CRE models, we call for more attentions to the research of improving encoder representations in the first training stage.

heading: Acknowledgements
text: We thank all the anonymous reviewers for their thoughtful and constructive comments. This paper is supported by the National Key Research and Development Program of China 2020AAA0106700 and NSFC project U19A2065.

heading: A Experiments Details
text: A.1 Datasets
Following previous work (Han et al., 2020; Cui et al., 2021; Hu et al., 2022; Wang et al., 2022a), our experiments are conducted upon the following two standard benchmarks with the training-testvalidation split ratio set to 3:1:1.
FewRel (Han et al., 2018) It is a RE benchmark dataset originally proposed for few-shot learning. The dataset contains 100 relations, each with 700 instances. Following the previous work (Wang et al., 2019; Han et al., 2020; Zhao et al., 2022), we use the original training and validation set of FewRel, which contains 80 relations.
TACRED (Zhang et al., 2017) It is a largescale RE dataset containing 42 relations (including no_relation) and 106,264 samples, which is constructed on news networks and online documents. Following Cui et al. (2021), we removed no_relation in our experiments. The number of training samples for each relation is limited to 320 and the number of test samples of relation to 40.
A.2 Experimental Details
Following previous work (Han et al., 2020; Cui et al., 2021), we use bert-base-uncased as our encoder and Adam as our optimizer. We set the learning rate 1e− 3 for non-BERT modules and 1e− 5 for the BERT module, if not specified. The batch size of training is 32. The memory size of each task is 10. The training epoch for stage 1 and stage 2 are set to 10 for FewRel and 8 for TACRED. Our experiments are conducted on a single NVIDIA 3090 GPU.
A.3 Baselines
We compare our proposed framework with the following baselines in our experiments:
• EA-EMR (Wang et al., 2019) proposes a memory replay and embedding alignment mechanism to alleviate the problem of catastrophic forgetting. • EMAR (Han et al., 2020) constructs a memory activation and reconsolidation mechanism to alleviate the catastrophic forgetting. • CML (Wu et al., 2021) introduces curriculum learning and meta-learn to alleviate order sensitivity and catastrophic forgetting in CRE.
• RPCRE (Cui et al., 2021) proposes relation prototypes and a memory network to refine sample embeddings, which effectively retains the learned representations in CRE; • CRL (Zhao et al., 2022) proposes to utilize contrastive learning and knowledge distillation to alleviate catastrophic forgetting. • CRECL (Hu et al., 2022) introduces prototypical contrastive learning to ensure that data distributions of all CRE tasks are more distinguishable to alleviate catastrophic forgetting.

heading: B Learning rate in Adversarial Tuning
text: We search various learning rates for previous classifier in adversarial tuning. The results shown in Table 3 indicate that the rate of 1e − 5 performs best, which is on par with that of the BERT encoder. It is worth noting that completely freezing the previous classifier (i.e., lr = 0) leads to performance degradation since the model easily overfits on the fixed output distribution of the previous classifier.

heading: C Case Study
text: We show a specific case including F1 scores of previous relations after the first training stage of the task T2. As shown in Figure 5, significant improvement in scores of previous relations is shown with the empirical initialization strategy, indicating its effectiveness.

heading: D Representations with Adversarial Tuning
text: We illustrate more cases in Figure 6 to show the effectiveness of adversarial tuning strategy.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
6
7 A2. Did you discuss any potential risks of your work? Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? 4
3 B1. Did you cite the creators of artifacts you used? 4
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? 4
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 4
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? 4
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? 4
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 4
C 3 Did you run computational experiments? 4
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 4
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 4
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Besides, more experiments can be conducted by combining our framework with previous leading CRE models, which we leave for future research. In addition, our work only focuses on strategies with the FFN layer. As the BERT encoder is the main component of CRE models, we call for more attentions to the research of improving encoder representations in the first training stage