An Ensemble-of-Experts Framework for Rehearsal-free Continual Relation Extraction (2024)

Abstract: Continual relation extraction (CRE) aims to continuously learn relations in new tasks without forgetting old relations in previous tasks. Current CRE methods are all rehearsal-based, which need to store samples and thus may encounter privacy and security issues. This paper targets rehearsal-free continual relation extraction for the first time and decomposes it into task identification and within-task prediction sub-problems. Existing rehearsal-free methods focus on training a model (expert) for withintask prediction yet neglect to enhance the models’ capability of task identification. In this paper, we propose an Ensemble-ofExperts (EoE) framework for rehearsal-free continual relation extraction. Specifically, we first discriminatively train each expert by augmenting analogous relations across tasks to enhance the expert’s task identification ability. We then propose a cascade voting mechanism to form an ensemble of experts for effectively aggregating their abilities. Extensive experiments show that our method outperforms current rehearsal-free methods and is even better than rehearsal-based CRE methods. 1

text: Findings of the Association for Computational Linguistics ACL 2024, pages 1410–1423 August 11-16, 2024 ©2024 Association for Computational Linguistics
In this paper, we propose an Ensemble-ofExperts (EoE) framework for rehearsal-free continual relation extraction. Specifically, we first discriminatively train each expert by augmenting analogous relations across tasks to enhance the expert’s task identification ability. We then propose a cascade voting mechanism to form an ensemble of experts for effectively aggregating their abilities. Extensive experiments show that our method outperforms current rehearsal-free methods and is even better than rehearsal-based CRE methods. 1

heading: 1 Introduction
text: Relation extraction (RE) aims to identify the relation between two entities mentioned in the sentence, which is essential for many tasks like knowledge graph construction (Peng et al., 2020). Traditional RE methods based on pre-defined relation sets have achieved remarkable performance but cannot handle emerging relations in the real world (Han et al., 2020). Hence, continual relation extraction (CRE) is proposed (Wang et al., 2019) to deal with such a problem, with the goal of continuously learning new relations while not forgetting previously learned ones.
∗Corresponding author. 1Our code and data are available at https://github.
com/NLPWM-WHU/EoE-CRE.
Existing CRE methods need to store a few typical samples for each learned relation and then utilize memory replay to avoid the catastrophic forgetting problem in continual learning. However, due to privacy and security issues, storing training data is not allowed in sensitive areas such as finance and biology (Wang et al., 2023d). In light of this, we consider the rehearsal-free continual relation extraction (RFCRE) problem for the first time, for which we expect to incrementally learn new relations without storing any training data.
Following the pioneering work in continual learning (Kim et al., 2022), we decompose RFCRE into two sub-problems: task identification and within-task prediction. Benefiting from parameterefficient tuning (Li and Liang, 2021; Hu et al., 2021; Ding et al., 2022), the within-task prediction problem has been well addressed by saving the task-specific parameters of each task (named as an expert). Then, the core challenge lies in the task identification problem, i.e., how to identify the task-id for each test sample.
Along this line, Wang et al. (2023d) employs a pre-trained language model (PLM) (Devlin et al., 2019) based method for task identification in continual text classification. However, when applying this method to the RE task, we find its performance is not very satisfying. The main reason is that only one PLM is involved in task identification, as shown in Fig. 1(a). The knowledge in PLM
1410
might be enough for the text classification task but is insufficient for the RE task, whose performance relies heavily on task-related knowledge.
To address this problem, we propose an Ensemble-of-Experts (EoE) framework for rehearsal-free continual relation extraction. Our goal is to aggregate the experts of different tasks from different stages during the continual learning who already possess rich task-related knowledge for both within-task prediction and task identification, as shown in Fig. 1 (b). Despite the appealing target, we now face two new research questions.
[RQ1]: How to increase each expert’s task identification ability besides the original within-task prediction ability?
[RQ2]: How to effectively aggregate multiple experts’ abilities by assigning the right experts for the given task?
For [RQ1], we find the challenge in task identification mainly comes from the analogous relations across tasks, e.g., father and sibling, and it is pretty hard for an expert to discern such relations. In view of this, we introduce a discriminative training (Zhan et al., 2021) approach to increase each expert’s task identification ability. Specifically, we develop a novel relation augmentation method to generate two types of analogous relations. One is to reverse the positions of two entities in an old relation (Han et al., 2021; Wang et al., 2022a). The other is to remove the context between two entities in an old relation. Since these augmented relations have not been seen by the expert of the current task, they will force the expert to distinguish old relations from the augmented ones during the discriminative training process.
For [RQ2], we find that directly aggregating the votes from all experts will bring about a transboundary risk. This is because the scope of task identification varies among different experts due to the temporal nature of continuous learning. For example, the tth expert has not seen the (t − 1)th task, so the (t − 1)th task is out of the scope of the tth expert. To handle this, we propose a cascade voting mechanism consisting of a two-phase voting procedure. Specifically, in the first phase, we use the first expert and the PLM to make an initial decision. Then, in the second phase, we dynamically aggregate votes from qualified experts for the final decision.
Our main contributions are as follows:
• To the best of our knowledge, we are the first to propose the rehearsal-free continual relation extraction (RFCRE) problem.
• We present an ensemble-of-experts framework for the RFCRE task, which aggregates experts’ knowledge from different stages in continual learning for both task identification and withintask prediction.
• Extensive experimental results on two datasets show that our method not only outperforms the state-of-the-art rehearsal-free methods but also achieves better performance than rehearsal-based CRE methods.

heading: 2 Related Work
text: Continual Learning (CL) The goal of continual learning is to extend knowledge from a continuous data stream without catastrophic forgetting (Wang et al., 2023a). Existing CL methods can be roughly categorized into three groups: (a) Regularization-based methods mitigate catastrophic forgetting by introducing parameter regularization terms (Kirkpatrick et al., 2017) or knowledge distillation loss (Li and Hoiem, 2017), (b) Rehearsal-based methods store a small set of old training samples to assist the model in replaying learned knowledge. Current CRE methods (Wang et al., 2019; Han et al., 2020; Cui et al., 2021; Zhao et al., 2022; Hu et al., 2022; Wang et al., 2022a; Xia et al., 2023; Zhao et al., 2023; Xiong et al., 2023; Song et al., 2023; Nguyen et al., 2023) mainly focus on retaining or recovering the old relation knowledge during the memory replay stage, such as prototype learning (Cui et al., 2021), curriculum learning (Wu et al., 2021), knowledge distillation (Zhao et al., 2022), contrastive learning (Song et al., 2023). (c) Network-based methods dynamically expand or split the network parameters to learn new tasks. Thanks to the development of parameter-efficient tuning (PET) methods (Li and Liang, 2021; Liu et al., 2022; Hu et al., 2021), several studies (Wang et al., 2023d,c, 2022c) try to assign task-specific parameters to each task and then query the task-id for the given test sample.
In contrast to existing rehearsal-based CRE methods, we target rehearsal-free CRE, which is the first attempt in this field. Task decomposition in CL Based on the parameter-efficient tuning technique, several CL
methods decompose the CL problem into task identification and within-task prediction (Kim et al., 2022). For example, L2P (Wang et al., 2022d) uses the PLM to retrieve a few prompts from a fixed prompt pool. EPI (Wang et al., 2023d) uses Mahalanobis distance for task identification, and ESN (Wang et al., 2023c) uses energy scores to conduct identification. Several studies directly summate (Wang et al., 2023b) and concatenate (Razdaibiedina et al., 2023) parameters for task identification.
Different from the above methods, we effectively aggregate experts from different stages for task identification via the discriminative training and cascade voting mechanism tailored for this purpose.

heading: 3 Problem Formulation
text: Relation extraction aims to determine whether the relation y holds for the head and tail entity in the given instance x. Continual relation extraction (CRE) aims to train the model from a series of class-incremental relation extraction tasks {T1, T2, · · · , TN}, where N is the number of tasks, each task Tk contains its training set Dk = {(xi, yi)}Nki=1 and relation set Rk. Specifically, xi is an instance of relation yi ∈ Rk, Nk is the number of Dk and Rk is the label space of Tk, where Ri ∩ Rj = ∅ for i ̸= j. The CRE methods aim to learn new relations from Dk and not forget previous relations. In other words, the trained model needs to classify all observed relations R̂k = ∪ki=1Ri correctly after learning task Tk. Unlike existing CRE methods that store a small set of representative samples to avoid catastrophic for-
getting via memory replay, we focus on rehearsalfree setting without saving any original training samples to prevent potential privacy safety issues.

heading: 4 Methodology
text: As shown in Fig. 2, when the new task Tk comes, the overall framework is divided into three steps: (1) Discriminative Training: This step strengthens experts’ task identification ability when encountering inter-task analogous relations. (2) Parameter Estimation for Dataset Dk: This step stores some statistics about the current training dataset via experts {θi}ki=0 for later task identification. (3) Inference: This step aims to aggregate the identification ability of different experts and then makes within-task predictions.

heading: 4.1 Discriminative Training
text: The core reason for the stability-plasticity dilemma in continual learning is the interference between old and new tasks (Wang et al., 2023a). Intuitively, the simplest way to eliminate it is to train separately with independent models, which may result in huge resource consumption. Benefiting from the PET methods, e.g., Prefix Tuning (Li and Liang, 2021; Liu et al., 2022), LoRA (Hu et al., 2021), we can insert a few additional parameters θk into the model and then only tune the inserted parameters to achieve the performance close to the full fine-tuning. In this work, we use LoRA (A brief introduction can be found in Appendix B) to obtain θk for the current task.
However, previous work (Wang et al., 2023d) only optimizes the θk based on the dataset Dk. Although it is sufficient for within-task prediction,
it does not perform satisfactorily on task identification when encountering inter-task analogous relations. The reason is that the lack of supervision for unknown analogous relations causes the expert to learn shortcuts for classifying relations Rk based on the dataset Dk. Hence, we introduce analogous relation augmentation to generate new relations based on the known relations to avoid expert learning from shortcuts. Then, we formalize |R̂k|-way (|R̂k| > |Rk|) classification task via merging generated new relations and known relations to obtain the expert θk. ➀ Analogous Relation Augmentation The key to relation data augmentation lies in how to construct challenging negative samples. Peng et al. (2020) have pointed out that relation extraction models tend to rely on shallow features leaked from entity mentions for relation classification. Thus, we apply two strategies to encourage the expert to have an in-depth understanding of entity and context information, including the reverse relation augmentation (Wang et al., 2022a) and undetermined relation augmentation.
Following Wang et al. (2022a), given an instance x from dataset Dk, we first determine whether its relation is symmetric 2. If the relation is not symmetric, we can construct a new relation by swapping the markers of the head and tail entities. For example, for the sentence “[E11] Beijing [E12] held the [E21] Winter Olympics [E22]”, the relation changed from host to place of hosting when we switched the markers of the head and tail entities. After that, we denote the dataset augmented by this strategy as Drk.
Wang et al. (2022b) constructs counterfactual examples by removing the context to eliminate entity bias. This implies a hypothesis: If an instance only retains entity information and removes the context information, there should be no relation between the entities. For example, it is hard for us to determine the relation between Beijing and Winter Olympics without context information; the relation may be host, participate, or apply. In view of this, we build a new relation by removing the context information of the instances named undetermined_relation. Note that this strategy is applied to both Dk and Drk, and we denote the dataset with undetermined_relation as Duk .
2Symmetric relations imply that swapping the order of the head and tail entities does not change the relation, such as per:siblings.
➁ Training After constructing the datasets Drk and Duk , we insert them into the original dataset. For convenience, we denote relation-augmented dataset as D̂k = Dk ∪Drk ∪Duk and merged relation set as R̂k, where the maximum size of R̂k is 2∗ |Rk|+1. Given an input instance x = {w1, w2, · · · , wn} ∈ D̂k with the corresponding entity pair, we insert two pairs of special tokens [E11]/[E12] and [E21]/[E22] at the beginning and end of the head entity and the tail entity, respectively. Except for the first task, we inject additional parameters θk into BERT (Devlin et al., 2019) as backbone to encode the input to obtain the contextual representations fθk(x) = {w1,w2, · · · ,wn}. Then, we directly concatenate the representation of [E11] and [E21] as the relational representation of xi, which is defined as
hi = w[E11] ⊕w[E21]. (1)
Then, we initialize the relation embeddings ϕk ∈ R|R̂k|×d as |R̂k|-way classifier weight to classify both old relations and new relations. Therefore, we can optimize θk and ϕk using cross-entropy loss, with the training objective as follows:
Lce(θk, ϕk) = − 1|D̂k|
|D̂k|∑
i=1
|R̂k|∑
j=1
I(yi = rj)logPθk (rj |xi),
(2)
where Pθk(xi) = softmax(hi ·ϕk) and yi denotes the gold relation 3. Note that we use full fine-tuning for the first task to provide a good initialization for subsequent tasks.

heading: 4.2 Parameter Estimation for Dataset Dk
text: After training the k-th task, we need to save taskspecific information for subsequent task identification due to the unavailability of the previous task data. Inspired by Wang et al. (2023d), we assume that the class-conditional distribution follows the multivariate Gaussian distribution based on Gaussian discriminant analysis. Then, with the task dataset Dk and θj , we can utilize the maximum likelihood estimator to estimate |Rk| class-conditional Gaussian distributions N (µjk,c,Σ j k) with a shared covariance Σjk:
µjk,c = 1 |Dk,c| ∑
yi=c
hi, c ∈ Rk, (3)
Σjk = 1 |Dk| ∑
c
∑
yi=c
(hi − µjk,c)(hi − µjk,c)⊤, (4)
3After training the k-th task, we discard the relation embeddings of augmented relations.
where hi = fθj (xi) and Dk,c denotes samples with a relation c in Dk. In practice, we further share the covariance of seen tasks by θj to avoid numeric deviation:
Σj = 1 k − j + 1 k∑
i=j
Σji (5)

heading: 4.3 Inference
text: Currently, we obtain experts {θ}ki=1 for each task, but we cannot access the task-id of each test instance at the testing stage. Thus, when given a test instance, we must decide which expert to use for within-task prediction. Wang et al. (2023d) rely on the class-conditional Gaussian distribution estimated by the original PLM and uses Mahalanobis distance (Lee et al., 2018) for task identification. However, the knowledge in PLM is insufficient for RE, and it also ignores the potential identification capability of experts with task-related knowledge from different stages for task identification.
Limited by the sequential temporal nature of continuous learning, directly aggregating votes from different experts suffers from transboundary risk. For example, the 2nd expert has not seen the first task, so the first task is out of the scope of θ2. Hence, we proposed a cascade voting mechanism to overcome it. For convenience, we sequentially explain the voting protocol of θk, cascade voting mechanism, and within-task prediction. ➀ Voting Protocol of θk Firstly, we map the test instance x into the representation h by the expert model h = fθk(x). Then, with the above induced class-conditional Gaussian distributions {N (µki,c,Σk)} |Ri| c=1 of task Ti by expert θk, we can define the confidence score through computing the Mahalanobis distance between h and {N (µki,c,Σk)} |Ri| c=1 :
CTik (x) = minc ( h− µki,c )⊤ Σk −1 ( h− µki,c ) , i ≥ k,
(6)
where CTik (x) denotes the confidence score of task Ti by θk. Then, the voting result Vxk of expert k can be defined as:
Vxk = argmin i CTik (x), i ≥ k. (7) . ➁ Cascade Voting Mechanism As shown in Fig. 2, if we directly allow experts at each stage to participate in voting, the available voting range for each expert is inconsistent. To this end, we propose a cascade voting mechanism to solve the transboundary risk. Specifically, we first introduce the original
Algorithm 1 Cascade Voting Mechanism Require: Voting results Vx0 and Vx1 of expert θ0 and θ1, Maxi-
mum allowed experts m, Confidence scores {CTij (x)|2 ≤ j ≤ k − 1, j ≤ i ≤ k}
Output: Voting result Vx 1: if Vx0 = Vx1 then 2: return Vx0 ; 3: end if 4: start = 0; 5: end = min(min(Vx0 ,Vx1 ),m); 6: result = dict(); 7: for e ∈ [start, end] do 8: Vxe = argmini CTie (x), i ≥ end; 9: result[Vxe ] += 1; 10: end for 11: return argmaxc result[c];
PLM θ0 as an additional expert4 to form a fair voting stage with θ1 in the first phase. Then, based on the voting results of the first phase, we dynamically aggregate votes from qualified experts from the final decision in the second phase. The detailed procedure of the cascade voting mechanism for the task Tk is shown in Algorithm 1, which mainly contains two phases:
First Phase (line 1 ∼ 5): If the voting results of θ0 and θ1 are consistent, we directly return the results. If inconsistent, the minimum value of the voting result min(Vx0 ,Vx1 ) is taken as the maximum range of allowed voting experts. Noted that to avoid the surge in the number of voting experts as tasks increase, we introduce an additional threshold m to limit the maximum number of experts selected.
Second Phase (line 6 ∼ 11): For each expert θj , j ∈ [start, end] is allowed to vote, we frame its voting range at [end, k] to ensure the fairness, and calculate the task with the highest votes as the voting result. ➂ Within-task Prediction After obtaining the voting result Vx, we can use the expert θVx with the corresponding relation embeddings ϕVx to predict the relation of the given instance x.

heading: 5 Experiments

heading: 5.1 Experimental Setup
text: Datasets We employ two widely used relation extraction datasets, i.e., TACRED (Han et al., 2018) and FewRel (Han et al., 2018), to evaluate the performance of our proposed method. To ensure a fair comparison, we follow the order and the division of tasks in Cui et al. (2021) by dividing
4To adapt the PLM to the relation extraction, we have tried several different representation extraction methods and selected Entity Avg. finally. Please see Appendix A for details.
wave. † and ∗ denote the statistically
significant improvements with p < 0.01 and p < 0.05 over the results by the best rehearsal-free baseline EPI.
each dataset into 10 sub-datasets according to relations, each representing a task. Please refer to the Appendix C.1 for the details of the datasets.
Metrics We use two metrics for evaluating the performance. Identification Accuracy (IA) denotes the accuracy of the task identification; Classification Accuracy (CA) denotes the relation classification accuracy on observed relations, which serves as the main metric.
Baselines We first compare our proposed EoE method with rehearsal-free continual learning methods. Since there is no existing RFCRE method, we migrate four rehearsal-free CL methods from other fields to the relation extraction task, and divide them into two groups: (1) Regularizationbased methods: EWC (Kirkpatrick et al., 2017) and LwF (Li and Hoiem, 2017). (2) Networkbased methods: L2P (Wang et al., 2022d), and EPI (Wang et al., 2023d). Both network-based
methods are implemented with prefix-tuning. For a more fair comparison, we extend EPI with a LoRA implementation.
We then compare our EoE method with five latest rehearsal-based continual relation extraction (CRE), including CEAR (Zhao et al., 2023), CDec (Xia et al., 2023), RationaleCL (Xiong et al., 2023), CFDR (Nguyen et al., 2023), and InfoCL (Song et al., 2023). 5 Implementation Details We implement the framework based on Pytorch (Paszke et al., 2019) and Huggingface (Wolf et al., 2020) and use bert-base-uncased (Devlin et al., 2019) as the backbone. For the first task, we use full fine-tuning, with the learning rates of the backbone and classifier set to 1e-5 and 1e-3, respectively. the number of epochs is set to 10, and the Adamw optimizer is employed. For the subsequent
5Please refer to Appendix C.2for the details of baselines.
tasks, we freeze the backbone’s parameters and use LoRA (Hu et al., 2021) to assign a small number of task-specific parameters to each task, with its learning rate set to 5e-4, rank set to 8, the alpha parameter for Lora scaling set to 16, and dropout set to 0.1. The number of epochs is set to 5, and the learning rate of the classifier is set to 3e-2. All experiments were conducted on an NVIDIA RTX 3090 GPU, and all results were averaged by taking 5 different task sequences.
For a more comprehensive comparison experiment, we migrated four rehearsal-free methods to relation extraction. Since the task boundaries are known during training, we mask non-current task relations at the training for FT, EWC, LwF, and L2P. For EPI, which has the closest performance, we additionally implement LoRA (Hu et al., 2021) to guarantee a fairer comparison.

heading: 5.2 Main Results
text: Table 1 presents the comparison results of our proposed EoE method and baselines. Based on the results, we have the following findings.
(1) Our EoE method achieves state-of-the-art performance in almost all cases. It surpasses the latest rehearsal-based methods focusing on relation extraction and the latest rehearsal-free methods adapted from other tasks. Moreover, on the last T10 on FewRel, even the performance of the best rehearsal-free method EPI is significantly worse than all rehearsal-based methods. In contrast, our EoE still wins in this case without the help of stored samples. In summary, the improvements of our method over both the rehearsal-free and rehearsalbased methods clearly demonstrate the effectiveness of our ensemble of expert approaches.
(2) As the task proceeds, there is a performance decrease for all methods. For rehearsal-based methods, the reason lies in the interference between old and new tasks. For network-based methods like EPI and ours, the reason is mainly due to
the increase in the number of tasks, which consequently brings about the increase in the difficulty of task identification. To have a close look, we compare the task identification accuracy between our method and EPI in Fig. 3. It can be seen that our method is significantly better than EPI which only utilizes BERT for task identification. We can conclude that the significant improvements in classification accuracy of our method over EPI can be primarily attributed to the experts’ ability of task identification.

heading: 6 Analysis

heading: 6.1 Ablation Study
text: To validate the effectiveness of each component in our proposed framework, we conduct an ablation study. Specifically, (1) w/o DT denotes we remove the discriminative training, (2) w/o UR denotes
we remove undetermined relation augmentation in DT, (3) w/o RR denotes we remove reverse relation augmentation in DT, (4) w/o CV denotes we remove the cascade voting mechanism and only use the first model for task identification, (5) w/o All denotes we remove DT and CV simultaneously, which means we only optimize the expert with standard cross-entropy loss on each dataset and directly use the first expert for task identification.
We present the ablation results in terms of classification accuracy (CA) and task identification accuracy (IA) in Table 2. From these results, we can observe that (1) Both DT and CV are effective, and the impact of the cascade voting mechanism is more significant. (2) Adding either reverse relations (RR) or undetermined relations (UR) can significantly enhance the experts’ task identification capability. (3) Adding both RR and UR can further improve the task identification capability of the experts on TACRED, but the results on FewRel do not meet our expectations. We believe the main reason lies in the characteristics of the datasets. FewRel is collected from Wikipedia, where many samples involve commonsense relations between entities and do not rely too much on the textual context. (3) The removal of DT and CV results in the biggest drop in performance. This proves that the two components are inextricably linked and play an important role.

heading: 6.2 Analysis on Discriminative Training
text: To further exploit why the discriminative training can improve the task identification capability of the expert when encountering analogous relations, we employ a relation-task similarity metric to divide all observed relations into three nearly equal-size groups, where the similarity between the i-th rela-
tion in a task t and the j task is defined as:
sim(i, j) = CTj0 (µ0t,i), j ̸= t, (8)
where CTj0 (µ0t,i) is the Mahalanobis distance between the i-th relation and the j-th task computed by BERT. Then, we sort all relations based on minj sim(i, j), i /∈ Rj and divide them into three groups, and present the task identification results in Tab. 3.
From these results, we can conclude that the analogous relations between tasks are the key reason for the decline in identification accuracy. Note that the relation-task similarity decreases from G1 to G3, while the increase of EoE with the discriminative training becomes more significant, e.g., a 1.4, 0.5, -0.1 gain on G1, G2, G3, respectively. The effect of DT is more obvious on the FewRel dataset than that on TACRED. The reason might be that the amount of each relation is the same for all tasks on FewRel, and thus the trend is much clearer.

heading: 6.3 Analysis on Cascade Voting
text: Overall, the generalization ability of an ensemble of multiple experts is greater than that of a single expert. Still, two points need to be verified. (1) The identification accuracy of experts should not be too bad and should be similar. (2) The identification results among different experts must have certain differences. Hence, to further explore why the cascade voting mechanism can bring performance gains, we report the identification accuracy only with θ0 or θ1 on each task after all tasks are finished.
From the results in Fig. 4, we can observe that: (1) There is no significant performance difference in identification accuracy between θ1 and θ2. We can also find that the expert θk significantly outperforms {θi}k−1i=1 on the task Tk. For example, θ2 outperforms θ1 on the T2 task by a large margin.
(2) There is a large variability in the prediction results of θ1 and θ2, which is the source of the improvement in identification brought about by the subsequent cascade voting.

heading: 6.4 Analysis on Computational Cost
text: The inference in our RFCRE task is divided into two steps: task identification and within-task prediction. The main difference between the best rehearsal-free baseline (EPI) and our EoE is that we aggregate experts from different stages for task identification. However, our EoE needs to use all previous experts to estimate the distribution parameters of the dataset after training at each stage, which may bring about extra computational cost. To reduce the overhead, we introduce a threshold m to limit the number of models that can participate in estimation. Fig. 5 shows the changes in classification accuracy under different thresholds. m = 1 denotes we only use θ1 for task identification 6. m = 2 denotes we incorporate {θ0, θ1, θ2}.
From Fig. 5, we can find that: (1) Using only one expert for identification has a significant drop compared to multi-expert identification since experts trained on different tasks have their own biases. (2) Simply adding one additional expert like the expert θ2 to select the results of the first phase in cascade voting can effectively improve the performance. (3) As more experts participate in the voting, the performance increases, but it is limited in magnitude. Thus, the number of experts can be selected according to the real world scenario.
The cascade voting mechanism consists of two phases, and only tasks with conflicting results in the first phase proceed to the second phase. To analyze the efficiency of the cascade voting mechanism, we calculate the percentage of each task that requires a two-phase voting procedure, as shown in Tab. 4. We can find that: (1) The proportion of two-phase voting rises as the number of tasks
6The reason θ0 is not needed here is that a cascade voting mechanism is not possible with only two experts.
increases. This is because the increase in the number of tasks leading to an increase in the difficulty of task identification. (2) Even up to the 10-th task, the proportion of two-phase voting is about 12-14%, so nearly 85% of test samples only need the first phase to complete the task identification.

heading: 7 Conclusion
text: In this work, we make the first attempt on the rehearsal-free continual relation extraction problem. We propose an ensemble-of-experts framework consisting of discriminative training and cascade voting. Specifically, we first introduce discriminative training to enhance the identification ability of experts when facing inter-task analogous relations. We then propose a cascade voting mechanism to aggregate experts’ abilities from different stages. Extensive experimental results show that our method significantly outperforms existing rehearsal-free and rehearsal-based continual relation extraction methods.
Limitations
The main limitation of our proposed framework lies in the extra time and space cost. Though we introduce a threshold to limit the number of experts participating in task identification, additional overhead is unavoidable as long as multiple experts are involved. Besides, except for the first task, we do not consider inter-task knowledge transfer of the subsequent tasks’ training while only assigning independent task-specific parameters for each task via PET methods.
Ethics Statement
Our work complies with the ACL Ethics Policy. We have not identified any significant ethical considerations associated with our work.

heading: Acknowledgments
text: This work was supported by a grant from the National Natural Science Foundation of China (NSFC) project (No. 62276193).

heading: B Brief introduction of LoRA
text: Pre-trained language models have a low intrinsic dimension during the adaption of the downstream task (Aghajanyan et al., 2021). The essence of LoRA lies in introducing additional parameters A ∈ Rd×r and B ∈ Rr×k to supplement the pretrained weight matrix W0 ∈ Rd×k, where rank r ≪ min(d, k). At the training stage, LoRA freeze the update of W0 and utilize a low-rank decomposition W0 + ∆W = W0 + AB to represent its update. This means that the forward pass h = Wx, where x is the input vector, is modified as follows:
h = Wx+∆Wx = Wx+ABx (9)

heading: C Details of Experimental Setups
text: C.1 Datasets
FewRel (Han et al., 2018): is a large-scale and balanced relation extraction dataset that contains 80 relations and 700 instances for each relation. Following Cui et al. (2021), we partitioned the dataset into 10 sub-datasets, each containing 8 relations. For each relation, we sampled 420 samples as the training set and 140 as the test set. TACRED (Han et al., 2018): is a widely used RE dataset containing 42 relations (including no_relation). Following Cui et al. (2021), we remove no_relation in our experiments and partition the dataset into 10 sub-datasets, each containing 4 relations. The number of training samples of each relation is limited to 320, while the number of test samples of each relation is limited to 40.
C.2 Baselines In this work, we compare five rehearsal-free methods as follows:
• FT: directly fine-tune the model without any strategy to prevent catastrophic forgetting, which can be viewed as the lower bound of continual learning.
• EWC (Kirkpatrick et al., 2017): maintains an importance matrix with the same scale as the model first and then uses L2 loss to constrain the update of important parameters.
• LwF (Li and Hoiem, 2017): utilizes knowledge distillation to force the predicted probability of the old relations between the old model and the current model to be the same.
• L2P (Wang et al., 2022d): introduces a promptbased continual learning framework that freezes the pre-trained encoder and then adapts to different tasks via a shared prompt pool.
• EPI (Wang et al., 2023d): leverages the pretrained language model to estimate the input distribution for each task with a Gaussian and then uses the Mahalanobis distance for task identification.
Besides we also compare five rehearsal-based continual learning methods focus on relation extraction as follows:
• CEAR (Zhao et al., 2023): proposes memoryinsensitive relation prototypes and memory augmentation during rehearsal replay to alleviate catastrophic forgetting.
• CDec (Xia et al., 2023): proposes a classifier decomposition framework with empirical initialization and adversarial training to alleviate classifier bias and representation bias.
• RationaleCL (Xiong et al., 2023): proposes a rationale-enhanced framework to improve the model’s robustness in the face of future analogous relations via multi-task rationale tuning and contrastive rationale replay strategy.
• CFDR (Nguyen et al., 2023): proposes a classwise feature decorrelation regularization to boost eigenvalues.
• InfoCL (Song et al., 2023): exploits fast-slow contrastive learning during new task training and current-past contrastive learning during rehearsal replay to learn more comprehensive representations.

heading: Future Work
text: , 2023): proposes a rationale-enhanced framework to improve the model’s robustness in the face of future analogous relations via multi-task rationale tuning and contrastive rationale replay strategy. • CFDR (Nguyen et al. , 2023): proposes a classwise feature decorrelation regularization to boost eigenvalues. • InfoCL (Song et al. , 2023): exploits fast-slow contrastive learning during new task training and current-past contrastive learning during rehearsal replay to learn more comprehensive representations