Continual Few-shot Relation Extraction via Adaptive Gradient Correction and Knowledge Decomposition (2024)

Abstract: Continual few-shot relation extraction (CFRE) aims to continually learn new relations with limited samples. However, current methods neglect the instability of embeddings in the process of different task training, which leads to serious catastrophic forgetting. In this paper, we propose the concept of the following degree from the perspective of instability to analyze catastrophic forgetting and design a novel method based on adaptive gradient correction and knowledge decomposition to alleviate catastrophic forgetting. Specifically, the adaptive gradient correction algorithm is designed to limit the instability of embeddings, which adaptively constrains the current gradient to be orthogonal to the embedding space learned from previous tasks. To reduce the instability between samples and prototypes, the knowledge decomposition module decomposes knowledge into general and task-related knowledge from the perspective of model architecture, which is asynchronously optimized during training. Experimental results on two standard benchmarks show that our method outperforms the state-of-the-art CFRE model and effectively improves the following degree of embeddings.

heading: 1 Introduction
text: The purpose of CFRE is to continuously train a model on limited new data. Compared to traditional continual relation extraction(CRE) models, it can learn new relations without accessing a large number of previous task data, and avoid catastrophic forgetting (French, 1999; McCloskey and Cohen, 1989) of the old relations.
Due to limited training data, the features learned by the model at each time step of continual learning are relatively unstable and are easily modified by the model when learning other class samples in subsequent time steps. As a result, traditional CRE methods cannot be directly applied to CFRE (Qin and Joty, 2022). To fully utilize data resources,
scholars have explored many methods (Wang et al., 2022b; Zhong et al., 2021; Zhang et al., 2022). And the methods based on memory replay (Chen et al., 2023; Wang et al., 2023; Qin and Joty, 2022) made great achievements in CFRE. However, these models mainly focus on the strategies of in-memory samples in the process of replaying or learning and perform direct fine-tuning of the model parameters through the gradient of the loss function. Furthermore, in-memory samples are used to generate gradients that benefit the performance on previous tasks, but the direction optimized by these gradients may contradict the optimization direction of the current task gradient, which leads to the instability of samples with respect to prototypes. In metric learning (Kaya and Bilge, 2019), the prototype of one class is the center of this class.
In essence, during continuous learning with minimal data usage, when samples are equivalently shifted with the embedding of prototypes, the la-
11805
bels of samples will not be easily changed and catastrophic forgetting of the model will be alleviated. Thus, we formally provide the definition of following degree of class y at time step t: the degree of deviation between class y samples and class y prototype before and after training at t. Obviously, the smaller the offset between the previous sample embedding and its prototype, the higher the following degree of this class, and the less unstable the samples are. In this way, the model can have less catastrophic forgetting of previous samples.
Intuitively, there are two main reasons for the low following degree of the model on a task: 1) the model is not sufficiently optimized for some samples, which makes the prototypes deviate from the optimization direction of the samples; 2) after optimization, all embeddings change too much compared with the previous will also increase the risk of low following degree. Relative offset distance (ROD) and absolute offset distance (AOD) are proposed to express the above two factors, respectively. As shown in Fig. 1, |x′4−x4| and |p′−p| represent AOD of x4 and p. (|p′ − x′2| − |p− x2|) represents ROD between x2 and p. The lower the AOD and ROD, the higher the following degree.
In this paper, an adaptive gradient correction algorithm is proposed to directly constrain and correct the vector space of the gradient in transformerbased language models. Since the modified gradients are orthogonal to the embeddings of the previous sample, this special optimization method for the transformer can effectively reduce AOD. Specifically, for each previous task, we calculate a gradient direction, which has the greatest impact on in-memory samples, as the correction criteria. When updating the gradient of the current task, the correction matrix based on this criteria is used to make a linear transformation of the gradient, which can constrain the subspace of parameters orthogonal to the previous tasks.
In addition, knowledge contained in model parameters is decomposed into general and taskrelated knowledge. Based on this decomposition, general knowledge is used to identify a generic representation of relations with the corresponding entities. We employ task-related knowledge to identify categorical decision boundaries between specific tasks based on general knowledge. In practice, we apply a pre-trained language model (PLM) to encode the general knowledge and use an adaptive gradient correction algorithm to avoid mutual coverage of knowledge. Since there is no gradient
transmission between task prototypes, we integrate task-related knowledge into task prototype embedding. These prototypes are updated discretely and continuously in three different training stages to reduce ROD. In the continuous optimization stage, we add an additional loss to increase the distance between prototypes, which can prevent confusion between the new and the old task prototypes.
To sum up, the contributions of this paper mainly include the following three aspects:
• We attribute catastrophic forgetting in CFRE to the low following degree between samples and prototypes, and analyze how to improve the following degree from the perspectives of AOD and ROD.
• According to the correction matrix calculated by in-memory samples, an adaptive gradient correction algorithm is proposed that makes the model directly adjust the gradient to reduce AOD.
• We design a knowledge decomposed method and corresponding update strategies to avoid the confusion of knowledge between different tasks, which can limit ROD during training.
Experimental results on two public datasets show that our method can effectively alleviate the catastrophic forgetting in CFRE.

heading: 2 Related Work
text: RE aims to extract the implied relation from sentences. For example, given the sentence "Steve Jobs is the co-founder of Apple", the model needs to determine the relation "CEO_of" between the entity "Steve Jobs" and "Apple". It is a basic step for many downstream tasks such as language understanding, question answering, and knowledge graph construction (Nasar et al., 2021).
Most traditional RE models are built based on a fixed dataset (Eberts and Ulges, 2020; Liu et al., 2022b; Shang et al., 2022; Li et al., 2018, 2021). However, RE is often an open vocabulary problem (Liu et al., 2022a), and it is difficult to model all relations for any limited set. Therefore, the continual learning ability (Chen and Liu, 2018) of the RE model has gradually attracted attention (Zhao et al., 2023; Xia et al., 2023).
But there will be serious catastrophic forgetting for the model in continual learning, that is, the model will forget the old knowledge when
learning new tasks. To solve this problem, the existing methods for continual learning mainly include: regularization-based approach (Zhai et al., 2019), memory-based approach (Cha et al., 2021), optimization-based approach (Schwarz et al., 2018), representation-based approach (Yan et al., 2022), and architecture-based approach (Wang et al., 2022a). The existing methods (Wang et al., 2022c; Zhou et al., 2022) are often based on a large number of labeled data for training, which is time-consuming and expensive. When the number of training is small, overfitting of memory data and knowledge coverage (Song et al., 2023) is also one of the causes of catastrophic forgetting. The existing methods mainly solve this problem from the following three levels: data level (Wang et al., 2022b), feature level (Zhong et al., 2021), and task level (Zhang et al., 2022).
Directly adjusting the gradient is also one of the methods to overcome catastrophic forgetting with limited data (Chen et al., 2023; Zhou et al., 2022). He and Jaeger (2018) proposed Concept Aided BackProp for disaster forgetting, in which the gradient is shielded by a conceptor to prevent the degradation of previous tasks. Zeng et al. (2019) introduced orthogonal projection and context-dependent processing module for the current gradient. Guo et al. (2022) put forward the paranoid factor related to the previous task to estimate the input space, but the accumulation strategy for embedding of different samples may lead to confusion in the projection space. Although, the GEM algorithm (Lopez-Paz and Ranzato, 2017) introduced gradient projection to make the loss of previous tasks slowly increase in subsequent training, the embeddings computed by this method are not equivariant and cannot be applied to transformer-based language models.
In contrast, our method innovatively uses the correction matrix calculated by in-memory samples to directly constrain the gradient update in backpropagation to reduce the overall AOD, so as to limit the risk of reduced following degree. A model architecture based on knowledge decomposition and the corresponding asynchronous optimization method is designed to further reduce the ROD of samples.

heading: 3 Methodology

heading: 3.1 Problem Definition
text: CFRE model extracts the relations from a series of tasks {T 1, T 2, · · · , TK}. Every task T k has its own training dataset Dktrain, validation dataset
Dkvalid, test dataset D k test and relation label set Rk. Each set contains a small number of sample pairs {(xi, yi)}|D|i=1, where the label yi ∈ Rk. For example, in N -way M -shot constraint, we make |Rk| = N and |Dktrain| = N × M . At time step k, the model will only train on Dktrain and we hope that the model will perform well on {D1test ∪D2test ∪ · · · ∪Dktest} after training.
The memory mechanism is applied to prevent catastrophic forgetting in CFRE. Memory is defined as a series of sample sets M̂K = ∪Kj=1M j , where each Mk = {(xi, yi)}|M k| i=1 corresponds to a task T k. At time step k, a portion of the sample is selected to be stored in the memory set Mk as classical samples. In this paper, only one sample is in memory for each category.

heading: 3.2 Training Process
text: Algorithm. 1 describes the whole training process of our method at time step k. It mainly includes three different training stages to continuously and discretely adjust ROD between prototypes and samples.
In the first stage, we initialize the current memory set Mk and proto-embedding P k. Then, the temporary memory {M̂k−1 ∪Mk} with all training data is applied to preliminarily update the PLM parameters θk−1 through the corrected gradient.
In the second stage, we introduce cosine similarity to find one sample that is most similar to the center of the category cluster and add it to memory M̂k−1. The PLM parameters update is similar to the first stage, except for using selected memory data.
In the final stage, we freeze the gradient of PLM and view proto-embeddings P̂ k as learnable parameters, fine-tuning them on training data.
The data and gradient flow are shown in Fig. 2.

heading: 3.3 Knowledge Decomposition
text: To prevent the parameters in the model from being covered and confused due to continuous task iteration, we abstractly decompose them into general and task-related knowledge. Since they do not interfere with each other during the update process, ROD for different categories can be effectively reduced.

heading: 3.3.1 General knowledge encoder
text: Because of the training with mask prediction on a large number of corpora, the knowledge of BERT parameters is independent of specific downstream
tasks. Therefore, we choose BERT as PLM to encode the general knowledge of the whole relation extraction model. Specifically, for a sentence x = {Tok1, T ok2, · · · , T okn}, we first construct a template function based on prompt learning (Liu et al., 2023) by adding a special [MASK] token to the sentence and get the sequence xseq = {[CLS], eh, [MASK], et, [SEP ], x, [SEP ]}. eh and et respectively represent the head and tail entity in the sentence x. Then the embedding of [MASK] token is taken as the relation representation of the whole sentence:
h[MASK] = fθ(xseq) (1)
To facilitate symbolic representation, xseq is replaced by x.

heading: 3.3.2 Task-related knowledge encoder
text: To ensure that the new task knowledge learned by the model does not conflict with the previous task and model general knowledge respectively, we encode the knowledge of different tasks into the corresponding category proto-embedding. Since the proto-embeddings of different categories are disconnected, the knowledge from new and previous tasks will not affect each other naturally. And this part of knowledge is updated and trained asynchronously with BERT, so it also does not
conflict with the general knowledge. The protoembeddings are updated with different strategies in the three stages of training.
In the first stage of task k, we apply all samples of each new relation in the current task to calculate the new proto-embedding. To get the most representative vector in the sample embedding space, the average method is introduced to aggregate all embeddings of samples. For the proto-embedding pkj of class j task k, the calculation formula is as follows:
P kj = 1 |Dkj | ∑
(xi,yi)∈Dkj
fθ(xi) (2)
where Dkj = {(xi, yi)|(xi, yi) ∈ Dktrain, yi = rj} and |Dkj | is the number of samples in Dkj .
In the second stage, there is only one sample for each class in memory, so we directly use the relation embedding of that sample as the protoembedding of the corresponding class.
The updating of proto-embeddings in the first two stages is discrete. In the third stage, we freeze the parameters of BERT and regard the proto-embedding as a parameter that can be updated by the gradient descent algorithm. The protoembeddings are continuously updated by the gradi-
Algorithm 1 Training procedure for T k(k > 1) Input: The PLM parameters θk−1 and protoembedding P̂ k−1 = ∪k−1j=1P j trained on T k−1, training data set Dktrain, memory set M̂
k−1, learning rate γ Output: θk, P k
Use all samples in Dktrain to initialize M k and P k Freeze the gradient of P̂ k−1, unfreeze the gradient of θk−1 for i ∈ {1, · · · , epoch1} do Calculate correction matrix C using M̂k−1
Calculate corrected gradient ∆θk−1 using C with Lgen on D = {M̂k−1 ∪Mk ∪Dktrain} Update θk−1 ← θk−1 − γ∆θk−1 Recalculate P k using Mk end for Select typical samples from Dktrain to update Mk M̂k ← M̂k−1 ∪Mk for i ∈ {1, · · · , epoch2} do
Calculate correction matrix C using M̂k−1 Calculate corrected gradient ∆θk−1 using C with Lgen on D = {M̂k ∪Dktrain} Update θk−1 ← θk−1 − γ∆θk−1 Recalculate P k using Mk end for P̂ k ← P̂ k−1 ∪ P k Freeze the gradient of θk, unfreeze the gradient of P̂ k for i ∈ {1, · · · , epoch3} do Calculate gradient ∆P̂ k with Ltask on D = Dktrain Update P̂ k ← P̂ k − γ∆P̂ k end for
ent.
pkj ← pkj − γ ∂Ltask
∂pkj (3)
where γ is the learning rate. Ltask is calculated using cross entropy function, which is similar to Eqn. 5. We set P k = ∪|R
k| j=1p k j .

heading: 3.3.3 Calculation of loss function
text: To optimize the learned embeddings of relations, the training and inference of our model are based on metric learning. At time step k, by measuring the similarity between each sample xi and protoembedding P̂ k, the relation distribution is calcu-
lated as:
p(ri|xi) = exp(d(fθ(xi), pi))∑|Pk| l=1 exp(d(fθ(xi), pl))
(4)
where d(·, ·) is the distance metric function (cosine similarity in this paper) and pl is proto-embedding in P̂ k.
Further, the cross entropy loss function is calculated to measure the classification error of the model:
Lce = − ∑
(xi,yi)∈D logp(ri|xi) (5)
While alleviating catastrophic forgetting, it is also important to correctly handle the information entropy of new tasks in the model. To avoid the confusion of similar relation between new task and previous tasks, we select the proto-embedding sets P simi = {pl|d(fθ(xi), pi) − d(fθ(xi), pl) < α} and Pnegi = {pl|max(d(fθ(xi), pl), l ̸= i}, which are easy to be confused with the correct category for each sample xi. α is the set similarity threshold. The following probability is reduced by cross entropy loss function:
Lsim = − ∑
(xi,yi)∈D
log exp(d(fθ(xi), pi))
∑|P simi ∪Pnegi ∪pi| l=1 exp(d(fθ(xi), pl))
(6)
The final general knowledge loss is calculated as:
Lgen = Lce + Lsim (7)

heading: 3.4 Adaptive Gradient Correction
text: Through the decomposition of parameters, the taskrelated knowledge will not interfere with each other and ROD of previous tasks will be kept at a low level in the process of continual learning. However, when updating the parameters of the general knowledge encoder, there will still be gradient interference between tasks, which leads to the coverage of the general knowledge in BERT. These coverage may make the embedding and AOD of previous task sample change dramatically, which will increase the risk of reducing the following degree.
To avoid the interference of previous task embeddings when BERT learns a new task, the memory embedding of the corresponding hidden layer is extracted to adaptively correct the feed forward
networks (FFN) and query matrix gradient in each layer during the model back-propagation. And the other gradients in BERT are frozen besides these two kinds of gradients.
To express simplicity, we first abstract all layers of the encoder into fully connected layers to introduce our idea. At time step k, we first extract the hidden state H l ∈ Rd×|M̂k−1| of samples in M̂k−1 before inputting each unfrozen layer. Then the gradient correction matrix C l of a certain layer is calculated according to H l (The superscripts are omitted for clarity):
C = I −H(HTH)−1HT (8)
Obviously, we can get CH = HTCT = 0. To guarantee the reversibility in the specific calculation, we add a small offset: (αI +HTH)−1. The calculation method of α is consistent with Guo et al. (2022).
When using the back-propagation algorithm to calculate the gradient of a certain layer, the unfrozen gradient of BERT is corrected by C l. The parameter W l update formula for a certain layer is shown as follows:
∆W l = ∂Lgen ∂W l C l
W l ←W l − γ∆W l (9)
where γ is the learning rate. For the parameters in a transformer, we only update the FFN layer parameters and the parameters related to the query matrix in the self-attention layer. The method of updating parameters in the FFN layer is similar to Equation 9. The gradient update method of the query matrix in the selfattention layer is designed as,
∆W q = (Cq)T ∂Lgen ∂W q
W q ←W q − γ∆W q (10)
where (Cq)T is the transpose row of matrix Cq, W q is parameters to calculate query matrix. The detailed discussion and further proof are included in the Appendix A.

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Datasets Consistent with previous work on CFRE (Chen et al., 2023), our experiment will be conducted on two common datasets. FewRel (He
and Jaeger, 2018) is a RE dataset that includes 80 relations, with 700 samples of each relation. We divide these relations into 8 tasks {T 1, · · · , T 8}, where each task contains 10 relations. M samples are randomly drawn to form Dktrain with the constraint of 10-way M-shot. TACRED (Zhang et al., 2017) is a large-scale RE dataset based on news networks and online documents, containing 42 relation labels and 106,264 samples. Samples in TACRED are imbalanced compared with FewRel. We remove the particular relation "n/a" (not available) and divide the remaining 41 relations into eight subsets. The first subset has one more relation than other subsets
Evaluation Metrics At time step k, we test the model on D̂ktest, the union of all visible relation test sets, which can simultaneously reflect the model performance on new and old tasks. Since CFRE may be affected by the task sequence, we run random seeds six times on different task sequences to ensure the randomization of that. The mean and variance of relation classification accuracy on six different task sequences are introduced as the performance of the model. The training details are discussed in Appendix B.
Baselines Four baselines are introduced to compare our method (AGCKD). CEAR (Zhao et al., 2023) is a CRE approach that designs memoryinsensitive relation prototypes and memory augmentation to overcome the overfitting problem. SCKD (Wang et al., 2023) is a contrastive learning scheme for CFRE, which employs serial knowledge distillation and pseudo-samples for contrastive learning to keep the representation of samples in different relations distinguishable. In ERDA (Qin and Joty, 2022), the embedding spatial regularization and data augmentation algorithms are proposed to optimize memory expression in CFRE tasks. ConPL (Chen et al., 2023) is the state-of-the-art method for CFRE. Prototype-based classification module, memory enhancement module, and consistency learning module are used to enhance the consistency of distribution as well as avoid catastrophic forgetting. In addition to these baselines, we also added two experimental settings to observe the upper and lower limits. In Joint Training setting, the model saves all training samples in the memory. Since the model can replay all past data at every time step, there is no catastrophic forgetting. In SeqRun setting, the memory does not save any samples. This setting may cause the
model to face severe catastrophic forgetting, so it serves as a lower bound.

heading: 4.2 Main Results

heading: 4.2.1 FewRel Benchmark
text: The accuracy of AGCKD for each task on Fewrel’s 10-way 5-shot, 2-shot and 10-shot is described in Table 1 and Fig. 3. From these results, we can observe that:
(1) By comparing the mean in Table 1, we can find that AGCKD is significantly higher than the traditional method at each time step. The performance in T 1 indicates that AGCKD can effectively adopt the general knowledge learned by BERT. Meanwhile, AGCKD also achieves state-of-the-art performance in T 8, reflecting that the interference between different task parameters in AGCKD is the least, which mainly benefits from adaptive correction of the gradient and efficient decomposition
of knowledge during model training. As a traditional method of CRE, CEAR requires many training data to learn the memory and embedding of a new task. When there is less training data, the features learned by the model will become unstable, which makes it collapse with the performance. ConPL and ERDA adjust the consistency of embedding indirectly only from the perspective of the loss function and data augmentation, which has limited ability to improve following degree. We further discuss the forgetting metrics of ConPL, ERDA and AGCKD in Appendix C.
(2) The standard deviation of AGCKD on the final task T 8 remains at a low level compared with the other methods, which indicates that it has a small fluctuation range when faced with different task sequences. It further reflects the robustness of AGCKD under different task sequences, thanks to the gradient correction matrix constraining the off-
set of the previous samples’ embedding during parameter update. To prove this, we calculate the sum of AOD and ROD for each task in the model and show them in Fig. 4. It can be seen that AGCKD has low-level AOD and ROD on all tasks. In combination with Table 1, we observe that the smaller the sum of AOD and ROD of the model, the higher the accuracy. This is also consistent with our previous theoretical analysis of the following degree and offset distance.
(3) As is shown in Fig. 3, the performance of AGCKD is close to Joint Training, which indicates that AGCKD is less affected by catastrophic forgetting during training. In this case, we consider that more information introduced by new tasks also unexpectedly leads to the decline of model performance. Because the training times of samples at different time steps may be uneven during joint training, AGCKD even exceeds Joint Training on some tasks.

heading: 4.2.2 TACRED Benchmark
text: The performance of AGCKD for each task on TACRED’s 5-way 5-shot and 10-shot is shown in Table 1 and Fig. 3. It can be observed from these results that AGCKD still has advantages over stateof-the-art methods. AGCKD has a strong generalization ability, which depends on our approach without any dataset-specific components.

heading: 4.3 Ablation Study
text: To verify the effectiveness of each part in AGCKD, we performed ablation experiments. Specifically, we separately remove adaptive gradient correction (w.o.AGC) and knowledge decomposition (w.o.KD). Average ablation results for the final tasks are presented in Fig. 5. The ablation experiments regarding the three training stages are shown
in the Appendix 4. Through the box line diagram, the adaptive gradient correction and knowledge deconstruction module have a great impact on the average accuracy and performance stability of the model. The separate use of these two modules leads to confusion about the corresponding part of knowledge, which greatly damages the performance of the model. According to the definition, while reducing AOD, it also indirectly reduces ROD. Thus, the adaptive gradient correction has a greater impact on the final result. This phenomenon again demonstrates the effectiveness of the adaptive gradient correction algorithm and the importance of reducing the impact of subsequent tasks on previous tasks in the embedding space.

heading: 5 Conclusion
text: In this paper, a method of direct decoupling parameters and modifying gradient is proposed to improve the following degree of samples, which can eventually reduce the catastrophic forgetting for CFRE. Specifically, we first propose the concept of the following degree and analyze it from the perspectives of AOD and ROD. The parameters of the model are decomposed into general and task-related knowledge based on metric learning. For general knowledge, an adaptive gradient correction algorithm is proposed to reduce the impact of gradient updates on previous knowledge, which can reduce the AOD of samples. For task-related knowledge, we update the parameters discretely and continuously in three different training stages to optimize ROD between prototypes and samples. The theoretical derivation and experimental results on two standard benchmarks verify the superiority of AGCKD.
Limitations
In practical calculations, a small ROD is a sufficient and unnecessary condition for better model performance. Specifically, if ROD is large and the distance between prototypes is also relatively large, the model performance will not also be poor. In future work, we hope to consider the distance between classes in ROD and obtain a necessary and sufficient condition for model performance.
Currently, in the field of NLP, since the transformer-based model is the most widely used language model, we have only explored gradient correction algorithms for the relevant structures in the transformer. The performance of AGCKD in CFRE based on a transformer shows us the potential for designing gradient correction algorithms in other model architectures. And we will further explore the algorithm for gradient correction of parameters in other network structures (such as CNN, RNN, etc.).

heading: A Mathematical Proof of Adaptive Gradient Correction Algorithm
text: To prove that the algorithm can adaptively correct the gradient so that the model updates have less impact on the previous task samples, we record H li ∈ Rd×1 as the layer l embedding corresponding to the in-memory sample of category i and Eli ∈ Rd×1 as that corresponding to any training sample of category i. Since the in-memory samples are calculated according to the maximum cosine similarity of the average class sample, the vector angle of H li and E l i is much smaller than that of H lj(j ̸=i) and E l i (also thanks to the optimization of metric learning). C li is essentially a linear transformation of any vector into the space which is orthogonal to H li . Accordingly, when the angle between H li and E l i is small, E l i is approximately equal to zero vector after linear transformation through C li . From a single vector to the entire C l matrix, we can get:∆W lEl = ∂Lgen
∂W l (C lEl) ≈ 0.
For the FFN layer in BERT, it avoids the interference of the gradient update of subsequent tasks on the embeddings of previous tasks:
OutFFNk = X +W FFNX
= X + (WFFN + γ ∂Lgen
∂WFFN CFFN )X
= OutFFNk+1 (11)
where OutFFNk is the output of FFN at time step k, X ∈ Rd×1 is [MASK] token embedding of any previous sample, and CFFN is calculated by memory M̂k.
For the self-attention module in BERT, we freeze the key and value matrices, while the gradient correction and parameter update are only performed on the query matrix. The common self-attention formula is as follows:
Q = XTW q
K = XTW k
V = XTW v
SelfAttention(Q,K, V ) = softmax( QKT√
d )V
(12)
where X = [x1, · · · , x[mask], · · · , xn] ∈ Rd×n is the token embedding sequence of any previous
sample. We set the updated Q as Q′:
Q′ =XT (W q + γ(Cq)T ∂Lgen ∂W q )
=[q′1, · · · , q[mask], · · · , q′n]T (13)
where vector q′n ∈ R1×d is the nth row of matrix Q′.
Q′KT =   q′1k T 1 q ′ 1k T 2 · · · q′1kTn ... ... . . . ... q[mask]k T 1 q[mask]k T 2 · · · q[mask]kTn ... ... . . . ...
q′nk T 1 q ′ nk T 2 · · · q′nkTn
 
(14)
Let A′ = softmax(Q ′KT√ d
), then the row vector corresponding to [MASK] token in matrix A′ is consistent with that in A, that is, A′ = [a′1, · · · , amask, · · · , a′n]T . Thus, the embedding of [MASK] token in A′V is consistent with that before the gradient update, i.e. the output of selfattention related to previous tasks is not affected by the updated gradient of subsequent tasks. In other words, our gradient correction method can also avoid the knowledge coverage caused by gradient updates in the self-attention layer.

heading: B Training Details
text: The BERT-base (Devlin et al., 2018) is used as the encoder and is trained using AdamW (Loshchilov and Hutter, 2018) optimizer at a learning rate γ = 2e − 5. The head of the attention mechanism is set to 8 and the embedding size of BERT is 768, i.e. d = 768. The batch size is set to 5. We train the model one time at the first training step (epoch1 = 1), and three times at the second step and third step (epoch2 = epoch3 = 1). We set α = 0.2 in P sim. The memory is size set to 1 for each class. AGCKD can complete all tasks in about 20 minutes using one NVIDIA 4060Ti GPU. We run the model 6 times with different task sequences and report the mean result.

heading: C Forgetting Metrics
text: The degree of catastrophic forgetting of AGCKD is measured by the forgetting metrics proposed by Chaudhry et al. (2018). After learning all tasks, the
following formula is calculated on all test datasets:
Fk = 1 n− k n∑
j=k+1
max l∈{k,...,j−1}
(al,k − aj,k) (15)
where al,k is the accuracy of the model on task k after the training step l and n is the total number of tasks. Fk(k ∈ [1, · · · , n−1]) is the forgetting metrics of task k after all training steps. Obviously, the smaller Fk, the less knowledge the model forgets.
Table 2 shows the forgetting metrics after training for each task on FewRel’s 10-way-5-shot. The average forgetting metrics of AGCKD are lower than that of other methods and are closest to Joint Training, which indicates that AGCKD can reduce catastrophic forgetting to a certain extent. At the same time, we find that even if all the training data are available, forgetting still exists. This may be because this index does not fully consider the information increment of the newly introduced task.
To observe the forgetting metrics more intuitively, Table 3 directly shows the total error rate and that related to T 1 at each time step k on D1test. The errors related to the first learning task of the model almost do not continue to increase, which intuitively reflects that AGCKD can effectively avoid the catastrophic forgetting of previous tasks.

heading: D Forgetting Metrics
text: To explore the impact of different training stages on model performance, we conduct ablation experiments for each of the three training stages. As shown in Table 4, the absence of each training stage reduces the model’s performance on the final task
to varying degrees. The absence of the second training stage has the greatest impact on the model. The second training stage focuses on in-depth learning of in-memory samples and current task samples, whose absence has the greatest impact on the model. PLM learns basic discriminative skills and stable general knowledge from a large amount of data in pre-training, which are sufficient to support model inference when the number of tasks is small. The features learnt by models using small amounts of data in the second stage instead lead to unstable general knowledge in the PLM, which makes models missing the second phase perform better in the early phase. However, when the number of tasks is increasing, the lack of in-depth learning of the current task samples leads to an accumulation of errors. When the number of tasks is increasing, the general knowledge in the PLM can no longer support the model to make accurate inferences, which leads to the worst performance in the later stages of the model missing the second training phase.

heading: Future Work
text: In future work, we hope to consider the distance between classes in ROD and obtain a necessary and sufficient condition for model performance. Currently, in the field of NLP, since the transformer-based model is the most widely used language model, we have only explored gradient correction algorithms for the relevant structures in the transformer. The performance of AGCKD in CFRE based on a transformer shows us the potential for designing gradient correction algorithms in other model architectures. And we will further explore the algorithm for gradient correction of parameters in other network structures (such as CNN, RNN, etc. )