Introducing Common Null Space of Gradients for Gradient Projection Methods in Continual Learning

Authors: chengyi  yang
mingda  dong
xiaoyue  zhang
jiayin  qi
aimin  zhou
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-28 18:19:53

Abstract: Continual learning aims to learn new knowledge from a sequence of tasks without forgetting.  Recent studies have found that projecting gradients onto the orthogonal direction of task-specific features is effective. However, these methods mainly focus on mitigating catastrophic forgetting by adopting old features to construct projection spaces, neglecting the potential to enhance plasticity and the valuable information contained in previous gradients. To enhance plasticity and effectively utilize the gradients from old tasks, we propose Gradient Projection in Common Null Space (GPCNS), which projects current gradients into the common null space of final gradients under all preceding tasks. Moreover, to integrate both feature and gradient information, we propose a collaborative framework that allows GPCNS to be utilized in conjunction with existing gradient projection methods as a plug-and-play extension that provides gradient information and better plasticity. Experimental evaluations conducted on three benchmarks demonstrate that GPCNS exhibits superior plasticity compared to conventional gradient projection methods. More importantly, GPCNS can effectively improve the backward transfer and average accuracy for existing gradient projection methods when applied as a plugin, which outperforms all the gradient projection methods without increasing learnable parameters and customized objective functions. The code is available at https://github.com/Hifipsysta/GPCNS.

Main paper: 
 introduction: 
 Machine learning algorithms lack the ability to continually learn new knowledge like humans. Specifically, a neural network will perform less well on old tasks after learning a new task, and this phenomenon is known as catastrophic forgetting (CF) [10, 24] . To address this issue, continual learning (CL) [30] has been proposed and applied to computer vision, natural language processing and multi-modal scenarios, including object detection [8, 22, 36, 44] , semantic segmentation [9, 25, 35, 46] , relation extraction [4, 13, 34] , neural machine translation [21, 29, 43] , cross-modal retrieval [38, 41] and visual question answering [16, 26, 42] .Continual learning considers the scenario of learning a stream of task-specific data sequentially, expecting to retain the knowledge related to old tasks when learning current task. Recently, gradient projection methods [3, 20, 27, 28, 33] have found that catastrophic forgetting can be effectively alleviated if the gradients under current task are orthogonal to the subspaces spanned by features from old tasks. These methods all project gradients onto the orthogonal direction of previous features, which have been demonstrated to be equivalent and hold a unified training paradigm called feature space continual learning paradigm (FSCLP) [45] .Figure 1 : FSCLP: Constructing a projection space through representation matrix, which is originated from collecting a sufficient number of (e.g. 100) samples from the dataset. GPCNS: Constructing a projection space throgh common gradient matrix, which is obtained after stacking final gradients from all previous task. We believe that feature and gradient information are equally important, and utilizing them simultaneously can construct better projection spaces.However, FSCLP methods primarily rely on features from previous tasks for constructing projection spaces, neglecting the valuable information contained in previous gradients. In addition, we note that the design idea of FSCLP approaches lies in alleviating catastrophic forgetting through minimizing parameter shifting, which is closely related to stability. Although these methods have proven effective in enhancing stability, they lack sufficient consideration in improving plasticity.To improve plasticity and effectively utilize the gradients from old tasks, we attempt to demonstrate that gradients from previous tasks are equally powerful and important as previous features when applied to construct projection spaces. We cannot simply replace the features under FSCLP with gradients, because we have found that existing algorithms designed for features cannot fully mine gradient data (see Table 4 ). Therefore, we propose a novel approach called Gradient Projection in Common Null Space (GPCNS), which adopts a novel training paradigm as shown in the lower half of Figure 1 . More importantly, since the design concept and information source of GPCNS are different from FSCLP, these two types of methods can be applied simultaneously to construct a better projection space. Motivated by this insight, we propose a dual information fusion framework to utilize both features and gradients from previous tasks. The contributions of this paper can be summarized as follows:(1) We analyze that the effectiveness of existing methods under FSCLP lies in minimizing parameter shifting, which only focuses on stability. To improve plasticity, we suggests that gradients under different tasks should be projected in mutually orthogonal subspaces.(2) We present a novel concept for projection space called common null space, which has rigorous mathematical proofs and geometric interpretation. In common null space, the mutual interference between gradients under different tasks can be minimized.(3) We propose GPCNS which projects gradients into the common null space of final gradients under all previous tasks. GPCNS effectively utilizes the gradient information of old tasks and posses higher plasticity (see Section 4.4), which are exactly what existing FSCLP methods lack.(4) We propose a collaborative framework for FSCLP and GPCNS, which integrates feature and gradient information to construct a better projection space. GPCNS effectively compensates for the shortcomings of FSCLP methods in utilizing gradient information and plasticity.2 Related Work 
gradient projection methods in cl: 
 We focus on the gradient projection methods in continual learning, which project gradients onto the orthogonal direction of the feature space related to previous tasks. Orthogonal Weight Modification (OWM) [40] constructs projection operators through recursive least squares algorithm. However, OWM exhibits unstable performance due to its low backward transfer. Adam-NSCL [33] projects gradients into the null space of uncentered covariances of features, but the higher computational cost limits its application. Gradient Projection Memory (GPM) [27] stores the bases of features for each task in a memory buffer, and then projects the gradients onto the orthogonal direction of old features. Subsequent gradient projection methods can be seen as the improvements based on GPM. Class Gradient Projection (CGP) [3] computes projection subspaces from classes instead of from tasks. Trust Region Gradient Projection (TRGP) [20] and Scaled Gradient Projection (SGP) [28] enhance GPM by scaling the bases of feature spaces according to task similarity and importances respectively. In addition, Data Augmented Flatness-aware Gradient Projection (DFGP) [37] and Connector [18] redesign the objective function in GPM and Adam-NSCL respectively, which are essentially regularization based methods.Gradient Episodic Memory (GEM) [23] and Averaged GEM (A-GEM) [1] adjust the current gradients based on the gradients computed with data in the memory. OGD [7] projects the new gradients orthogonally to the memorized gradients through Gram-Schmidt process, but it does not consider features and fail to achieve high performance. TRGP [20] and Adaptive Plasticity Improvement (API) [17] utilize the length of gradient projection to measure the correlation between tasks and determine whether to expand the network respectively. CUBER [19] and DFGP [37] utilize gradient information to design objective functions that encourage feedback knowledge transfer and flatter loss landscapes respectively. However, they all consider the gradients of old tasks as auxiliary information. Moreover, FS-DGPM [5] and CGP [3] have mentioned gradient subspaces, but they refer to the projection space for current gradient, rather than constructing the projection space through previous gradients. To our knowledge, this is the first work that gradient and feature information are considered equally important for constructing projection spaces in continual learning. 
methods: 
 In this section, we analyze the causes of catastrophic forgetting and the effectiveness of FSCLP. Then we propose GPCNS and a collaborative framework for GPCNS and FSCLP respectively. represents the training data under task 𝑡, and 𝑁 𝑡 is the data size. 𝑿 𝑡,𝑖 and 𝑦 𝑡,𝑖 are the 𝑖-th input data and label under task 𝑡, respectively. The method described below is for a certain layer in a network, which can be generalized to all the layers, thus we omit the layer notation 𝑙.Forgetting Issue. When learning a new task, the parameter tensor will deviate from its optimal value for former tasks due to the increased knowledge related to current task. This process can be formally described asEQUATIONwhere [3, 20, 27, 28, 33, 37] project the gradients onto the orthogonal direction of task-specific feature space to ensureEQUATIONwhere X denotes the feature space and X ⊥ is its orthogonal direction. From Eq.(2), we observe that: (1) FSCLP aims to minimize the parameter shifting to maintain the performances on old tasks. This idea is closely related to stability, while it lacks consideration for plasticity. (2) Only previous features (𝑿 𝑡 -1,𝑖 ) are applied to construct the projection space for current gradients.GPCNS is different from existing FSCLP methods, because it is designed from the perspective of plasticity. Moreover, it utilize previous gradients rather than features to construct projection spaces. This idea stems from the observation of the parameter updating formula under gradient projection:EQUATION)We find that if the projected gradients under different tasks always appear in mutually orthogonal subspaces, then current gradients can be updated with confidence rather than considering mutual interference. This idea can also be equivalently expressed as establishing an orthogonal set composed of the projected gradients under different tasks:EQUATION)Figure 2 : Illustration of the common null space for the third task. For simplicity, we assume that there are only two bases in each subspace. The range space of the projected gradients in task 2 (red area) is orthogonal to the gradients in task 1 (green area), i.e. R (𝑃𝑟𝑜 𝑗 (∇L 2 )) ⊥ R (∇L 1 ). The gradient generated in each epoch under task 3 will be projected into the common null space (see dashed area) for all previous tasks, namely tasks 1 and 2. This proposition can also be equivalently stated as the range space of 𝑃𝑟𝑜 𝑗 (∇L 3 ) is the common null space of ∇L 1 and 𝑃𝑟𝑜 𝑗 (∇L 2 ) according to Lemma 1 (see supplementary materials), i.e. R (𝑃𝑟𝑜𝑗 (∇L 3 )) = N (∇L 1 ) ∩ N (𝑃𝑟𝑜 𝑗 (∇L 2 )).Here 𝑃𝑟𝑜 𝑗 (∇L 𝑡 ) ∈ R 𝑚×𝑛 denotes the projected gradient under task 𝑡, and Eq.( 4) can be further simplified asEQUATIONwhere 𝑮 𝑡 ∈ R 𝑡𝑚×𝑛 is the common gradient matrix, formed by the vertical concatenation of the projected gradients from all previous tasks, namely:EQUATION)Here ∇L * 𝑡 ∈ R 𝑚×𝑛 is the gradient under task 𝑡 at the end of training. We only take the final gradient because the projected gradients under the same task are in the same subspace, namely N (𝑃𝑟𝑜 𝑗 (∇L 𝑡 )) = N (𝑃𝑟𝑜 𝑗 (∇L * 𝑡 )). We denote N (•) and R (•) as the null space and range space respectively, their definitions are presented in supplementary materials.If Eq.( 5) holds, the gradients under GPCNS will be projected into the common null space of 𝑃𝑟𝑜 𝑗 (∇L 𝑡 ) under all previous tasks, namely 𝑃𝑟𝑜 𝑗(∇L 𝑡 +1 ) ∈ N (∇L 1 ) ∩ N (𝑃𝑟𝑜 𝑗 (∇L 2 )) ∩ • • • ∩ N (𝑃𝑟𝑜 𝑗 (∇L 𝑡 )).To illustrate this idea, we demonstrate the projection space for the third task as an example in Figure 2 , which is also the common null space for ∇L 1 and 𝑃𝑟𝑜 𝑗 (∇L 2 ).To obtain the common null space, we perform SVD on 𝑮 𝑡 :EQUATIONHere𝑼 𝑡 = 𝑼 𝑡 , 𝑼 𝑡 ∈ R 𝑡𝑚×𝑡𝑚 , 𝑽 𝑡 = 𝑽 𝑡 , 𝑽 𝑡 ∈ R 𝑛×𝑛 and 𝚺 𝑡 ∈ R min{𝑡𝑚,𝑛}×min{𝑡𝑚,𝑛} . Then we have 𝑮 𝑡 • 𝑽 𝑡 = 𝑼 𝑡 𝚺 𝑡 ( 𝑽 𝑡 ) ⊤ 𝑽 𝑡 = 𝑼 𝑡 𝚺 𝑡 •0 = 0, since 𝑽 𝑡 is an orthogonal matrix. Thus, 𝑽 𝑡 ∈ R 𝑛× (𝑛-𝑡𝑚)is the null space of 𝑮 𝑡 (see Definition 3 in supplementary materials). However, we do not adopt 𝑽 𝑡 because it will not exist if 𝑡𝑚 ≥ 𝑛. Algorithm 1 Algorithm for GPCNS end for 21: end for Instead, 𝑰 -𝑽 𝑡 𝑽 ⊤ 𝑡 ∈ R 𝑛×𝑛 is also the null space of 𝑮 𝑡 (see proof in supplementary materials), where 𝑰 ∈ R 𝑛×𝑛 is the identity matrix.Input: Datasets {D 1 , D 2 , • • • , D 𝑇 }Before constructing the projection operator, we select 𝑘 𝑡 vectors in 𝑽 𝑡 through the criteria:EQUATIONHere ∥ • ∥ * denotes the nuclear norm (sum of the singular values) and 𝜖 𝑡 ∈ [0, 1] is the threshold. (𝑮 𝑡 ) 𝑘 𝑡 ∈ R 𝑘 𝑡 ×𝑘 𝑡 contains 𝑘 𝑡 vectors with top-𝑘 𝑡 largest singular values. The bases contained in 𝑽 𝑡 will be selected as 𝑽 𝑡 = 𝑽 𝑡 [: , 0 :𝑘 𝑡 ] ∈ R 𝑛×𝑘 𝑡 .To provide GPCNS with more bases to choose from, we define the matrix 𝑯 1 = 𝑽 1 ∈ R 𝑛×𝑘 1 in task 1. When 𝑡 ≥ 2, we update the matrix 𝑯 𝑡 by stacking 𝑯 𝑡 -1 ∈ R 𝑛×ℎ 𝑡 -1 and 𝑽 𝑡 horizontally: We scale the gradients in GPCNS through the following formula [28] :EQUATIONEQUATION)Here 𝜎 𝑡,𝑖 is the 𝑖-th singular value of 𝑯 𝑡 under task 𝑡, namely𝚺 𝑡,𝐻 𝑡 = diag([𝜎 𝑡,1 , • • • , 𝜎 𝑡,min{𝑚,𝑛} ]). The value of 𝜆 𝑡,𝑖 are range from 0 to 1 with the above construction.𝝀 𝑡 = 𝜆 𝑡,1 , 𝜆 𝑡,2 , • • • , 𝜆 𝑡,𝑛 ⊤is the importance vector under task 𝑡. We perform scaled gradient projection on gradient ∇L 𝑡 +1 as follow:EQUATIONHere 𝚲 𝑡 ∈ R 𝑛×𝑛 is a diagonal matrix containing 𝝀 𝑡 in its diagonal.The main steps of our GPCNS are summarized in Algorithm 1.To construct projection spaces through both gradient and feature information, we propose a collaborative framework between FSCLP and GPCNS, which has two additional stages as following.1) Obtaining Bases Related to Features. After obtaining 𝑽 𝑡 ∈ R 𝑛×𝑘 𝑡 through step 10-14 in Algorithm 1, we further utilize GPM [27] , TRGP [20] or SGP [28] as an algorithm under FSCLP to compute a memory matrix 𝑴 𝑡 ∈ R 𝑛×𝛽 𝑡 which stores the bases related to features. Here 𝛽 𝑡 is a positive integer, its value depends on how many bases are selected to represent the feature space.The first dimension of 𝑴 𝑡 is equal to 𝑛 because the gradient projection formula ∇L 𝑡 = ∇L 𝑡 -∇L 𝑡 𝑴 𝑡 𝑴 ⊤ 𝑡 in FSCLP methods [3, 20, 27, 28] require that the rows of 𝑴 𝑡 equals the columns of ∇L 𝑡 ∈ R 𝑚×𝑛 .2) Integrating Gradient and Feature Information. To integrate two types of information, we stack 𝑽 𝑡 and 𝑴 𝑡 horizontally:EQUATION)To filter the redundant bases, we perform SVD on Then, we also utilize the scaling matrix to adjust the projected gradient (see Eq.( 10)), and the final gradient projection formula isEQUATIONThe pipeline of our collaborative framework is demonstrated in Figure 3 and the main steps are summarized in Algorithm 2 (see supplementary materials). 
experimental setting: 
 Datasets. We evaluate our methods on Split CIFAR-100 [14] , CIFAR-100 Superclass [39] and Split MiniImageNet [31] . Split CIFAR-100 contains 60,000 RGB images over 100 classes splitted into 20 tasks with 10 distinct classes per task. Each class contains 500 training images and 100 testing images, and the size of each image is 32 × 32. CIFAR-100 Superclass is divided into 20 tasks where each task contains 5 semantically related classes from CIFAR-100. Split MiniImageNet is a 100-class subset of the original ImageNet [6] , which is splitted into 20 tasks. Each class contains 500 training images and 100 testing images. These images are in RGB format with 84 × 84 as their sizes.Implementation Details. For the convenience of comparison, we adopt the same backbone with GPM, TRGP, and SGP on each dataset. Specifically, we apply a 5-layer AlexNet [15] on Split CIFAR-100, a LeNet on CIFAR-100 Superclass and a reduced ResNet-18 [11] on Split MiniImageNet to conduct experiments. All the methods are trained and tested under task-incremental learning setup that each task has a separate classifier head [12] . All the experiments are conducted under the optimization of Stochastic Gradient Descent (SGD), and the batch sizes are set as 64. Each task in Split CIFAR-100 and Split MiniImageNet are trained for 200 epochs, and in CIFAR-100 Superclass are trained for 50 epochs. Other details are presented in supplementary materials.Baselines. To follow the consistent experimental standards of GPM, TRGP, CGP and SGP, we require that the baselines not have the following situations: (1) Increasing parameters during the training process (e.g. API [17] ). ( 2) Replacing cross entropy loss with customized objective functions (e.g. Connector [18] and DFGP [37] ).(3) In addition, methods without open source code cannot be considered as baselines. (e.g. SD [45] ). The reasons are as follows: (i) Increasing the number of parameters and replacing the loss function are essentially network expansion methods and regularization methods respectively. (ii) Our motivation is to improve gradient projection, rather than stacking different types of techniques. (iii) It is difficult to accurately reproduce methods without publicly available code.Referring to the baseline selection in SGP [28] , we select the following methods: OWM [40] , A-GEM [1] , Experience Replay with Reservoir sampling (ER_Res) [2] , Adam-NSCL [33] , GPM [27] , FS-DGPM [5] , CGP [3] , TRGP [20] and SGP [28] . In addition, Multitask refers to learning all the tasks simultaneously, which can be seen as the upper bound of CL.Evaluation Metrics. We employ average accuracy (ACC) and backward transfer (BWT) [23] as evaluation metrics. ACC represents average test accuracy across all tasks, and BWT measures the average decline in the test accuracy for previous tasks after learning the current task. Their definitions are as follow:𝐴𝐶𝐶 = 1 𝑇 𝑇 ∑︁ 𝑡 =1 𝐴 𝑇 ,𝑡 ; 𝐵𝑊𝑇 = 1 𝑇 -1 𝑇 -1 ∑︁ 𝑡 =1 𝐴 𝑇 ,𝑡 -𝐴 𝑡,𝑡 .Here 𝑇 is the total number of tasks, and 𝐴 𝑇 ,𝑡 is the accuracy on 𝑡-th task after learning the 𝑇 -th task sequentially.As shown in Table 1 , introducing GPCNS can effectively improve the performances of FSCLP methods on all the benchmarks. Specifically, TRGP + GPCNS improves ACC by 1.12%, 1.26% and 4.28% compared with TRGP on CIFAR-100, Superclass and MiniImageNet respectively. GPM + GPCNS achieves the accuracy gains of 1.35%, 0.46% and 0.85% compared with GPM on these three datasets. Although GPCNS does not significantly improve the ACC of SGP, the improvements of FSCLP + GPCNS on BWT are more significant. TRGP + GPCNS improves BWT by 0.84%, 1.16% and 0.53% on three datasets. SGP + GPCNS achieves the BWT gains of 1.1%, 0.66% and 0.31% respectively. In addition, we find that even when GPCNS has higher accuracy than GPM or TRGP, its BWT is still lower. The reason might be that gradient matrices have much smaller size than representation matrices (see Table 1 in supplementary materials), which leads to less bases to be selected for balancing plasticity and stability when constructing projection spaces. From the above results, we can confirm that utilizing gradient and feature information simultaneously can indeed construct a better projection space, which improves ACC and further alleviates catastrophic forgetting compared to pure FSCLP methods. Ablation on Gradient Information and Feature Information.To further verify the effectiveness of feature and gradient information, we conduct ablation experiments on task level, and the results are shown in Figure 4 . On the top row, we compare FSCLP + GPCNS that performed better on different datasets with FSCLP methods, respectively. The results show that FSCLP + GPCNS has a visible improvement almost on each task for all the datasets because of additionally considering gradient information. The middle row displays the test accuracy curves of FSCLP + GPCNS and GPCNS for each task. Their fluctuations are similar, while FSCLP + GPCNS always has higher accuracy because of having found better bases for projection. When it comes to the bottom row, FSCLP + GPCNS has the highest BWT on all the datasets, indicating that the impact of forgetting on this framework is minimal and its performances are stable.Ablation on Gradient Scaling. We also conduct ablation experiments on gradient scaling. In Table 2 , we observe that non-scaling methods reduce ACC by less than 0.5% on Superclass, and by less than 3% on CIFAR-100 as well as MiniImageNet compared to methods with scaling. In addition, we find that even if GPCNS does not utilize gradient scaling (see Eq.( 10)), it still has higher accuracy than GPM. We know that both TRGP and SGP introduce gradient scaling on the basis of GPM. Therefore, gradient information and feature information are equally powerful and important.Ablation on 𝑮 𝑡 and 𝑯 𝑡 in GPCNS. To verify the necessity of step 15-18 in Algorithm 1, we also conduct ablation experiments on 𝑮 𝑡 and 𝑯 𝑡 in Table 3 . The results show that if 𝑯 𝑡 is not utilized, the ACC will decrease by 0.49%, 0.32%, and 2.73% on CIFAR-100, Superclass and MiniImageNet respectively. Note that FSCLP + GPCNS adopts 𝑮 𝑡 rather than 𝑯 𝑡 , since 𝑴 𝑡 provides many additional bases for selecting.Ablation on Common Null Space. To verify the effectiveness of projecting gradients into common null space, we replace the features in GPM with gradients for ablation experiments in Table 4 . We observe that the accuracy of both GPM (Grad) and SGP (Grad) on CIFAR-100 Superclass collapse (see pale pink cells in Table 4 ). Therefore, FSCLP is not applicable to gradient information and proposing GPCNS with new paradigm is necessary.Ablation on norm type and top-𝐾 signular value selection. To show the effectiveness of adopting nuclear norm, we conduct ablation experiments by replacing nuclear norm with Frobius norm. To verify the necessity of selecting bases with top-𝐾 signular value, we also ablate this step and conduct experiment on three dataset. The results in Table 5 show that either replacing the nuclear norm or excluding the top-𝐾 singular values selection for filtering bases will result in a slight decrease in accuracy. To verify the better plasticity of GPCNS and its effect on improving plasticity for FSCLP methods when used as a plugin, we record the test accuracy of the final model on the first and last tasks.In Figure 5b , we can observe that GPCNS always has the highest accuracy on the last task, which validates its advantage on plasticity. Correspondingly, we can also observe that FSCLP method has better stability than GPCNS in Figure 5a . More importantly, FSCLP + GPCNS leverages the strengths of both techniques in terms of stability and plasticity, resulting in improved ACC and BWT. In Figure 6 , we compare the time consumption and memory usage of GPCNS and FSCLP + GPCNS with baseline methods. GPCNS spends the least training time on Split MiniImageNet and Split CIFAR-100, and also maintains a relatively low level on CIFAR-100 Superclass. In addition, we observe that the increased times of FSCLP + GPCNS due to simultaneously processing gradient and feature information are quite limited. In terms of memory, GPCNS has the second lowest memory overhead on all three datasets, and the memory usage of FSCLP + GPCNS has only slightly increased compared to pure FSCLP method.The above results show that the computational costs of our GPCNS and FSCLP + GPCNS are maintained at a relatively low level. The reasons are as follows: (1) Only the final gradient of each task is required in both methods. (2) The sizes of gradient matrices stored and decomposed in our methods are generally much smaller than the sizes of representation matrices applied in GPM [27] , CGP [3] , TRGP [20] and SGP [28] (see Table 1 supplementary materials).We vary the value of the scale coefficient 𝛼 in Eq. (10) . Specifically, 𝛼 is set as {1.0, 1.5, 3.0, 4.5, 6.0} successively. Figure 7 shows ACC and BWT when GPCNS and TRGP + GPCNS are performed on Split MiniImageNet. We observe that the optimal 𝛼 for GPCNS and TRGP + GPCNS on Split MiniImageNet are 3 and 1, respectively. More significantly, TRGP + GPCNS achieves positive BWT when 𝛼 equals 1, 1.5 and 4.5, which indicates that the old knowledge can be effectively transfered to new knowledge with the help of both gradient and feature information. 
discussion: 
 Comparison between FSCLP and GPCNS. In Table 6 , we compare our GPCNS with FSCLP methods, which demonstrates significant differences between GPCNS and previous gradient projection methods. GPCNS projects the gradients of previous tasks into the common null space, effectively utilizing gradient information and achieving better plasticity. More importantly, the advantage of GPCNS happens to be what the FSCLP method lacks. Therefore, utilizing GPCNS as a plugin along with the previous FSCLP method can make a gradient projection method more comprehensive and powerful. 
conclusion: 
 In this work, we analyze the effectiveness of FSCLP and find the limitations of FSCLP are neglecting gradient information and lacking plasticity. Then we propose the idea that gradients for different tasks should exist in mutually orthogonal subspaces, and further derive the concept of common null space. We propose GPCNS that projects current gradients into the common null space of the final gradients under all previous tasks. More importantly, we present a collaborative framework that GPCNS is served as a plug-and-play extension for FSCLP methods, enabling the simultaneous utilization of gradient and feature information. Finally, extensive experiments show that GPCNS exhibits higher plasticity that FSCLP methods lack. When GPCNS and FSCLP methods are employed under our collaborative framework, GPCNS can compensate for the shortcomings of existing methods in terms of plasticity and the utilization of gradient information, thereby constructing improved projection spaces and achieving enhanced performances. 
 
 Appendix:  


Decision: The paper introduces a method designed to enhance continual learning by mitigating catastrophic forgetting and improving plasticity. Pre-rebuttal this paper received diverse ratings, i.e. 1 WA, 1 BA, and 1BR. The main concerns were lack of a deeper analysis (R1), insignificant results (R1), missing discussion of scalability  issue (R1), Uncertainty of generalization ability (R2). Post-rebuttal the paper has consistent ratings: 1 WA and 2 BA. The rebuttal addressed the concerns of the reviewers. AC agrees with the reviewers and tends to accept this paper.

Reviews: review: paper_summary: paper_summary: The paper introduces a method designed to enhance continual learning by mitigating catastrophic forgetting and improving plasticity. It achieves this by projecting current gradients into the common null space of final gradients from all previous tasks. The proposed method, GPCNS, can be integrated as a plugin with existing gradient projection methods, providing a collaborative framework that enhances both the stability and plasticity of learning systems. The evaluation on three benchmarks demonstrates that GPCNS outperforms traditional methods by improving backward transfer and average accuracy without increasing the number of learnable parameters or requiring customized objective functions.
strength_weakness: strength_weakness: The concept of using common null space of gradients to project current task gradients is novel and addresses the plasticity issue in continual learning.

The paper provides rigorous mathematical proofs and a clear geometric interpretation of the common null space approach.

Extensive experiments across multiple benchmarks demonstrate the effectiveness of GPCNS in improving both backward transfer and average accuracy.

The authors have provided the code, which is beneficial for reproducibility and further research by the community.

The paper is well-written, with clear explanations of the methods and results. It also provides detailed diagrams and algorithms that aid in understanding the proposed methods.
limitations: limitations: While the paper demonstrates improvements over existing methods, it lacks a deeper analysis of how the approach compares against more recent or less conventional continual learning strategies.

The paper does not discuss the scalability of the method in detail, especially in scenarios with a very large number of tasks, which could be a limitation in practical applications.

The method's performance may depend heavily on the correct setting of hyperparameters like the scale coefficien.

Although the proposed method introduces novel ideas, the performance gains reported in the paper are relatively modest. The ablation studies, especially those pertaining to the gradient scaling and use of common null space, show only incremental improvements. This suggests that while the method is theoretically sound, its practical impact might be limited.

The paper lacks a comprehensive review of existing works that utilize the null space of gradients approach. This oversight is significant because it fails to position the proposed method within the broader context of the field. A thorough comparison with state-of-the-art methods, especially those that also leverage null space concepts in continual learning, is necessary to truly validate the novelty and effectiveness of GPCNS.
To address the gaps in literature review and comparative analysis, the authors should include a broader range of baseline methods, particularly focusing on those that employ similar strategies involving the null space of gradients, such as, 
PNSP: Overcoming catastrophic forgetting using Primary Null Space Projection in continual learning. PRL 2024.
Balancing stability and plasticity through advanced null space in continual learning. ECCV 2022.
paper_summary: paper_summary: The paper aims to enhance plasticity and effectively utilize the gradients from old tasks in Continual learning field,  and propose Gradient Projection in Common Null Space (GPCNS), which projects current gradients into the common null space of final gradients under all preceding tasks. Moreover, to integrate both feature and gradient information, we propose a collaborative framework that allows GPCNS to be utilized in conjunction with existing gradient projection methods as a plugin that provides gradient information and better plasticity.
strength_weakness: strength_weakness: The main contribution of this paper is to propose a new continuous learning strategy, gradient common null space projection (GPCNS), which aims to solve the problem that existing methods ignore gradient information and plasticity when dealing with catastrophic forgetting. By projecting the current gradient into the common null space of the final gradient of all previous tasks, the authors effectively utilize the gradient information of the old tasks and improve the plasticity of the model. In addition, the paper proposes a collaborative framework that combines GPCNS with an existing feature space-based Continuous Learning paradigm (FSCLP) to integrate feature and gradient information to build better projection Spaces. The experimental results show that GPCNS is superior to traditional methods in terms of plasticity, and can significantly improve the average accuracy (ACC) and backward transfer (BWT) of existing methods when applied as plug-ins.
limitations: limitations: 1. Uncertainty of generalization ability
The paper mainly validates the effectiveness of GPCNS on specific benchmark datasets, but does not fully explore the generalization ability of the method for a wider range of tasks and more complex scenarios.
2. Hyperparameter adjustment:
Although the paper mentions hyperparameter α has a significant effect on model performance, but there is no systematic method or theoretical guidance to select the optimal hyperparameter values.
3. Cost calculation considerations:
The paper points out that GPCNS has advantages in training time and memory usage, but the computational cost of gradient matrix accumulation has not been fully evaluated as the number of tasks increases.
paper_summary: paper_summary: This paper proposes Gradient Projection in Common Null Space to project current gradients into the common null space of final gradients under all preceding tasks to enhance plasticity and effectively utilize the gradients from old tasks. In addition, this method is plug-and-play, it can cooperate with other gradient projection methods to learn the valuable information contained in previous gradients. The proposed method outperforms all the gradient projection methods without increasing learnable parameters and customized objective functions. Ablation studies are also presented, showing the significance of each proposed technique.
strength_weakness: strength_weakness: 1. The authors propose a novel gradient projection method to use the gradient of the current task to make the model learn more knowledge, enhancing plasticity.
2. This method can cooperate with existing gradient projections to learn more knowledge.
3. Extensive evaluations indicate that this method significantly outperforms previous gradient projection methods.
limitations: limitations: 1. In this paper, the bases of gradient space and feature space are concatenated simply. Have you tried another effective way to fuse them?
2. Can these gradient projection methods be transferred into transformer architecture?
3. Can you provide the forward transfer metric to validate the plasticity of your method?