Prompt Gradient Projection for Continual Learning
Authors: Jingyang Qiao
zhizhong zhang
Xin Tan
Chengwei Chen
Yanyun Qu
Yong Peng
Yuan Xie
Venue: ICLR 2024 spotlight (2024)
Keywords: Continual Learning
Prompt Tuning
Gradient Projection
Anti-forgetting
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: Gradient projection against forgetting in prompt-tuning based continual learning method

Abstract: Prompt-tuning has demonstrated impressive performance in continual learning by querying relevant prompts for each input instance, which can avoid the introduction of task identifier. Its forgetting is therefore reduced as this instance-wise query mechanism enables us to select and update only relevant prompts. In this paper, we further integrate prompt-tuning with gradient projection approach. Our observation is: prompt-tuning releases the necessity of task identifier for gradient projection method; and gradient projection provides theoretical guarantees against forgetting for prompt-tuning. This inspires a new prompt gradient projection approach (PGP) for continual learning. In PGP, we deduce that reaching the orthogonal condition for prompt gradient can effectively prevent forgetting via the self-attention mechanism in vision-transformer. The condition equations are then realized by conducting Singular Value Decomposition (SVD) on an element-wise sum space between input space and prompt space. We validate our method on diverse datasets and experiments demonstrate the efficiency of reducing forgetting both in class incremental, online class incremental, and task incremental settings. The code is available at https://github.com/JingyangQiao/prompt-gradient-projection.

Published as a conference paper at ICLR 2024
PROMPT GRADIENT PROJECTION FOR CONTINUAL LEARNING
Jingyang Qiao'; Zhizhong Zhang'; Xin Tan', Chengwei Chen?, Yanyun Qu', Yong Peng',
Yuan Xie
East China Normal University, The Navy Military Medical University,
3 Xiamen University, *Central South University
5227590101085 08stu. ecnu. wcn..edu..c... cn, {zzzhang, xtan)0cs ecnu. edd. @1chhn91@ali iunn. com, yyquêxmu edu. cn, yong.peng&ccu..eo yxieêcs ecnu.edu..... edu.cn
ABSTRACT
Prompt-tuning has demonstrated impressive performance in continual learning by querying relevant prompts for each input instance, which can avoid the introduc- tion of task identifier. Its forgetting is therefore reduced as this instance-wise query mechanism enables us to select and update only relevant prompts. In this paper, we further integrate prompt-tuning with gradient projection approach. Our observation is: prompt-tuning releases the necessity of task identifier for gradient projection method; and gradient projection provides theoretical guarantees against forgetting for prompt-tuning. This inspires new prompt gradient projection ap- proach (PGP) for continual learning. In PGP, we deduce that reaching the orthog- onal condition for prompt gradient can effectively prevent forgetting via the self- attention mechanism in taaiontransformmrr. The condition equations are then real- ized by conducting Singular Value Decomposition (SVD) space between imput space and prompt space. We validate our method on diverse datasets and experiments demonstrate the efficiency of reducing forgetting both in class incremental, online class incremental, and task incremental settings. The code is available at
element-wiss wiie sum
gradiitttttrtttttt--eee gradhent-projectioon
INTRODUCTION
Learning continually while not forgetring is a long-standing pursuit of machine learning systems (Kumaran al.) [016,] McClelland et al. 1995 Arani al. 2022). Incremental learning, or contin- ual learning is such a fabulous way to train a model with continuously expanded datasets by adding novel classes domains (Ring 1997 Hadsell et al. 2020 De Lange et al.) 2021). In general, con- tinual learning includes two distinct settings, i.e., class- and task-incremental learning (Van de Ven & Tolias 2019) abbreviated as CIL and TIL respectively. The main difference is whether the task identifier, i.e., the samples belong to which training tasks, is given for inference.
Recently, the appearance of the prompt-tuning paradigm provides new sight for iiaseenncreettll learning (Wang et al.) 2022a Li et al., 2023). In this framework, a tiny set of trainable tokens, i.e., prompts, are combined with image features, and forwarded into a fixed Transformer architecture. As the instance-wise query mechanism can select relevant prompts according to the input sample, only these relevant parts are updated during training (Wang al. 2022c). Since the idea of prompt- tuning is borrowed from the area of natural language processing (NLP) (Lester et al. 2021 & Liang 2021), its deep mechanism against forgetting has not been revealed yet (Zhou et al. 2023).
Fortunately, it is observed that learning would not forget if the gradient is updated in the orthogonal direction to the subspace spanned by the old inputs, i.e., gradient projection approaches (GP) (Saha et al. 202T). However, one obvious limitation is that GP is only applicable for task incremental learning, because the gradient constraints would greatly restrict the learning of new tasks compared with normal training (Zhao et al. 2023). Thus, it needs task identifier to instruct updating.
"Equal contribution

---

Published as a conference paper at ICLR 2024
ACC-Imit
ACC-Tinylm
ACC 20CIFAR
ACC-Tinyi
ACC 20CITAR
FOR-ADCIFAR
ACC
FOR L0CIBAR
ACC L0CIFAR
4.5 4.6
20CIFAR
FORTinylm
FOR-2OCIFAR
FOR-Im
L2P-PGP
FOR-ami
Dual DualPramptere
(a) L2P ES. L2P-PGP
(b) DualPrompt v.s. DualPrompt-PGE
Figure I: Radar chart of comparisons in terms of average accuracy and forgetting between baselines and our methods. L2P (Wang et al. (022c) and DualPrompt (Wang et al. 2022b) are two state-of- the-art prompt-tuning approaches for continual learning. ACC refers the average accuracy metric (higher is better). FOR refers to the forgetting metric (lower is better). Different scale standards are adopted for two metrics on benchmark datasets.
Based on this observation, we propose to combine prompt-tuning and gradient projection for further anti-forgetting. This combination enjoys: i) prompt-tuning with the instance-wise query mechanism releases the limitation of task identifier for gradient projection; ii) gradient projection provides the theoretical guarantees against forgetting for prompt-tuning.
In this paper, we propose novel prompt gradient projection (PGP) for continual learning. We recall the pipeline of prompt-based continual learning (prompt-tuning) and deduce the orthogo- nal condition of anti-forgetting for prompt gradient via the self-attention mechanism in vision- transformer. We solve the condition equations by conducting Singular Value Decomposition (SVD) on an element- wise sum space between input space and prompt space. That allows us to obtain the gradient projection matrix in an efficient way.
We validate our approach in four benchmark datasets: CIFAR-100, ImageNet-R, TinyImageNet, and CUB200, with three baselines of L2P (Wang et al. 2022c). DualPrompt (Wang et al.) 2022b), and CLIP (Radford et al. 2021), where extraordinary anti- forgetorggt property is observed shown in Figure ! We are the first to explicitly provide forgettngg mechanism for prompt-based continual leamning, and hope our study will further inspire follow-up works. Our contributions are:
(1) Prompt gradient projection is the first work to study the anti-forgetting mechanism of prompt- tuning. Our approach obtains the orthogonal condition of anrg-forg for prompt gradient and hence the retention of old knowledge has a rigorous theoretical guarantee.
(2) We provide new viewpoint about stability and plasticity by investigating the selection of prompt gradient projection matrix. It appears that the essence gradient projection is actually a trade-off, where the optimal solution is updating prompt in the orthogonal space of previous tasks.
(3) We apply our approach both prompt-tuning and prefix-tuning paradigms and show the ef- fectiveness of our approach. Our approach achieves state-of-the-art results terms of forgetting metric and average accuracy metric, under the settings of class incremental learning, online class incremental learning, and task incremental learning.
2 RELATED WORKS AND PRELIMINARIES
Continual learning is defined as training deep neural networks (DNN) on time-variant data, i.e., a
sequence of tasks, marked as D {DI,.., Dr where i-th task D, {(X, (xx: v= contains tuples of input sample X E X and corresponding label yf E Vr. When a task X, arrives, a model fo would be trained for the current task, while the data from previous tasks is unreachable. In this work, we mainly focus on class incremental learning, without knowing the task identifier during inference.
2

---

Published as a conference paper at ICLR 2024
2.1 PROMPT-BASED CLASS INCREMENTAL LEARNING
A simple yet effective prompt-based (prompt-tuning) CIL model: Learning to Prompt (L2P) (Wang jet al., (022c) is first proposed. In it, prompt p. tiny set of trainable tokens, combined with image features, are sent into traioo-rransfoomerr, instructing the model to resist forgetting. To pick appropri- ate prompts for task-specific training, L2P deployed a prompt pool P Pincluding plenty of prompt-key j=1 where K2 represents the j-th key and M is the total number of prompt-key pairs.
pairs, {py, kg 1M
Based on L2P, DualPrompt (Wang et al. 2022b) divided the prompts into two parts: expert prompt and general prompt for distinct features learning. DualPrompt also replaced prompt-tuning with prefix-tuning, which has been successfully proven in the area of NLP. DyTox (Douillard et al. (022) designed novel task attention block, which utilized the task tokens to infer task identifier. Coda- Prompt (Smith et al., 2023) replaced the prompt pool with a decomposed prompt that consists of a weighted sum of learnable prompt components, allowing itself optimized in end-to-en fashion with high plasticity. LGCL (Khan et al. 2023) introduced the text information into the learning of prompt pool, improving performance without any additional learnable parameters.
performance, forgetting still exists compared with other incremental approaches (Saha et al. 2021). Since the problem of forgetting is not explic- itly modeled in this framework, its mechanism against forgetting has not been revealed yet.
Although prompt-based CIL shows
2.2 BACKGROUND OF GRADIENT PROJECTION METHOD
Gradient limitation, i.e., restricting the gradient direction, originated from mathematical theory, pro- vides an important explanation of the stability-plasticity dilemma (Kirkpatrick et al., 2017 Serra
ct al. 2018 Chaudhry et al. 2018 Farajtabar et al., 2020: Zeng et al. [201]] Wang et al. [02
Recent studies found that learning would not forget if the gradient is updated in the orthogonal di- rection of the subspace spanned by the old features. Gradient projection method (GPM) (Saha et al.)
2021] updated the weights in the direction orthogonal to the subspace spanned by all previously learned inputs. This ensured that new learning processes did not interfere with previously learned tasks. Trust Region Gradient Projection (TRGP) (Lin et al. 2022b) selected old tasks in the trust re- gion to learn new tasks by a layer-wise scaling matrix, together with orthogonal gradient projection. Simple Linear Connector (Connector) (Lin al. 2022a) merged two models by using a weighted sum function where one model is updated normally and another is updated with gradient projection.
To further illustrate the anti-forgetting reason of gradient projection, we denote the inputs of task t for layer as St. the learned model for task t as (W:) and L is the total number of layers. In the subsequent sections, after learning task t + 1. If the update direction is orthogonal to the old features, it follows that AW and i St, where the index "t, i" means the i-th input image of task / (Saha al., 2021 Lin et al. 2022b). Therefore, as the model W4+1 is updated as W++1 = W AW, validating the performance of model W141 on task t, we have:
omit layer L for simplicity. Let AW, denote the model change
W = (W, + AW,) Wiz,, + AW = W,
(I)
which indicates that no interference introduced to old tasks after learning new concept, thereby addressing the forgetting issue.
However, one limitation of gradient projection methods that fail in the class-incremental inference is that the projected gradient needs task identifier to find relevant update parameters. In this paper, we will illustrate: prompt-tuning can break the constraints of needing task identifier in gradient pro- jection and therefore the combination of prompt and gradient projection shows advanced properties in class incremental learning.
METHOD
The flowchart of our method shown in Figure 2 In Figure 2 aa), prompts are chosen according to the similarity between the key vector and the query feature. The picked prompts are then
In L,P, the key vector is initialized randomly in the form of one-dimension vector and trained to match the query feature of the corresponding task.
3

---

Published as a conference paper at ICLR 2024
(a)
V
2.0
Store& Store&Update
Vz,
V1.
Projection Matrix
Classilier PER Model
Training Part
Condition
Prompt Gradient
xáp"
Ap
Ap'
pAp' 0
Prompt Gradient Projection
Prommt-KKyy
Task
Pair
%
Query Function
Match
Prompt Pool
(b)
Prumpt Gradient Prujection
Ap
Ap"
Ap
SVD
ApVV?
X
Input Space
Prompt Spuce
Sum Spuce
U.D
F
Ven
@ Element-Wii Wise Sum Forward Propagation
Backward Propagution
Gradient of Projected Gradeiut of
Figure 2:
Flowchart of our work, (a) Process of forward/backwarr (black/red line). An instance- wise query mechanism is adopted during forward propagation. In the backward propagation, PGP enabled and utilized to update the chosen prompts.
(b) Process of prompt gradient projection. We sum input space and prompt space obtain the space. Then with SVD, we attain the new orthogonal vectors from the sum space and update the projection matrix, which is described in Appendix C Finally, we project the gradient by multiplying with the projection matrix.
concatenated with visual embedding sequences for prediction. During the backward propagation, modify the prompt gradient by meeting the requirement of orthogonal condition from gradient projection methods. Figure 2bb shows process of prompt gradient projection. element-wise sum to obtain the so-called sum space. With SVD in this space, we obtain the gradient projection matrix, and modify the gradient with this projection matrix to finish the PGP process.
use the
3.1 PROMPT GRADIENT PROJECTION
From the perspective that the old inputs from previous tasks have the same outputs after learning a new task, we have the following proposition:
proposition 1 To better preserve old knowledge, the update of network would satisfy the following equation:
11) fo(p:,
where 2, denotes the feature embeddings from old task t, py and P++1 denote the prompts trained ! depicts the mathematical, ideal condition of
at task t and 1, respectively. Proposition previous gradient projection works (Saha et al., 2021] Wang et al.] 2021). could be achieved by (i) limiting gradient direction minimal interference with old knowledge; (ii) projecting gradient onto orthogonal space of old inputs. But as a byproduct, both require the task identifier, an additional prerequisite for inference.
anti-forgetting.
In order to realize proposition Jnnimiteddyy we start from the implementation of prompt-based con- tinual learning (PCL). In this framework, after the training of task t 1, we concatenate the prompts Po+1 and the embedding sequences Iı, i.e., inputs from f-th task, along the embedding dimension: Z
P++1] x;
With the weights of Wg, Wt. W., PCL adopts the transformer architecture that
allows us to obtain query (Q[+1 = W2 _ and key (K!+) W.Zt+1). Thus the attention matrix Dosovitskiy et al. 2020) is calculated as:
Q1+1 Q RT-1T
4 [+ softmax
(3)
(4)
4

---

this condition is also Published as a conference paper at ICLR 2024
Here, the denominator denotes normalized factor and hence our focus turns to the numerator part K It actually can be further expanded as W W 2z++1* WT. Notice that W3 and Wk. the weights of visual encoder, are frozen and unchanged during training. The trainable parameters can be denoted as:
2 2
[p+1 3t
[p
xf'
P Petipf+1
P+11 F
(4)
x:P++1
By contrast, the old embedding Zi, is obtained through the concatenating prompts trained at task t and embedding sequences zz:
74 7
Pt
Pt
P.P P I.Pi xæ T
T
(5)
To achieve Eq. 2 i.e. the condition of anti- fotgefooggtting, the new prompts require to be:
Pet P441it+1 p(pl
2.Pt+1 xiPi
(6)
P411 = pixt
In E) we divide 1++1 into Pt and Ap. where Ap is the gradient of prompts when training task
1 Therefore, for the first term, we extend pt4 as:
Pt+1P
(p: (p) + Ap) = p.p. +p: + Appl + ApAp".
(7)
Here we ignore the high-order infinitesimal term of ApAp". Thus if pt p.Ap" = O, the condition, i.e. Pt+ Pt+1P441 p.P, can be realized. In the same way, the second condition can be transformed to:
xxpit x. (pi + Ap") x.Ph. +x,Ap" x.pl.
Eliminating zapi on both sides, we have x, Ap" = O. Note that this condition also satisfies the third term in Eq.(6) because xtp7+1 1 is the transpose of Pc-1 For deduced, and included in Appendix
Therefore, our key observation is reached: restricting the gradient of prompts by the following equations can realize anti-forg:::ing:
x.App" 0,
p.App 0.
To solve this equation, we decompose with SVD: = U.E.V Here, U, and V, contain singular vectors corresponding to singular values Eg, and diagonal matrix Σ, can be further divided as:
Σ Σ11
(10)
Σ,,0
where 1r,1 denotes the non-zero elements of D, (non-zero singular values) and E,,0 denotes the near-zero elements of Σ, (Deisenroth et al. 2020). Correspondingly, Ve can be divided into two
parts along the column dimension: V V.1. Vo Thus, we have:
Σ,1 0
zz [V.1, Vi,ll = U,
(11)
0
As result, we obtain the following equation:
0
VvVe. = Up
~ O.
(12)
2i,
Let Ap- = ApV.oV 4.0- we can get:
= x, A, = = O.
(13)
Eq.(13) (3) allows us to successfully meet the first requirement in Eq. by taking V.D as the gradient projection matrix. We also have similar conclusion for the second requirement in Eq. fact, to simplify the implementation process of Eq. D. we combine pt and with element- toinent-wise sum:
S₁ 2t Pt-
(14)
Thus we conduct SVD on s, and therefore the obtained projection matrix Vi, can realize stApT 0, which equals to I, Ap = 0 and PpAp" 0. In the training stage, we update the gradient with a projection of Ap' = ApV.
Here omit the factor of learning rating since this simplification wouldn't influence conclusion.
5

---

Published as a conference paper at ICLR 2024
3.2 GRADIENT PROJECTION FOR PROMPT POOL
For prompt pool, there is another learnable parameter: key. Prompt-based continual learning often deploys a query-key pair for seeking the matched prompts. In this case, the old knowledge would also be interfered with, as the update key would influence this matching process. Fortunately, the gradient projection method would be generalized well in such case.
First of all, let recall the pipeline of PCL. To choose the relevant prompts, we first calculate cosine similarity between query feature and key:
Th
Φgg, k)
(15)
|a9l| ||
where k represent query feature and key respectively. To achieve untifforgg in the scenario of prompt pool, we have the following proposition:
proposition 2 Old knowledge could be preserved, if the following equation holds:
of ka+1 o kr.
(16)
For further illustration, expand k441 with Ky and Ak; where Ak is the gradient change from task t to task + 1, and have:
of of K1-1 ot (ke Ak) of ke + ot Ak.
(17)
The above formulations suggest of Ak O, which is similar to Eq. 9). In fact, notice that, qf AA = is slightly different from Eq. 9). that uses the transposition form. Therefore in our implementation, when sampling the orthogonal space of query features, we need first to transpose this feature matrix, like of = V.EUT?
3.3 BALANCE BETWEEN STABILITY AND PLASTICITY
We consider singular value decomposition (SVD) of s; as 8, Ú,E. V.T. V. consists of singular vectors decomposed of st. Here, we define a threshold, [o, 1] to split V. into two parts Vi = [V:1 1.2), where V,2 = Vi[: 1 en], V?2 = Vil;,en en n] and n is the column size of V.. We project the gradient 9++1 as:
9++1 g 1.222T.
(18)
Therefore, there are three situations for V?2 Firstly, if V?2 = 0, we have:
g++1 = O.
(19)
In this situation, all trainable parameters are frozen and the old knowledge will be preserved com- pletely. Secondly, if V?2 Ve,s, we have
9++1 9h+1 V..VV.0
(20)
In this situation, we update prompts by projecting gradient onto the orthogonal space of old inputs. Hence, samples from old tasks can have the same outputs for the new model. The old knowledge will be preserved well and the model can also learn some knowledge with updating. Thirdly, if V22 V,, we have:
g++1 St+ D, = 9t+1 V.V. 9t+1-
(21)
In this situation, the parameters updated normally without projection. In our implementation, rearrange V, sorted according the corresponding singular values. We control this balance. More detailed discussion could be seen in Appendix B
Here Vro obtained by decomposition of 8, mentioned above.

---

Published as a conference paper at ICLR 2024
Table 1: Main results of class incremental learning in terms of accuracy and forgetting on 10-Split- CIFAR100, 10-SSlit-CIFARR 100, and 10-Split- Ipplit-ImagNee-RR Exemplar means the total buffer size for rehearsal methods. For detailed metrics information please refer to Appendix
10-Split-CEAAR000
20-Split-CFAARO00
D-SSpii-ImaaeNet-R
Method
Exemplar
ACC(t)
Forgetiigg()
ACC(1)
Forgetting())
ACC(t)
Forgetting()
BiC
5000
81.42
17.31
73.02
6,23
64.63
22.25
DER++
5000
83.94
14.55
66.73
20.67
ICaRL
5000
66.00
5.33
78.02
5.80
DER+MOG
2000
67.62
14.64
65.84
13.72
BiC
1000
66. 11
35.24
63,12
21.89
$2.14
36,70
DER++
1000
61.06
39.87
55.47
34.64
ICaRL
1000
61.25
14.19
71.32
15.98
FT
X
33.61
86.87
33.52
53.69
28.87
63.80
EWC
X
47.01
33.27
36.73
35.19
35.00
16.16
60.69
27.77
39.12
57.91
38.54
52.37
L2P
83.77
6.63
81.29
8.96
60.44
L2P-PGP(Ours)
X
84.34
5,59
82.00
8.39
61.40
8,03
DualPrompt
X
86.50
5.77
82.98
8.20
68.13
4.68
WealPrompt-PP((Our
X
86,92
5.35
83.74
7.91
69.34
4.53
Upper-Bound
90.85
90.85
79.13
EXPERIMENTAL SETUP
Datasets: We evaluate our method on 1) 10/20-SpIi-CIFARR000 (Krizhevsky et al. 2009), con- structed by splitting the 100 classes into 10 tasks/20 tasks. 2) 10-Split-Tinny Imaeeta (Abai & Rajmalwar 2019). constructed by splitting the 200 classes into 10 tasks. 3) 10-Split-ImageNet-P
[Hendrycks al. 202I). constructed by splitting the 200 classes into 10 tasks.
Implementation: We use L2P (Wang et al. 2022c), DualPrompt (Wang et al. 2022b), and CLIP (Radford et 202T) as our baselines, with prompt gradient projection for updating. We follow their original settings, and the only difference is we train DualPrompt with extra 15 epochs on CIFARI00 suggested by (Khan et al. 2023). Detailed experiment information could be seen Appendix G
Competitors: We compare our results with representative SOTA CIL methods including ICaRL (Rebufti et al., (2017). BiC (Wu et al. 2019), DER++ (Buzzega et 2020). LWF (Li & Hoiem 2017). EWC (Kirkpatrick et al. 2017) DER+MCG (Cai et al. 2023). We adopt average accuracy (simplified as accuracy/ACC) and forgetting (simplified as FOR) as our validation metrics (Wang
al. 2022b). Results and comparisons of task incremental learning can be found in Appendix
5 RESULTS AND DISCUSSION
Class Incremental Setting" FT main results are shown Table We observe that DualPrompt with PGP obtains the best results and achieves a new SOTA. When comparing DualPrompt with DualPrompt-PCH it appears that PGP brings a decent improvement anti-forgetting. On 10-Split-IIAAR 00, PGP improves DualPrompt by 0.42% forgetting and 0.42% accuracy. Similarly, 20-Split-CIAAL Siit-CIFARR000 100, PGP improves DualPrompt by 0.29% on forgetting and 0.76% on accuracy, and on improves DrapPrompt by 0.15% forgetting and 1.21% accuracy.
We compare our method with state-of-the-ar CIL approaches, and the
PGP
For L2P, PGP also brings evident performance improvements. On 10-Split-CIFA CIRIRCIIFRROO,, PGP obtains an improvement of 1.04% forgetting and 0.43% accuracy. On Do-Split-ImageNet.e our method also obtains an improvement of 0.97% on forgetting and 0.96% on accuracy.
Analysis of Training Time and Memory Space: We present the comparison between L.2P-PGP and 1.2P-R (I.2P with rehearsal exemplar) in terms of training time and memory cost in Table[] For fair comparison, we maintain complete consistency in experimental settings such as batch size, training epoch, and prompt length et al.
"Exeermment results of CLIP model please refer Appendix
7

---

Published as a conference paper at ICLR 2024
It is worth noting that our method doesn't 't require any exemplar for rehearsal and therefore not only uses less memory space, but avoids the privacy leaking problem (Shokri & Shmatikov 2015 as well. At the same time, approach has a lower forgetting and shorter training time.
Table Comparison of ACC, forgetting, memory, and training time between L.2P-GGP and L.2P-R.
Method 1.2P-R
Exemplar 1000
ACC(t)
Forgetring())
Memory
Training Time
84,21
7.72
1.12 GB
0.787h 0.756 h
L2P-PGP
84.26
5.64
$ MB
Online Class Incremental Setting: online class incremental learning a challenging class incre- mental task that only allows training each task for one epoch. We compare PGP and its baseline on this setting shown Table
Table 3:
Main results of online class incremental learning in terms of accuracy and forgetting. The comparison is made between our approach and the corresponding baselines.
10-Spli-CIEARIIM
20-Split- CIFARIO
10-Split Tiny Image
Method
ACC(1)
(orgetting()
ACC(T)
Forgetting())
ACC(1)
Forgetting())
L2P
79.99
8.19
77.63
11.33
78.69
5.83
L.2PPGPP
80.29
7.73
78.34
9.33
79.47
5.19
DualPrompt
80.93
5.51
79.02
6,89
82.20
3.62
DualPrompt-PGr
5.41
79.41
3.57
Accuracy on Splleeiivtmmggeke
Forgetting Splt mageftet
Accuracy on Splt-Tenyteeeeet
Forgetting 10llt Tiyylssggttt
84
Accuracye
380
L2F
L2P
DustPromet
DusiPrompt
DuslPrometPGP
Po
10
Figure 3: Task-by-task performance changing curves in terms of accuracy and forgetting under online class incremental setting.
On 10/20-Split-IFAAA CI0AR and 10-Split-inyylmaeeNet is observed that PGP is able to improve accuracy and reduce forgetting for both I.2P and DualPrompt. On 10-SplitCIFARR1N and 20- Split-CIFAR100, 100, we discover that our method can improve L2P by 0.30% and 0.71% on accuracy respectively, while reducing 0.46% and 2.00% on forgetting. On 10-SplitTTinllmagNee we also find that our method improves L2P by 0.78% on accuracy and 0.64% on forgetting. Similar to L2P, for DualPrompt, we take 20-Split- Split-CIFAR100 dataset an example, PGP brings the method improvement of 0.39% accuracy and 0. 1444 on forgetting.
Figure shows the curves of accuracy and forgetting with the task number increasing on 10-Split- TinylmageNet. We observe that on all tasks, accuracy of our method is always higher than baseline, and forgetting is always lower than baseline. These phenomena demonstrate that our method has advantages over baseline with fewer training epochs.
ABLATION STUDY
Impact of Projection Settings: As shown in Table 4 and Figure we evaluate the results by performing gradient projection on the gradient of prompt (12p-p), key (12p-k) and both of them (12p- pk), respectively. Original I.2P is named "12p-o". Table quantiittttt indicates that 12p-pk has the best anti-forge performance since it has the strictest constraint. Coneretely, compared with 12p- o, 12p-p. 12p-k and 12p-pk decreases the forgetting by 0.99%, 0.76%, and 1.04%, while increasing the accuracy by 0.49% 0.30% and 0.57% on 10-Split-CIAAR 100. On 10- 10-Split-Tinme Split-TinylmageNee 12p- p. 12p-k, and 12p-pk decrease the forgetting by 0.42%, 0.34%, and 0.78%, while increasing the accuracy by 0.65%, 0.37%, and 0.73% in comparison with 12p-o.
We have observed that 12p-p performs better than 12p-k in both terms accuracy and forgetting. The reason might be the prompt directly participates in the image encoding process. Figure also
8

---

Published as a conference paper at ICLR 2024
Table 4: Ablation study of various gradient projection manners. 12p-o, 12p-p, 12p-k, and 12p-pk denote the original L2P, gradient projection on prompt, key, and both prompt and key, respectively.
10-SpiitCIFAAROO CIFARIOO
10-Split- Diempeee-inyymmageNet
Method
Forgetting())
ACC(1)
Forgetting())
ACC(1)
12p-o
6.63
83.77
5.68
81.92
12p-p
5.64
84.26
5.26
82.57
12p-k
5.87
84.07
5.34
82.29
12p-pk
84.34
4.90
82.65
Forgetting for 10 Sppii----IA
Forgetting To-SplitTTnylmageve
Accuracy for 10 10-Split-CiAAR100
Accuracy for 10 1o-SpiitTTylllageeeee
Ope
84
62
Gpot
10
Order
Order
Figure 4: Task-by-task performance changing curves in terms of accuracy and forgetting under
various gradient projection manners.
shows the changes of forgetting values with the task number increasing, where the line of 12p-p. 12p-k, and 12p-pk is always below 12p-o.
Impact of Distinct Threshold: We study the hyperparameter sensitivity by setting e with values in [0.60, 0.70, 0.80J. and conduct experiments on 10-Split-IIFAR100 and (O-Spli--TinllmaeNee datasets respectively, shown F
In Figure we can clearly find that with increasing, forgetting shows clear degradation, indi- cating that the ability of anti-forgetting (stability) becomes stronger, while the accuracy (plasticity) shows a trend of decline. These two phenomena perfectly illustrate that if V/! has fewer columns (low e), the model would have better plasticity but worse stability, and if V.2 has fewer columns (high e), the model would have better stability but worse plasticity. The above conclusions also have been reported in GPM (Saha et al. 2021) and Adam-NSCL. (Wang et al., 2021). Thus, gradient projection also trade-off between plasticity and stability when meeting prompt- continual learning.
Accuracy Forgetting
Accuracy
Forgelting
c=0.8
67
0=0.8
06 91
88
x=0.7-
.8
00.60 69
8=0.7
5.00
c=0.6
g=0.6
Forgetting(%)
Accuracy(%)
Forgetting(%)
Task (y(%)
Figure 5: Performance histograms in terms of forgetting and new task accuracy by varying e.
CONCLUSION
In this paper, we propose prompt gradient projection, which deduces the gradient condition for prompt to reduce forgetting. Then the gradient projection matrix is obtained by conducting SVD on sum space. Finally, we discuss how to balance plasticity and stability from the perspective of gradient projection. We validate our approach in benchmark datasets under various incremental settings and demonstrate the effectiveness of our approach. This paper is an initial attempt towards combining prompt-tuning and gradient projection. We hope work would inspire the focus the anti-forgetting mechanism of prompt-basee continual learning and can be extended to more parameter-efficiet paradigms i.e., adapter-tuning and LoRA-tuning and large models.

---

Published as a conference paper at ICLR 2024
8 ACKNOWLEDGMENT
This work is supported by the National Key Research and Development Program of China (2021ZD01 1000), Science and Technology Commission (No.21511 01221511100700 1007700) National Natu- ral Science Foundation of China (No.62222602, No.62106075, No.62176092, No.62302167, No.62176224, No.61972157 No.U23A20343, No. No.72192821), Natural Science Foundation of Shanghai (23ZR 1420400), Shanghai Sailing Program (23YF1410500) and CAAl-Huawei Mind-
Spore Open Fund.
REFERENCES
Zoheb Abai and Nishad Rajmalwar. Densenet models for tiny imagenet classification. arXiv preprint arXiv:1904. 10429, 2019.
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based on complementary learning system. arXiv preprint arXiv:2201. 2022.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark ex- perience for general continual learning: strong, simple baseline. Advances in neural information processing systems, 33:15920- 15930, 2020.
Tenghao Cai, Zhizhong Zhang, Xin Tan, Yanyun Qu, Guannan Jiang, Chengjie Wang, and Yuan Xie. Multi-centroid task descriptor for dynamic class incremental inference. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7298-7307, 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 9650-9660, 2021.
Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny, Efficient lifelong learning with a-gem. arXiv preprini arXiv: 0812.80122000 2018
Xiangning Chen, Cho-Jui Hsieh, and Boqing Gong. When vision transformers outperform resnets without pre- trainiiiig strong data augmentations. arXiv preprint 1oXiv:2106.0/548 2021.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. continual learning survey: Defying forgetting classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366-333 2021.
Mare Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong. Mathematics for machine learning. Cambridge University Press, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 1929,, 2020.
Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVE Confer- on Computer Vision and Pattern Recognition, pp. 9285-99899999 0295,929,,20 2022.
Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for contin- ual learning. In International Conference on Artificial Intelligence and Statistics, pp. 3762-3773. PMLR, 2020.
Raia Hadsell, Dushyant Rao, Andrei A Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends cognitive sciences, 24(12): 1228-10000 2020.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang. Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, al. The many faces of robustness: A critical analysis of dustrf-distribution generalization. In Proceedings of the IEEE/CVF International Conference Computer Vision pp. 3340,8349, 2021.
10

---

Published as a conference paper at ICLR 2024
Muhammad Gul Zain Ali Khan, Muhammad Ferjad Naeem, Luc Van Gool, Didier Stricker, Fed- erico Tombari, and Muhammad Zeshan Afzal. Introducing language guidance in prompt-based continual learning, arXiv preprint arXiv:23085 15827, 2023.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabssa-Barwwinss et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of 2009.
features from tiny images.
Dharshan Kumaran, Demis Hassabis, and James I. McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in cognitive sciences, 20 (7):512-534, 2016.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint 2 104.08691, 2021.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint
2021.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence. 40(12):2935-2947) 2017.
Zhuowei Li, Long Zhao, Zizhao Zhang, Han Zhang, Di Liu, Ting Liu, and Dimitris N Metaxas.
Steering prototype with prompt-tuning for rehearsal-free continual learning. arXiv preprint arXiv:2303.09447, 2023.
Guoliang Lin, Hanlu Chu, and Hanjiang Lai. Towards better plasticity-stability trade-off in incre- mental learning: A simple linear connector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 89-98. 2022a.
Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Trgp: Trust region gradient projection for continual learning. arXiv preprint arXiv:2202.029.) 2022b.
James L McClelland, Bruce L McNaughton, and Randall C O'RReilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PMLR, 2021.
Sylvestre-Alviss Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampeer. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Mark B Ring. Child: A first step towards continual learning. Machine Learning, 28:77-104, 1997.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. arXiv preprint 1tX.i:2103309762.2 2021.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning, pp. 4548-4557. PMLR, 2018
Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pp. 1310-1321,200
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante- Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual de- composed attention based prompting for rehearsal free continual learning. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 1 11909-11911 11919, 2023.
11

---

Published as a conference paper at ICLR 2024
Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv:1904.07334, 2019.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 184-194 194,193.7 2021.
Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers:
An occam's 's razor for domain incremental learning. Advances in Neural Information Processing Systems, 35:5682-5663 2022a.
Zifeng Wang. Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen- Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, pp. 631-648. Springer, 2022b.
Zifeng Wang. Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vin- cent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Pro- ceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 139-149, 2022c.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 374-382, 2019.
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent pro- cessing in neural networks. Nature Machine Intelligence, 1(8):344-32, 2019.
Zhen Zhao, Zhizhong Zhang, Xin Tan, Jun Liu, Yanyun Qu Yuan Xie, and Lizhuang Ma. Rethinking gradient projection continual learning: Stability/plasticity feature space decoupling. Proceed- ings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 3718-3727, 2023.
Da- Wei Zhou, Yuanhan Zhang, Jingyi Ning, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Learning without forgetting for vision-lange languaee models. arXiv preprint arXiv:2305.192770 2023.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision, 13009))2337-2348, 2022.
12

---

Published as a conference paper at ICLR 2024
A APPENDIX
B PROOF OF INSIGHT WITH BALANCING STABILITY AND PLASTICITY
Lemma Singular value decomposition (SVD): For matrix Am, we can factorize it into three matrices and obtain matrix Um,t, Σn,, and Vi, where Um and Ví, orthogonal matrices, Σn, contains the sorted singular value along its main diagonal:
Am, Un,.E., T
(22)
Theorem For any embedding sequences It, embedded from samples of task t, using singular value decomposition (SVD) in lemmal] !
we can obtain matrix Us Et, and V.. Then, we randomly split V, along the column dimension into two parts: [V:, V?]. Because is an orthogonal matrix, we can have:
VIT V ViV,T V2V?T -1
v/V=" = [V!
(23)
Ver
On Plasticity: For any matrix V and gradient g++1 when learning task + 1, we have the gradient after projection on as 9241s and calculate the cosine similarity between 96+11 and 9:+1
< 11411990 9+41 > >=< S4+1VVT -9++1 >
vec(gi+1 VV 1)" vec((())))
vec(ar+1 1V) V)vec(ge+1)
(24)
veo(g++VV) vec 2 O.
which means that whatever V is, as if 9++1 and V both are not equal to zero, after projection on V. the new gradient is always positive. Thus, the model is always learning new knowledge. The only distinction is the gradient direction, which can be measured by calculation of cosine similarity between gradient before projection and after projection. We set gradient project matrix as V,2 and have:
9441-94+1 =< 91+1 9<14 V.V.V. 99+1>
=< V,' 1,90+1 >
=< 9641 - 94+1 VA 19 >
(25)
=< 9041,994111111 > - < 94+1 94tiV. V. 99-1
<< 9141,9++1 >
From the inequality, if projection matrix V22 is not an identity matrix, which means that matrix V/2 a zero matrix, the direction of gradient after projection always has angle with the direction of the original gradient, incurring that decrease of loss function becomes slow.
For further research on the relationship between matrix Vil and decreased speed of loss function, we mainly focus on two situations for value of V.!:
(1).V/ have
9t+1 G++1 >=< 9++1,
(26)
In this situation, it equals that do not operate the gradient, and parameters are updated nor- mally. When V.1 = 0, it has V22 = V.. In fact, the same phenomenon has been shown in the previous situation that 2 Vi.
(2).V/ = Vr, we have:
g D141 =< It+1, M++1 > -< I4+1 V.V.T I1+1 >
St+ 1 1+41 - -1
(27)
= O.
13

---

Published as a conference paper at ICLR 2024
In this situation, it equals that freeze the network update process, and trainable parameters stable and not changed. Thus, the network will not adopt any other new knowledge. When V, Vi, it has V22 O. In fact, the same phenomenon has been shown in the previous situation that V22 = 0.
In conclusion, with V changing from 0 to V,, the decreased speed of the loss function becomes more and more slow, leading to worse plasticity. However, under this trend, V?2 is changing from V, to O, giving the anti-forgetting more and more strength. We can recognize that the essence of the gradient projection method is kind of trade-off strategy between plasticity and stability. However, different from other dilemmas, it has an optimal solution, which is projecting gradient in the direction orthogonal the subspace spanned by the old inputs, which can not only own the best ability of anti-forgetting, but also have minimal damage to plasticity.
C METHOD OF UPDATING PROJECTION MATRIX
We update our projection matrix Vi, like GPM Saha et al. 2021). which is detailed described as follows. Assume that we have sampled embedding sequences from current task samples 2, and trained prompts Pt. Here, is the task identifier. We utilize Principal Component Analysis (PCA) to compress and align the dimensions of 24 and Pt- Then, element-wise add 2y and pt to obtain 8t- Besides that, we set threshold e.
For task #1 training, perform SVD 81 81 VI.. We collect the minimum former / columns V1 as matrix = [111, V12, vul according to the following criteria:
| 2 e//
(28)
Here, ||-I|r j is the Frobenius norm of the matrix and e (0 < <c11) 1) is the threshold hyperparameter. Vi, can be obtained by Vi,0V = LLT.
For task #2 training, before performing SVD and subsequent former-rank approximation, we elim- inate the common directions in $2 which are already present in L, so that newly added column vectors are unique and orthogonal to the existing column vectors. Thus, we perform the step 82 LL" Afterward, SVD is performed on 82(= V2") and former m new columns of V2 are chosen with minimum value of m satisfying the following criteria:
IL" 22 + || 1 2 2|
(29)
Here, L is updated by adding new column vectors as [1]1. U12 ... U1, 821 622, ... 02m]. Then, we can update V1,0 V220 according 1220 V2,00 1 LL". Once the update complete we move on to the next task and repeat the same procedure as in task #2.
D COMPUTATION COST FOR MAINTAINING THE ORTHOGONALITY OF THE TASK SUBSPACES
In this section, we will discuss the added computation cost for maintaining the orthogonality of the task subspaces under the following situations.
D.1 LARGER MODELS
If we change the backbone from a smaller one to a larger one, it could have different results of added computation cost for distinct tuning paradigms. i) For prompt-tuning, because we only prepend the prompt into the first transformer layer, the added computation could be omitted. ii) For prefix- tuning, larger models usually mean more network layers or wider input dimensions, and we need to expand the prefix-inserted layer or prefix width, which is the origination of the added computation cost. For expanding the prefix-inserted layer, each layer can have a nearly similar computation cost if the number of samples is the same. Thus, modeled as an approximate linear function with the layer numbers of the backbone. Similarly, the same conclusion can also be drawn from expanding the prefix width.
conclude that the added computation cost can be
14

---

Published as a conference paper at ICLR 2024
D.2 INCREASED NUMBER OF TASKS
Observing the training processes of multi-datasets the number newly added column vectors the projection matrix is constant in a certain range. As the added computation cost is mainly focused i) calculation of the projection matrix and ii) multiplication between the projection matrix and its transpose, we can see that although it could not appear exponential explosion, it still a potential risk in our method with the increased number of tasks.
empirically summarize that in each task,
E GRADIENT PROJECTION BASED ON PREFIX- TUNNING PARADIGM
In this section, we prove that the gradient projection method can be utilized prefix-tuning with mathematical deduction.
Distinct from prompt-tuning paradigm, prefix-tuning only prepends prefixes in key vector and value vector, without query vector of prepended transformer layer. Additionally, different from prompt usually only prepended in the first transformer layer, prefix can be prepended in any transformer layers. These advantages help models based prefix-tuning own a better performance than those based on prompt-tuning both in natural language processing and computer vision.
For baseline based on prefix-tuning if we want to preserve old knowledge, we need realize:
fo (PIr, = -o (P.+11.,,7..)
(30)
fo refers ViT model, T2, denotes inputs at task in layer Pt. and 1.+ represents the prefixes trained at task t and prepended in layer and the prefixes trained at task t + and prepended in layer 1 respectively.
Assuming that a set of prefixes have been trained at task t + 1, and we input samples from task t. Now, we prepend prefix key vector, and have:
Qt.1 W W,t.
(31)
Pit1d Wki1
K2,1 =
(32)
where, Wad and Wa.1 are weights of ViT, frozen and unchanged. With Eq. we have the results that 1-th task samples on + 1-th model. We mainly focus on the part:
QucKT W 1a (PT+1 (Wa,z(,)7 [Wa. (W
(33)
As stable item W. Wo, WT,, k. only focus the item W Wq Changing Pl... with Pe. we can obtain the results that t-th task samples on t-th model. Because our aim making W Wa equal to W W',', considering that Wa, is frozen, our final aim can be simplified as:
3T.PP IT.PT..
(34)
which has the same form Eq. meaning that can also achieve Eq.34) by the gradient pro- jection method. Thus, we can draw the conclusion that the gradient projection method could also help models based prefix-tuning to resist forgetting.
15

---

5 and the prompt-inserted locations are Published as a conference paper at ICLR 2024
F METRICS
Two metrics: Average Accuracy (simplified as accuracy/ACCC) and Forgetting (simplified as FOR) used to evaluate the performance. We use average accuracy metric, for averaging the classifica- tion accuracy of all classes. We adopt forgetting metric to indicate the average loss of aceuracy of past tasks after learning task. Formally, average aceuracy and forgetting are defined as:
Average Accuracy = T AT
(35)
Forgetting T Ar. max
Σ
(36)
where is the number of tasks, Ar, the accuracy of i-th task samples on the T-th model, and A the accuracy of i-th task samples on the j-th model.
G EXPERIMENTAL DETAILS
Consistent with previous works (Wang et al. 2022c by Smith et al. [223)) we use VIT B/16 (Doso- vitskiy et al. 2020) pre-trained on ImageNet-216 as our image encoder, which is kept frozen during training. We train and test
A6000-48GB GPU for baselines and our method. We set the Adam optimizer with 31 = 0.9 and 32 = 0.999.
For hyperparameters. in L2P-PGP, we set 0.50 for extraction of prompt gradient projection matrix and c 0.97 for key gradient projection matrix. While in DualPrompt-PGr we set e 0.50 for extraction of prompt gradient projection matrix. To accelerate the speed of gradient projection matrix extraction and reduce the training space, we add PCA into our process, which can be used to compress the sampled feature space.
In comparison with L2P and L2P-PGP, for 10/20-Split--e Sii---pll-CIRROOOO and 10-Split-Tiny TinylmageNet, we both train the network for 5 epochs with batch size of 16 and prompt length is set 5, while both set epochs as 50, batch size as 16, and prompt length as 30 for
In comparison with Prompt and DualPrompt-PGP, for S020-Split-CIFARII work for 20 epochs with batch size of 24, and expert prompt length is set at 5. While we both set epochs 5, batch size as 24, and expert prompt length for with expert prompt length at 20. Besides that, in
train the net-
epochs as
50 and batch size as 24 for all benchmark datasets, the general prompt length kept the same.
For CLIP-PGP, the experimental setting is that, on the vision side, we only set single trainable image prompt shared by all tasks. As for the text side, we follow the operation (Zhou et al.) 2022], we set trainable text prompt for each class, which only trained at the corresponding task. In comparison with CLIP and CLIP-PGP, we both set the image prompt length 5, epochs 5, and batch size as 32 for 10-SplitCIIFA 100. Specifically CLIP-PGP, we set e = 0.90 for extraction of image prompt gradient projection matrix.
H RESULT TABLE WITH THE STANDARD DEVIATION VALUES
We conduct 3 runs of our method and competitors, additional results with the standard deviation values on different datasets are shown in Table 5
I COMPARISON WITH BASELINES AND SPPER-BOUND
We compare the performance of prompt-based methods with and without PGP in Table 6 To be consistent with previous works (Wang et al. 2022c), we report the difference between accuracy performance of the Upper-Bound and the model metric. We observe that PGP again sets SOTA in this setting. As we compare the Diff performance of DualPrompt and L2P with and without PGP, again notice obvious improvement.
16

---

Published as a conference paper at ICLR 2024
Table Class incremental learning on different datasets along with the standard deviation values.
10-SptitCIEARI00
20-Spii-CIFARI00
10-Sphit-ImageNNe
Method
Exemplar
ACC(1)
Forgetiing()
ACC(1)
Forgettin(())
ACC(T)
Forgetingl
BiC
5000
81.42-0.85
17.31+1.02
73.02+0.93
6,23+11177
64.63+1.27 66.7340.87
22.25+1.77
DER++
83.94=0.34
14.55=0.75
20.67=1.24
ICaRI.
5000
66.00 0.66
5.33-0.94
78.02+0.71
5.80++1002
DER-MCG
2000
67.62=0.04
14.64:0.53
65.8420.18
13.72#1.28
BiC
1000
6.76
35.24.1 1.64
63.12.1..35
1..89.1.93
52.14.1..08
36.701...05
DER+
1000
61.06-0.87
39.87:10.99
55.47111.11
34.6411.50
ICaRL
1000
61.25500.33 0.63
14.19.1.14
71.32.10.86
15.9
FT
33.61=0.85
86.87=0.20
33.52=00.44
53.69:#0,52
28.87
63 1.80==1.0 .000
FWC
47.01-0.23
1..27-+1.17
36.73+0.55
35.19--1.96
35.00++0.33
56 16+0.88
LWF
60.69 0.63
27.77.1.117
39.12.10.87
57.91
38.54.1
52.37 10.64
L2P
83.77=0.16
6.63-0.05
81.29=00.3
8.9620.38
60,44+0.41
9,00++.86
L2P-PGP(OUrs)
84.34=0.08
5,59=0.05
82.00=1.566
8.39
61.40:10.34
8.03:10.03
DualPrompt
86.50=0:45
5,77=0002
82.98=0.47
8.20=0008
68. 13+00.10
1.68+0.119
DualPrompt-PGP((Orr))
86.92 0.05
5.35 0.19
03.74-00.01
7.91-00.15
69.34-+0.05
4.53++0.44
Upper-Bound
90.8520.12
79.13:0018
Table 6: Comparison with baselines in terms of differences between accuracy performance of the Upper-Bound and the model. The Upper-Bound denotes the model performance when trained with access to all tasks at the same time. Diff Upper-Bound ACC Method ACC.
CI-AR00
20-SpII-CIFARI000
10-Split-ImageNet-R
Method
ACC(1)
Dif((L)
ACC(t)
Dif(()
ACC())
Diff(L)
Upper-Boue Bound
90.85
90.85
79.13
L2P
83.77
7.08
81.29
9.56
60.44
18.69
T.2P-PGP
84.34
6.51
82.00
8.85
61.40
17.73
DualPrompt
86.50
4.35
82.98
7.87
68.13
11.00
DualPrompt-PGr
86.92
3.93
83.74
7.11
69.34
9.79
J TASK INCREMENTAL SETTING
We compare L2P-PGP with L2P and representative SOTA competitors: EWC (Kirkpatrick et al.
2017), LWF (Li & Hoiem 2017), A-GEM (Chaudhry al. 2018). OWM (Zeng al. 2019)
Adam-NSCI.
Wang et al. 2021), Connector (Lin et al. 2022a), results as shown in Table
Both 10-Split-CIFAR 100 and 20-Split-CIFAR 100 datasets, although L.PP has already achieved higher accuracy and lower forgetting compared with other CNN methods, our method further im- proves its accuracy and reduces its forgetting with the aid of prompt gradient projection and L2P- PGP achieves new SOTA performance. On S.-Split-CIFARR 100 dataset, PGP improves L2P by 0.10 on accuracy, 0.05 on forgetting, and on 20-SplitCCFARROO PGP improves L2P by 0.11 on accuracy, 0.11 1 forgetting.
Table 7: Task incremental learning results on different datasets.
20-Sp0it-CIFAR100
Method
ACC(t)
Forgetting(4)
ACC(t)
Forgetting())
EWC
70.77
2,83
71.66
3.72
LWF
70.70
6,27
74.38
9.11
A-GEM
49.57
1.13
61.91
6.88
OWM
68.89
1.88
68.47
3.37
Adam-NSCL
73.77
1.60
75.95
3.66
Connector
79.79
0.92
80.80
5,00
L2P
97.43
0.22
98.47
0.39
L2P-PGP
97.53
0.17
98.58
0.28
17

---

Published as a conference paper at ICLR 2024
K CONTINUAL LEARNING RESULTS ON MULTI-MODEL BACKBONE, COMPARISON BETWEEN CLIP-PGP WITH CLIP
We conduct our experiments on D0-SplitCCIFARIO0 dataset under class incremental setting and task incremental setting respectively, as shown in Table 8 Results show that, our method has improved the performance a for both the above settings, proving that our method is also useful in the vision-language models, which further enlarges the scope of our method.
Table 8: Comparison to CLIP model with/without gradient projection method on 10-Split- CIFARI00 with class/task incremental settings.
Settings
Class Incremental
Task Incremental
Models
Accuracy
Forgetting
Accuracy
Forgetting
CLIP
73.76
5.60
92.69
2.34
CLIP-PGP(Ours)
79.47(+5.71)
4.23(-1.37)
93.00(+0.31)
1.58
L CLASS INCREMENTAL LEARNING RESULTS ON DIFFERENT BACKBONES, COMPARISON BETWEEN OURS WITH BASELINES
To show the efficacy of proposed method on different pre-trained backbones, we evaluate our method by extending two distinct pre-trained models, namely ViT-DINO and ViT-SAM (Caron al. 2021 Chen et al. (021). The results are shown in the Table 9 Additionally, we tested our method on Cu-Split-CI-ARTOO and 5-Split-CUB200 dataset based on three pre-trained ViTs: ImageNet-21K, DINO, and SAM, further validating the effectiveness of our method on non- Ion-eNt datasets (Wah
ct al. 2011 Krizhevsky et al. 2009).
Table Comparison distinct pre-trained backbones between baselines and ours. Red parts show significant improvements (>1).
10-Split-CIFAR1000
5-Split-CUBBl
Method
Wwetrained-Datasee
ACC(1)
Forgetting(4)
ACC(t)
Forgetting(L)
1.2P
ImageNet-21K
83.77
6.63
74.88
5.39
1.2P-PGP
ImageNet-2IK
84.34(+0.57)
(-594)
75.15(+0.27)
4.51(-0.88)
DualPrompt
ImageNet-21K
86.50
5.77
82.02
4.23
DusPPrompt-PGP
ImageNet-2IK
86.92(+0.42)
5.35(-0.42)
82.46(+0.44)
3.76(-0.47)
1.2P
SAM
83.93
6.68
73.98
6.77
1.2P-PGP
SAM
84.26(+0.33)
5.64(-1.04)
76.45(+2.47)
5.91(-0.86)
DualPrompt
SAM
86.11
6.08
82.02
4.73
DuaPPrompt-PGP
SAM
86.92(+0.81)
5.04(-1.04)
82.28(
4.65(-0.08)
L2P
DINO
67.35
9.69
10
9,77
L2P-PGP
DINO
70.60(+3.25)
4.73(-4.96)
44.80(+0.70)
6.06(-3.71)
DualPrompt
DINO
64.18
23.87
50.88
10.10
DuaPPrompt-PGP
DINO
73.33(+9.15)
(0.27(-13.60)
51.03(+0.15)
9.06(-1.04)
M PGP WITH PROMPT NUMBER AND PROMPT WIDTH
In this section, for L.2P-PGP model, we set distinct parameters in prompt numbers and prompt widths on 10-Split-IIFARI0 100 dataset, and further validate the efficiency of prompt gradient projection method. Results are shown in Table 10 In our setting, we set single prompt mode, that all tasks share single prompt for training. We think, in this way, method and avoid interference caused by choosing prompts. Results show that, models with prompt gradient projection, all have higher accuracy and lower forgetting than those without, which proves that our method could be effective in distinct prompt numbers and widths, even with a hard single prompt setting.
deeply uncover the potential of our
18

---

Published as a conference paper at ICLR 2024
Table 10: Comparison L2P with L2P-PGP on 10-Spli-CIFARI00 dataset. Width and number mean prompt width and prompt number respectively.
L2P
L2P-PGP
L2P
L2P-PGP
Width
ACC(T)
(OR(L)
ACC(t)
(LR(L)
Number
ACC(1)
FOR(L)
ACC(f)
FOR(L)
5
82.64
6.73
82.77
6.58
82.64
6.73
82.77
6.58
10
82.09
7.07
82.16
6.74
84.17
5.92
84.19
5.60
15
83.09
6.38
84.21
5.62
83.23
6.66
83.82
6.62
20
83.42
6.38
83.87
5.89
83.87
7.13
84.44
6,58
25
83.69
6.49
83.85
6.39
84.11
6.60
84.15
6.52
30
83.87
6,46
84.39
6.44
N PGP WITH PREFIX WIDTH AND PREFIX PREPENDED LAYER
In this section, for DualPrompt-PPi PGP model, we discuss whether prompt gradient projection could be efficient in different prefix widths and prepended layers. As the setting in Appendix M we choose single prefix mode based the same reason. We conduct experiments 10-Split- Meeii--iinImmnnn ImageNee,e Final results are shown in Table [1 and Table We also show some cases with curves of accuracy and forgetting metrics changing all tasks, Figure and Figure 7
10-Split-CIFAR 100 and
Comparison DualPrompt with DualPrompt- on 10-Split- (O-Split-CIFARI0 dataset. Width and layer mean prefix width and prefix prepended layer index respectively.
Table 11:
DualPrompt
DualPrompt-PPi
DualPrompt
DualPrompt-PGP
Width
ACC(t)
FOR(L)
ACC(t)
(OR(L)
Layer
ACC(t)
FOR(J
ACC((1)
FOR(L)
5
81.08
7.64
81.49
7.08
81.08
7.64
81.49
7.08
6
81.32
7.12
81.70
6.89
0,1
82.22
5.78
82.75
5.67
81.67
7.51
81,95
6.77
83.85
5.62
84.69
4.38
8
81.67
7.48
81.92
7.06
0,1.2,3
84.55
5.03
84.58
4.84
9
81.74
6.49
81.88
6.21
0,1,2.3,4
84.59
5.60
84.74
5.04
10
81.58
6,93
81.63
6.78
greie
pidle
pollx
Acouacy under procerded byyers (.323344
Acouaccy under proterded byyer (1.1.))
Accuncc under dth
Accuracy prefi widin
prete
proftc
Taacc Dreur
Taak Ordor
Trak Order
Trak 0r00
Figure 6: Changing curves of accuracy and forgetting metrics with different prepended layers and prefix widths on 10-Split-CIFAR 100 dataset.
Results are similar to the discussion in Appendix M Whether on 10-Split-CIFAR1000 10-Split- TinylmageNet, models with prompt gradient projection always have better accuracy and lower for- getting than those without. We think it proves that our method can be effective in distinct prefix widths and prepended layers. Notice that we name the baseline as "prefix" and our method as "prefix-pgp".
19

---

Published as a conference paper at ICLR 2024
Table 12: Comparison DualPrompt with DualPrompt-PGP in different settings 10-Split-
TinylmageNet dataset. Width and layer mean prefix width and prefix prepended layer index re- spectively.
DualPrompt
DualPrompt-P5
DualPrompt
DualPrompt-PGr
Width
ACC(T)
FOR(L)
ACC(T)
(L)(L)
Layer
ACC(t)
FOR(L)
ACC(1))
FOR(L)
5
81.58
4.63
81.79
4,51
81.58
4.63
81.79
4.51
6
81.39
4.66
81.67
4.50
0,1
82.98
4.29
83.33
3.98
4.93
83.66
83.76
3.96
8
81.36
4.63
81.65
4.44
0,1,2,3
83.64
4.62
84.51
3.72
9 81.55
4.80
81.93
4,70
0,1,2,3,4
83.61
4.68
83.95
4.23
10
82.20
4.34
82.22
3,96
Pergtting urder crspanted eves (01244)
unda grpprdedd (C122)
Perpeting under preto addin
Pagellyy auati
10
Taak
10
Trsl:
wan preta
prafic preflepop
proftc
87-
protts prp
prete
prete 031
Aecureeg
2
Figure 7: Changing curves of accuracy and forgetting metrics with different prepended layers and prefix widths on 10-Split-Tie Toplit-inylmmgeNett dataset.
O T-SNE VISUALIZATION
To better visualize the improvement of our method, we choose L2P and L2P-PGP models. Training on CD-Split-CIFAR dataset, we show the T-SNE results of samples from task across models in various tasks. We pick up logits processed by classifier to report.
20

---

Published as a conference paper at ICLR 2024
t-SNE on 2-th Model
t-SNE on 2-th Model
t:SNE 4-th Model
t:SNE 4-th Model
t:SNE on 6-th Model
t-SNE on 6-th Model
t:SNE on 8-th
t:SNE on 10 Model
t:SE on 10-th Model
Figure 8: T-SNE results L.2P and I.P-PGGP on 10-Split-CIFAR100 dataset. The left column rep- resents L2P, and the right column represents L2P-PGP. The red circle means the drawback existing L2P, and the blue circle shows the improvement our method.
21

---

Published as a conference paper at ICLR 2024
P ALGORITHM
Algorithm 1: Prompt Gradient Projection For L2P (Training phase)
Input: Pre-trained ViT model fe. embedding layer Φg, classifier head for number tasks T, training set {[X), yf = sampling set {[X%, v ) /i= prompt pool {p) ) projection matrix V2,0, number of training epochs E, learning rate 7) loss function L..
Output: prompt pool { classifier head fe
initialize: fex { for / I... do
for I, .... E do
Draw mini-matee batch B [XX, yt = for (X, y) in B do
Embed X into sequence 2t by (X).
Select prompt Ps. from {p;}
Prepend 24 with pr by Tp = [p]:
Obtain prediction by - (c(fo(x))).
end
Calculate per batch loss La by accumulating (y 0). # Gradient projection
if then
Update p by + p-
else
Update p b -N
end end
# Gradient projection matrix update
Initialize the sets of sampled embedding sequences and prompts: X, 0). Pt 0.
for (X), v'a)) in {(X,, as: v..) )=1 do
Sample of embedding sequences X, by concatenation of X, and do (X)).
end
for p in { JM and p E Ps
Sample set of prompts Pt by concatenation of P, and p.
end
Update Vg, by X, and P. according to Appendix C
end
Algorithm 2: Prompt Gradient Projection For L.2P (Testing phase)
Input: Pre-trained ViT model fo, embedding layer Φo, classifier head fe. number of tasks T,
test set ((x))=))= prompt pool {p
Output: prediction y
for t = 1,..,, do for X in {x do Embed X into sequence x, by x, (o
Select prompt Ps from {p
Prepend xa with by Sp [pri
Obtain prediction by = fc(fo(T))).
end
end
22


Reviews:
[{"review_id": "dL2f4kGC90", "reviewer": "Reviewer_Q1LH", "summary": "The submission proposes to combine prompt-tuning based continual learning methods with gradient projection methods. The submission explains how to project the gradient of the learnable prompt tokens and prompt key such that there is no forgetting. The evaluation is performed on 3 datasets and various different continual learning setups.", "questions": "- Figure 1 is a bit unintuitive as normally, models that fill the radar chart are considered to be more powerful but in this case for FOR metrics being close to the center is better and for ACC metrics being fuller is better. Changing the FOG figure so higher is better would make the Figure more readable in my opinion.\n\n- From what I could understand, the gradients for the learnable prompt tokens and prompt keys in the prompt pool are projected. What about gradients for the classifier attached to the backbone? Are these modified as well?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Combining prompt tuning with gradient projection and therefore gaining understanding of the learnable prompts' gradient space in context of CL is novel and important\n- The gradient projection derivation is sound and clearly explained.\n- Evaluation is done on multiple datasets and settings, and the proposed method shows superiority against competitors such as L2P and DualPrompt.", "weaknesses": "- The comparison of prompt-tuning based CL methods with other CL methods does not seem fair to me. Prompt-tuning based methods use pretrained backbones which will give an performance edge when comparing against non-prompt-tuning based CL methods.\n\n=========== Post-rebuttal changes ========\n- The authors have resolved my concern on this particular issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YBjx4vnTnt", "reviewer": "Reviewer_epe6", "summary": "The paper presents Prompt Gradient Projection (PGP), a novel approach that combines Prompt Tuning with Gradient Projection to address the challenge of forgetting in continual learning. Prompt Tuning reduces forgetting in class-incremental learning by selecting and updating relevant prompts based on input samples, while Gradient Projection prevents forgetting in task-incremental learning by ensuring gradient updates in orthogonal directions to old features. PGP integrates these techniques, releasing the need for task identifiers in Gradient Projection and providing theoretical guarantees against forgetting. By deducing the orthogonal condition for anti-forgetting in prompt gradients and using Singular Value Decomposition for efficient computation, PGP significantly reduces forgetting and improves accuracy in various learning settings, achieving state-of-the-art results on benchmark datasets.", "questions": "The paper raises several important points that require clarification and further exploration:\n\nReference Request: In the paragraph preceding equation 1, the statement \"If the update direction is orthogonal to the old features, it follows that $\\Delta W_t x_{t,i}=0$\" lacks a specific reference. It would be beneficial to provide a reference or additional context to support this claim and enhance the paper's credibility.\n\nTime Complexity: The paper would benefit from a more detailed discussion of the method's time complexity, along with a comparative analysis against relevant baseline methods. Understanding the computational efficiency of the proposed approach in relation to other methods is crucial for assessing its practical utility.\n\nPerformance vs. Time Complexity Trade-off: The paper mentions a maximum improvement of around 1% in reducing catastrophic forgetting in the reported experimental results. It is essential to provide a more in-depth discussion of the trade-off between the promising performance of the proposed algorithm and its associated time complexity. Explaining why this algorithm should be prioritized over other existing algorithms, considering the modest improvement, would provide valuable insights.\n\nFigure 3 Clarification: Figure 3 requires further elaboration. While it is presented that both algorithms exhibit similar patterns, with Dual-Prompt appearing to be shifted up(accuracy) or down (forgetting) by a certain scale, a more detailed explanation of this observation is necessary. Clarifying the significance of these patterns and the implications for the proposed method's effectiveness is essential.\n\nAddressing these concerns will contribute to a clearer and more comprehensive understanding of the paper's content and its contributions to the field.\n\n================================\nI appreciate the authors for offering a thorough rebuttal! \nI find this paper to be intriguing and innovative, leading me to raise my score to 8.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper introduces a novel approach, Prompt Gradient Projection (PGP), which combines Prompt Tuning and Gradient Projection to tackle the issue of forgetting in continual learning. The problem statement is well-defined, and related work is thoroughly reviewed, showcasing a strong understanding of the research landscape. The paper's strength lies in its extensive and insightful experimental studies, including ablation studies that highlight the efficiency of PGP and the influence of various factors. Furthermore, the inclusion of proofs, extensive experiments, and detailed algorithms in the appendix enhances the paper's credibility and reproducibility.", "weaknesses": "The clarity and readability of the current paper are areas of concern. The paper's overall structure and writing could be improved to enhance its accessibility. Several specific issues were identified:\n\nReadability and Clarity: The paper's overall readability and clarity could be enhanced. Specific instances of unclear language were noted, such as in the abstract and the last paragraph of page 3. Further revisions are needed to make these sections more lucid.\n\nFigure 2: Figure 2 was found to be unclear and challenging to follow, especially for readers seeking to grasp the method's essence. Consider revising or providing additional explanatory details to improve the comprehensibility of this figure.\n\nProof Presentation: While the proof on page 5 is well-explained up to equation 12, the subsequent section, where multiple elements are combined, is challenging to follow. Expanding on the last two paragraphs on page 5 could enhance clarity and understanding.\n\nTerminology and Consistency: In page 6, there is mention of dividing $V_t$ into $V_{t,1}$ and $V_{t,2}$\" while also referencing $V_{t,0}$\". Clarification is needed to ensure consistency and prevent potential confusion, possibly by referring back to the explanation on page 5.\n\nNotation Clarification: On page 6, there is a reference to \"$V_{t,}$\" which requires clarification to make it more understandable to the reader. Providing a clear definition or context for this notation would be beneficial.\n\nOverall, addressing these issues will significantly improve the paper's accessibility and enhance its overall quality, making it more suitable for publication.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2O07fbdGGk", "reviewer": "Reviewer_LEcv", "summary": "The paper proposes three steps to derive the prompt gradient projection (PGP) approach for continual learning of pre-trained models with prompt tuning. First, using the self-attention mechanism in ViT, the work deduces the gradient restriction conditions for prompt learning based on the feature and the prompt vectors. Second, to simply the solving of these conditions, the work uses a sum space of the feature and the prompt vectors and conducts Singular Value Decomposition (SVD) on this space. Finally, PGP exploits balancing plasticity and stability by rearranging the singular values split on the basis of a threshold.", "questions": "Please see the weaknesses. Overall, it is a good incremental contribution to the field of prompt-tuning for continual learning. However, I am worried about the limited efficacy of the proposed method as well as the scope of the reported experimental baselines. Therefore, I cannot recommend an acceptance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "Given the rising popularity of using pre-trained models for downstream tasks, this is a timely topic to study how such models can be adapted for effective CL.\n\n- Clear explanations and equations built up from scratch.\n- Reasonable experimental settings.", "weaknesses": "- Given that the L2P and DualPrompt backbones have been pre-trained on ImageNet, the limited performance gain on the 10-Split-TinyImageNet is a bit concerning (Table 3). Also, the gains on other dataset settings follow a similar trend. Alternatively, the work could have used a backbone trained on a different dataset altogether [1] and then evaluated its performance on the said settings. A third option would be to use other CL datasets like CUB-200, ObjectNet, etc. that have limited domain and style overlap with the ImageNet dataset.\n- The work claims that one of the major advantages of the proposed PGP method is its training time and memory cost. However, Table 2 does not report the training time of the baseline (L2P-R). As such, it is unclear what conclusions can be derived about the training time.\n- What concerns me is the scope of the proposed PGP method. While the authors use the classic L2P and DualPrompt as their baselines, it would have been more interesting to see how the proposed method complements the continual learning of more powerful pre-trained models such as vision-language models [2]. Also, the baselines considered in this paper are rather weak. Many latest methods are missing. Such as LoRA, K-Adapter, and other parameter-expansion methods.\n\nMinor comments:\n\n- Definition of V_t missing - the paper introduces it as a two part column vector (before eq. 11) without explaining what does it contain. The same for U_t (prior to eq. 10).\n\nReferences:\n\n[1] Zhou, Da-Wei et al. “Learning without Forgetting for Vision-Language Models.” ArXiv abs/2305.19270 (2023): n. pag.\n\n[2] Thengane, Vishal G. et al. “CLIP model is an Efficient Continual Learner.” ArXiv abs/2210.03114 (2022): n. pag.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)