Vector Quantization Prompting for Continual Learning
Authors: Li Jiao
Qiuxia Lai
YU LI
Qiang Xu
Venue: NeurIPS 2024 poster (2024)
Keywords: continual learning
incremental learning
life-long learning
image classification
deep learning
Primary area: machine_vision
TL;DR: This study focuses on one critical deficiency inherent in current prompt-based continual learning methods, i.e., the end-to-end optimization of prompt selection with task loss while keeping its discrete nature as task knowledge representation.

Abstract: Continual learning requires to overcome catastrophic forgetting when training a single model on a sequence of tasks.   Recent top-performing approaches are prompt-based methods that utilize a set of learnable parameters (i.e., prompts) to encode task knowledge, from which appropriate ones are selected to guide the fixed pre-trained model in generating features tailored to a certain task. However, existing methods rely on predicting prompt identities for prompt selection, where the identity prediction process cannot be optimized with task loss. This limitation leads to sub-optimal prompt selection and inadequate adaptation of pre-trained features for a specific task. Previous efforts have tried to address this by directly generating prompts from input queries instead of selecting from a set of candidates. However, these prompts are continuous, which lack sufficient abstraction for task knowledge representation, making them less effective for continual learning. To address these challenges, we propose VQ-Prompt, a prompt-based continual learning method that incorporates Vector Quantization (VQ) into end-to-end training of a set of discrete prompts.  In this way, VQ-Prompt can optimize the prompt selection process with task loss and meanwhile achieve effective abstraction of task knowledge for continual learning. Extensive experiments show that VQ-Prompt outperforms state-of-the-art continual learning methods across a variety of benchmarks under the challenging class-incremental setting.

Vector Quantization Prompting for Continual Learning
Li Jiao', Qiuxia Lai',, Yu Li', Qiang Xu
1 Communication University of China Harbin Institute of Technology, Shenzhen The Chinese University of Hong Kong
(x10000,xlai)ccuc edu.eeu..uueeu............. cn;li, yu@hit. edu. en qxuêcse. cuhk. edu, hk
Abstract
Continual learning requires to overcome catastrophic forgetting when training a single model sequence of tasks. Recent top-performing approaches prompt- based methods that utilize set of learnable parameters (i.e., prompts) to encode task knowledge, from which appropriate ones selected to guide the fixed pre- trained model in generating features tailored to a certain task. However, existing methods rely predicting prompt identities for prompt selection, where the identity prediction process cannot be optimized with task loss. This limitation leads sub-optimal prompt selection and inadequate adaptation of pre-trained features for a specific task. Previous efforts have tried to address this by directly generating prompts from input queries instead of selecting from set of candidates. However, these prompts are continuous, which lack sufficient abstraction for task knowledge representation, making them less effective for continual learning. To address these challenges, we propose VQ-Prompt, a prompt-based continual learning method that incorporates Vector Quantization (VQ) into end-to-end training a set of discrete prompts. In this way. VQ-Prompt can optimize the prompt selection process with task loss and meanwhile achieve effective abstraction of task knowledge for continual learning. Extensive experiments show that VQ- Prompt outperform state-of-the- continual learning methods across a variety benchmarks under the challenging incremennllllt setting. The code is available at
com wwtm:/iaalleeeggmm// Prompt)
Introduction
Humans have the remarkable capability to continually acquire and integrate knowledge of new concepts or categories without forgetting old ones, whereas deep learning models struggle with catastrophic forgetting (40] when tasked with learning sequence of classes [42 10 39]. Continual learning aims at addressing catastrophic forgetting in deep neural networks (DNNs) by striking a balance between plasticity for learning new incoming data effectively and stability retain prior knowledge. Approaches in this field vary: some methods dynamically expand network architec- reconfigure their internal structures [48
tures 64 29 58
for new tasks. Others penalize or alter parameter update rules to prevent interference across tasks [34 7 47 23 Additionally, certain methods interleave stored 36). Despite recent advances, continual
17 24
the update of crucial parameters from previous tasks
65
past data with current ones for training [18 8 44 45 learning remains an open challenge for DNNs.
Recently, prompt-based continual learning has emerged as a promising solution to mitigate catas- trophic forgetting in sequential task learning. This approach enhances pre-trained Vision Trans-
CCoresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Defteeniable X
Discrete Prompety
Ditressablely Discrete Prompe
Difireniable Discrete Prompdl
Prompt Pool
Promppt Pool
Promp:
Ingut-dapenden Prompt
Irgul-dependees Prompt
Prompt
Quary
Modulo
Id
Query
Module
Query
Module
Vitte Quenicaiion
(u)
(b)
(c)
Figure 1: Concept comparison. (a) Prior prompt-based continual learning methods predict prompt identities for prompt selection, which cannot be optimized end-to-end with task loss. (b) Some methods enable end-to-end training by directly generating prompts from the queries using learnable parameters. However, these prompts are continuous, lacking the necessary abstraction to effectively represent the task knowledge essential for generating features tailored to a certain task. (c) Our method incorporates Vector Quantization (VQ) into the prompt generation pipeline to enable end-to- end training of discrete prompts with task loss. See I for details.
former (ViT) with small set of learnable parameters, known as "prompts". These prompts encapsulate task-specific knowledge, shifting the learning focus from the entire model to the prompts themselves, which guide the pre-trained model in generating task-relevant outputs. During inference, the most suitable prompt containing necessary task knowledge is selected from the prompt pool based on the input image to direct the behavior of the frozen pre-trained model.
Current comptt-based methods either involve key-query matching mechanism to select prompts based on the similarity between the image features and the key parameters paired with prompts [2] 611, or explicitly predict the prompt indices and perform the selection accordingly [60] [99 However, the non-differentiable nature of indexing impedes the prompt selection from being optimized end-to- end with task loss. This limitation can lead to diminished performance, as inaccurate prompt selection may fail to tailor the pre-trained features for the specific task. Efforts to address this issue include implementing differentiable prompt selection, such as generating prompts as a weighted sum from the prompt pool (5), or deriving prompts from the intermediate features of the input image [51] 26).. However, the resulting prompts are continuous, which lack the necessary abstraction to effectively represent the task knowledge essential for guiding the pre- pre-trained model to generate features tailored to certain task. A concept comparison is shown in Fig,
The assumption that discrete prompts better represent task knowledge than continuous prompts for continual learning can be supported by both theoretical insights from cognitive science and empirical evidence. Discrete prompts mimic the organizational structure of memory and knowledge in the human brain, which is typically understood to consist of discrete units such as concepts and facts [19]. This clear separation of information helps prevent interference among different knowledge domains, and enables models to provide distinct guidance for feature extraction specific to each task. Such knowledge abstraction aligns with categorical perception in human cognition, where sensory inputs perceived as distinct categories (c.g., colors, phonemes) rather than continuous spectrum [41]. Furthermore, empirical comparisons in $5.2 demonstrate the effectiveness of discrete prompts when optimized end-to-end with task loss (e.g.. V.-.VQQPPrrroo VQ-Prompt V.S. CODA-P EyoPrompt). In summary, discrete prompts hold significant promise for improving the continual learning capabilities of models, bringing them more in line with human learning.
Optimizing prompts with task loss while preserving their discrete properties as representations of concepts poses non-trivial challenge. In this paper, we introduce Vector Quantization Prompting (VQ-Prompt) for continual learning, which can optimize prompts using task loss while preserving their discrete characteristics as concept representations. This method involves initially generating continuous prompt and then replacing it with its nearest match from a predefined prompt pool. To address the non-differentiaabliiy inherent in prompt quantization, we apply gradient estimation to propagate task loss to the continuous prompt, while additional vector quantization regularization terms further refine the learning of the prompt pool. To further stabilize task knowledge learning, we use representation statistics to mitigate the classification bias towards previous tasks, thereby enhancing continual learning performance.
Our contributions are three-folds: (1) We propose VQ-Prompt, an end-to-end learnable discrete prompting mechanism for continual learning, addressing a critical yet overlooked aspect in the current literature. (2) We leverage gradient estimation to pass the task loss to prompt-related parameters while regularizing the learning of the prompts with vector quantization terms, which facilitates the

---

end-to-end training of the discrete prompt pool. (3) We incorporate representation statistics during training to further stabilize task knowledge learning and improve the overall continual learning performance. Extensive experiments show that VQ-Prompt consistently outperforms state-of-the-art continual learning methods on a variety of benchmarks.
2 Related work
Continual Learning refers to the process where multiple tasks are learned sequentially without
forgetting 42 10 39]. Generally, continual learning has three scenarios [54]. iaskeincrrmental
learning (TIL) learns different classes for each task and assumes having task identities available test time. nomain-inncrementaa learning (DIL) maintains the same set of classes for different tasks while changing the data distributions across tasks, and task identities are not provided for inference. For caassinncremenal learning (CIL), each task involves new classes and all the learned classes are to be classified without task identities available during inference. In this paper, we focus on the more representative and challenging CIL scenario. Numerous efforts have been devoted to
alleviating catastrophic forgetting. Architecture-basee methods address this by either dynamically expanding network architectures [64] 22 588 or modifying internal network structures [48 17 [2
for new tasks. Regularization- bbssdd methods focus on limiting updates to vital parameters from
carlier tasks (20. 27
65
1
49], or modifying the rules for parameter updates to reduce task
interference [34 7 47
23 Rehearsal-based methods incorporate previous data with current data
4 6 36). Despite recent advances, continual during training to mitigate forgetting [18 8 [44 45
learning remains challenging and evolving field.
Recently, there has been a surge in methods leveraging prompting techniques from natural language processing (NLP) [28 30 for continual learning. These methods instruct frozen pre-trained transformer using learnable prompts that encode task knowledge. During training, prompt selection is either through key-query similarity matching [62] or
Prompt-based Continual Learning Methods.
indicated by task identity [6] 12 60 59. In inference, the appropriate prompt is chosen through similarity matching with key feature centroids. However, both kinds of prompt selection non-diffcrentiable, making it challenging to optimize them end-to-end with the task loss, particularly when the gap between the pre-training task and unknown future tasks is large. To address this, CODA-Prompt [50] adopts a soft prompt selection, i.e., generating prompts as a weighted sum from the prompt pool. APG (51) and EvoPrompt (26] learn to derive prompts from intermediate image features. Nevertheless, all three methods generate prompts that are continuous, which lack the necessary abstraction to effectively represent the task knowledge essential for instructing the pre-trained model to produce features tailored to a certain task. In this paper, we present a new prompting framework for continual learning capable of optimizing prompts with task loss while preserving their discrete properties as the representation of task knowledge.
Vector Quantization in Representation Learning. Vector Quantization (VQ) is technique used in signal processing and data compression to represent a set of vectors (data points) with a smaller set of "coding vectors" (CVs). Unsupervised VQ algorithms such Self-organizing Maps (SOMs) [22 and Neural Gas (NG) networks (38] attempt to obtain a set of CVs that optimally represent the data. Supervised VQ algorithms such as Learning Vector Quantization (LVQ) [2] focus on reducing the misclassification rates by refining decision boundaries between classes. In generative modeling, VQ
has been used to learn structured discrete latent spaces in VQ-VAE (55) and VQ-GAN [33 to achieve higher fidelity images. Recently studies have explored combining VQ with continual learning to constrain the feature space, aiming to enhance class separation and retrain prior knowledge across increments [52. 53 9 37]. In this paper, instead of utilizing VQ to confine the feature space, we employ VQ enable end-to-end learning set of discrete prompts that effectively encode task knowledge in a learning system that evolves over time.
3 Preliminary
Problem Formulation. In insremmantmm learning (CIL), model is required to sequentially learn a series of tasks with disjoint class sets, and to accurately classify all seen classes during evaluation. Formally, let D,- = (a), ,,,, denote the training set of the f-th task, where of E X, is an input image, y e is the target label, and N₂ is the number of samples. The label spaces of all the tasks mutually exclusive, i.e., D Ve = 0, where T is the total number of tasks. Consider deep
3

---

learning model = == with a backbone f()) and a classifier 6(-). During training on task t, the model only has access to D,, which raises risk forgetting old tasks. After learning task t, the model is expected to perform well on all classes in D4.t = 1 and further on Vi.r after completing training on all tasks.
Prompt-based Learning is an emerging approach in NLP 32 that involves incorporating extra instructions into pre-trained models to guide their performance on specific tasks. Rather than relying on extensive retraining or task-specific fine-tuning, this technique leverages prompts to shape the behavior of pre-trained models, providing adaptable instructions that helps them handle a wide range of downstream tasks more effectively. In vision-related continual learning, prompting is typically employed with Vision Transformer (ViT) [
ViT consists of sequence of multi-head self-attention (MSA) blocks [56] For clarity, we take one MSA block example illustrate the prompting. We denote the input query, key, and value of the MSA block as hQ: hx and hy, respectively. Here, h. E RRLxD I. is the sequence length, and D the embedding dimension. The output of the MSA computed as:
MSA(hq. hg,, hv) = Concat(h....,,.)))))
(I)
hm Attention(hq W hg m hv W.Y)
where WO, W.. W and W are projection matrices, m= m=1,.-. M is the head index, M is the number of heads, and ho=hh = hy h for SA.
Previous prompt-based continual learning methods mainly implement Prompt Tuning (Pro-T) 28 and Prefix Tuning (Pre-T) [3). Pro-T prepends the same prompt p E RL5xD hQ. hg, and hy. The prompting function of Pro-T is defined as:
Sp. h) MSA([p: hQ), (p; hx]. [p; hvl),
where [ means concatenating along the sequence length dimension. The output dimension is LL)++)) D. Pre-T splits p along the sequence length dimension into pk, pv E LRL2/22D which prepended to h and hv, respectively:
[p. h) MSA(hQ, [px; hx), hv]).
The output dimension is the same as that of h. In continual learning, the pre-trained ViT backbone is kept frozen as general feature extractor, and the prompt parameters are trained to capture task knowledge. Proper prompts corresponding the input samples are selected to guide the feature extraction during inference. Following 61 50 59],
we adopt Pre-T strategy in our method.
Method
As shown in Fig. 2 our VQ-Prompt approach begins by constructing a continuous prompt through a soft selection from the prompt pool ($4.1). The continuous prompt is then quantized to an element in the prompt pool, which is inserted into an MSA block of a frozen pre-traine pre-trained transformer. This process is made end-to-end trainable through gradient estimation and vector quantization regularization ($4.2). such that the prompting parameters, namely the keys and the prompt pool could all be optimized using the task loss. In this way. VQ-Prompt can yield discrete prompt for each input while maintaining end-to-end optimization. To better stabilize task knowledge learning, representation statistics of previously learned classes are employed to mitigate the classification bias ($4.3)
4.1 Prompt Formation
Most previous prompting based continual learning approaches construct their prompts by selecting from the prompt pool based on key-query similarity [62 61 or other task identity prediction mecha- nisms 60 59 making the prompt selection process non-differentiable In our prompt formation, first generate a continuous prompt by aggregating all the elements in the prompt pool based on the similarity scores between the query and the keys. Specifically, given a query q from the input image, the similarity score is calculated as:
Softmax Kq),
(4)

---

Prompt Keys K
Prompt Pool P
Classifier
Seatistics
NN Lookeupi
Inverr pooompss
i
Input Image
Query
Cosine Similarity
Weighted Sum
Pre-trained Transformer
Function
P
Gradient Estimation
Figure 2: VQ-Prompt framework. An imput image is passed through query function (e.g., fixed pre-trained ViT) to generate a query q, which is then used to compute similarity scores with prompt keys K. These scores a serve as weights to aggregate elements from the prompt pool P to form continuous prompt p'. This prompt is subsequently quantized to an element within the prompt pool p. and then fed into a specific MSA block of frozen pre-trained transformer. To ensure differentiability. the prompt quantization process employs gradient estimation and prompt pool regularization. The representation statistics of features from learned classes are used to stabilize task knowledge learning. More details are shown in 4
where K NRD is the prompt key matrix, e (RD is the query, N is the number of keys, and D is the embedding dimension. Then, the continuous prompt is obtained by:
p' = Σ a.P, 1=1,
(5)
where PR is the i-th element in the prompt pool, and Lp is the length of the prompt. Such a prompt formation process is differentiable and can be viewed simplified version of CODA-P (50) Here, we do not learn an extra attention parameter for weighting the query, nor do we increase the number of elements in the prompt pool or the number of keys during sequential task learning.
4.2 Vector Quantization Prompting (VQ-Prompt)
Nearest-neighbour Look-up. The continuous prompt p' obtained in Eq. (5) is conditioned specific instance, i.e.. it varies with the input images, making it insufficiently abstract to capture task knowledge effectively. We further perform prompt quantization by performing the nearest neighbour (NN) look-up in the prompt pool P using p'. The quantized prompt to be fed to the MSA block is obtained by:
p = Pk, k = arg min l|p' - Pylle, p
(6)
Such prompt selection pipeline can be viewed as specific non-linearity that maps the continuous prompt to 1-of-N elements the prompt pool.
Gradient Estimation. Because the arg min operation in Eq. (6) is non-differentiable, we use the ttraigh-thhouggh estimator 3 to approximate the gradient of p' using the gradient of p. Despite its simplicity, this estimator has demonstrated its effectiveness in our experiments. Specifically, in the forward process, the quantized prompt p is passed the MSA block the pre-trained transformer. During the backward computation, the gradient of is transferred unaltered to p', and the optimization of prompt pool P and keys K guided by the similarity scores (c.f., Eq. (4). This gradient estimation is justified, valuable information on how prompt parameter learning could instruct the transformer features to minimize the cross-entropy (CE) loss during task learning, In this way, each prompt and key element adjusted according to its relevance to the current learning context, rather than undergoing wholesale changes. This allows for more updates of tesk-relevant elements without disrupting less relevant ones, thereby maintaining previously acquired knowledge while adapting to new tasks.
and p' share the same Lp x -Dddimensional space, and the gradient of p provides
Vector Quantization (VQ) Regularization. Though the prompt pool P could receive gradients from the task loss through straight-throud gradient estimation of mapping from p to p', to enhance the learning of the prompt embedding space, add an extra VQ objective. This VQ objective uses
5

---

the Ly error to move the selected element p of prompt pool towards the continuous prompt p':
Lvo = ||sg[p" - -l12
Here, sgi-] stands for the stop- goo-ggradeent operation [55], which constrains its operand to be a non- updated constant during training.
To ensure that the learning processes of the prompt keys K and the continuous prompt p' align closely with the characteristics of the element p from the prompt pool P. we further introduce commitment regularization term. This term is defined mathematically as follows:
Commii = Ip' sg[p]]12
(8)
The incorporation of this commitment loss ensures that the prompt formation process described in $4.1 optimized to yield prompts to commit the elements in the prompt pool P. thereby promoting consistency and stability in the prompt learning process.
4.3 Stabilizing Task Knowledge Learning with Representation Statistics
Though prompts effectivel capture task knowledge to guide the backbone f()) in producing instructed representations, the classifier Φ6-) may develop a bias towards new classes in continual learning scenarios [16] 22. This bias adversely affect the learning of the prompts for subsequent tasks. To mitigate this issue and stabilize task knowledge learning, we employ a strategy similar to [59], which leverages the representation statistics of previously learned classes correct classifier bias and stabilize prompt learning. Specifically, after completing task I, we calculate the mean μ4, and variance o for each class E Vi:r with the learned prompt parameters and the pre-trained backbone. By modeling each class as Gaussian distribution, we generate pseudo features through sampling from these distributions. These pseudo features are then used fine-tune the classifier, thereby mitigating its bias towards recent classes. The balanced classifier could help stabilize task knowledge learning in the prompts, alleviate catastrophic forgetting, and enhance overall continual learning performance.
4.4 Overall Optimization Objective
The overall loss function is defined in Eq. (9) which extends the task loss Lee with two terms, namely quantization objective LvQ weighted by Ag. and a commitment term LCommit weighted by Ae
L -= LCe + XyLve + Xc
Here, LCE is the cross-entropy loss that supervises the learning of the image classification task, Lvo the VQ regularization defined in Eq. 77) that updates the prompt pool elements to move towards the continuous prompt p'. and Lcommir is the commitment term defined in Eq. (8) that forces the prompt formation process to commit to the prompt pool elements. During training, the pre-trained backbone f(() is frozen, while the classifier (+-) and prompting parameters K and P are optimized across all the tasks. During task knowledge stabilization, only the classifier d6-) is actively trained.
Experiment
5.1 Experimental Setups
Datasets. We consider three representative benchmarks for evaluating CIL. ImageNer-R [4] includes 200-class images that either hard samples for ImageNet newly collected data with different styles, thus can serve as a challenging benchmark for continual learning with pre-trained models. In the experiments, performance. Split CIFAR-100 randomly splits the original CIFAR- 100 (5) into 10 disjoint tasks, each containing 10 classes. Split CUB-200 is built on CUB-200-2011 (57 a fine- aine-grained classification dataset, by randomly splitting the 200 classes into 10 tasks, where each task contains 20 classes.
divide it into 5, 10, and 20 disjoint tasks and report the corresponding
Baselines. We evaluate approach against a comprehensive set of baselines contextualize its performance. Following [50). we include "Joint Training" as the upper-bound performance, setting a benchmark for optimal results. To establish lower bounds, we employ two sequential learning baselines, denoted as "FT" and "FT++", with the latter refraining from updating the logits of previously learned classes during the training of new tasks.
We also consider 5 prompt-based approaches: I.2P (2). DualPrompt 61. HiDe-Prompt (59] CODA- Prompt (50] and EvoPrompt (26), where the last two yield continuous prompts. For L2P, we include

---

Table 1: Comparison on ImageNet-R. Results "5-task", "10-task", and "20-task" settings included. Backbones are pre-trained on ImageNet-1K 1 denotes larger values are better. See $5.2
5-task
10-task
20-task
Method
Pub.
FAA (t)
CAA
FAA
CAA
FAA (1)
CAA (1)
Joint-Train.
82,06
82,06
82.06
FT
18.74 + 0.44 48.39 +
10.12
0.51
35.23 + 0.92
4.75 0.40
22.8 0.37
FT++
60.42 +: 0.87 71.59
48.93 : 1.15
66.79 1: 0.92
35.98 + 1.38 59.68 + 0,95
1,2+++
02
CVPR22
70.83 + 0.58 78.34
69.29
0.73
78.30 + 0.69 65.89 + 1,30
77.15 + 0.65
Deep L2P++
62
CVPR22
73.93 4 0.37 8.51
71.66
79.63 0.90 68.42 1 1.20 78.68 1.03
DualPrompt
61
ECCV22
73.05 + 0.50 79.47 0.40
71.32
78.94 67.87 + 1.39 77.42 + 0.80
CODA-P
50
CVPR23
76.51 1: 0.38 0.54 75.45
81.59 0.82 72.37 1: 1.19 79.88 1.06
Prompt
59
NeurIPS23
76.29 1 0.10 78.77
7.174
78.76 0.11 76.46 1 0.06 78.76 0.11
0.11
EvoPrompt
26
AAA124
77.16 + 0.18 82.22 + 0.54 76.83 + 0.08 82.09 + 0.68 74.41 + 0.23
80.96 + 1.42
VQ-Prompt
79.23 i 0.29 82.96 0.50 78.71 0.22 83.24 1 0.68 78.10 1 0.22 82.70 1.16
denotes results obtained by running the official code with ImageNet-IK pre-trained weights.
Table 2: Comparison on Split CIFAR-100. Backbones are pre-trained on ImageNet-IK, See $.2 for details.
Table 3: Comparison on Split CUB-200. Back- bones are pre-trained on ImageNet-2IK. de-
notes backbone frozen. See $5.2
10-task
10-task
Method
Pub.
Method
Pub.
FAA
CAA
FAA
CAA (1)
Joint-Train.
91.38
Joint-Train.
88.00
FT
29.21 + 0.18 37.37 + 0.89
FT FT+
11.04 + 0,78 31,96 1 0.74
FT++
49.91 1 0,42 74.76 + 0.93
37.81 2.86 63.55
1.62
LWF 31
TPAM117
64.83 1.03
LWF 31
TPAM117
69.75 1.37 80.45 + 2.08
12P++ 62
CVPR22
82.50 + 1.10 88.96 + 0.82
BIC
63
CVPRI9
81.91 2.59 89,92 + 1.57
Deep L2P++
CVPR22
84.30 1.03 90.50 + 0.69
DualPrompt 61
ECCV22
66.00 0.57 77.92 1 0.50
DualPrompt 61
ECCV22
66.00 :: 0.57 77.92 : 0.50
CODA-P 50
CVPR23
70.03 0.47 74:26 + 0.24
CODA-P
50
CVPR23
70.03 + 0.47 74.26 + 0.24
"SLCA 67
ICCV23
84.71 + 0.40 90.94 + 0.68
EvoPrompt
26
AAAI24
87.97 0.30 92.26 + 0.86
HiDe-Prompt 59
NeurIPS23
86,61 0.18 87.01 0.03
VQ-Prompt
88.73 0.27 92.84 = 0.73
VQ-Prompt
86.72 0.94 90.33 + 1.03
its two variations from [50]:
"L2P++" and "Deep L2P++". L2P++ uses Pre-T instead of Pro-T T and inserts the prompts to the first MSA block, which achieves better performance than the original L2P B3 Deep L2P++ extends L2P++ by incorporating prompts into the first five MSA blocks.
In addition to prompt- based methods, we include a classical regularization-base method LwF 31 and rehearsal-basee method BiC 633, providing more comprehensive overview for evaluation.
Evaluation Metrics. We present Final Average Accuracy (FAA) and Cumulative Average Accuracy (CAA) for comparison. FAA refers the last average accuracy after learning all the tasks, which is equivalent to "Last-Acc" in [67. CAA is the average of historical FAA values after learning each task, which is equivalent to "Inc-Ace" in [67]. The formal definitions of the metrics are in $A.1
Implementation Details. We follow prior works (62 61 50, 59 67 26 and use ViT-Base
pre-trained with supervised learning on ImageNet- IK 46 or ImageNet-21K (43 as the backbone. The number of keys and prompt elements is 10. The prompt length Lp is 8. The embedding dimension D 768 which is the same the feature dimension of ViT-Base.
Our method is trained using an Adam W optimizer (35] with an initial learning rate of 0.0025 and a cosine decay schedule. The batch size is 128 for Split CIFAR-100 and Split CUB-200, and 64 for ImageNet-R. The number epochs set to be 20 for training on all three datasets. The classifier bias mitigation process described in $4.3 requires ten epochs of training. Each experiment is run single NVIDIA GeForce RTX 4090 GPU, More details are presented in $A.2
5.2 Comparison Results
In this section, we present comprehensive comparison with established baselines across various datasets and pre-training regimes. The performances of different methods reported in separate
7

---

Table 4: Results on 10-task ImageNet-R with different self-supervised pre-training paradigms.
iBOT-IK [68
DINO-IK [S]
Method
Pub.
FAA (1)
CAA (1)
FAA (t) CAA (t)
DualPrompt
G1
ECCV22
61.51 1.05
67.11 1:111:00 0.08
58.57 1.45 64.89 1 0.15
CODA- CODA-Prompt
30
CVPR23
66,56 1 0.68
73.14
63.15
0.39 69.73 0.25
HiDe-Prompt
39
NeurIPS23
71.33 0.21
73.62
1 0.13
68.11
1.11100
0.18 71.70 0.01
VQ-Prompt
71.68 1 0.72
76.66
0.40
68.42 0.28 74.43 0.58
tables due to the varying experimental settings such as the number of tasks and the pre-training dataset, to ensure a fair and accurate comparison.
Results on ImageNet-R. Table [ shows the results across five runs on ImageNet-R with 5-task, 10-task, and 20-task splits using the ViT-Base backbone pre-trained with supervised learning on ImageNet-IK. KK Our VQ-Prompt consistently outperforms other methods across key metrics such as FAA and CAA for all task splits, including the latest prompt-based method EvoPrompt.
Results on Split CIFAR-100. Table presents the results across five runs on CIFAR- split into 10 tasks, with the ViT-Base backbone also pre-trained ImageNet- IK with supervised learning. Our VQ-Prompt achieves superior results compared to other methods using the same pre-training weights.
Results on Split CUB-200. Table displays the results on Split CUB-200. Following [67, we use ViT-Base backbone pre-trained on ImageNet-21 for this dataset. VQ-Prompt achieves superior comparable performance compared with all other methods, including SLCA [67], which trains the entire network without freezing the pre-trained feature extraction backbone. This highlights its potential and efficacy in continual learning for fine-grained classification tasks.
Other Pre-training Regimes. Table 4 summarizes the experimental results across three runs on the 10-task ImageNet-R dataset, utilizing different self-supervised pre-training paradigms, namely iBOT-1K 688 and DINO-IK [5]. These results demonstrate that our method consistently outperforms state-of-the-art prompt-based continual learning methods, underscoring its robustness and efficiency leveraging self-supervised pre-training for continual learning tasks. Specifically, VQ-Prompt shows greater advantage in CAA than FAA, indicating its superior ability to leverage past knowledge for aiding current tasks, despite a slightly higher degree of forgetting relative to some baselines. This trade-off between adaptation to new tasks and forgetting of previous ones is common challenge in continual learning. With self-supervised pre-training, our method tends to prioritize adaptability to new tasks to ensure that the model remains relevant and effective in dynamic environments.
5.3 Ablation Study and Additional Analysis
In this section, we assess the effectiveness of different components illustrated in $ The experiments performed on 10-task ImageNet-R with the ViT-Base backbone pre-trained on ImageNet- IK.
Elfectiveness of VQ Design. Our VQ design (c.f., $4.2 enables end-to-end training of the discrete prompt selection in continual learning. Here, we compare it with an alternative intuition design choice, i.e., rewriting Eq. (4) as a Softmax K q/r), and reducing the temperature т of the softmax operation. A lower temperature leads to a "sharper" distribution of a, allowing the prompt formation in Eq. (5) to more closely approximate discrete prompt selection during end-to-end training with task loss. This baseline is denoted as "Soft-Prompt". Fig. 3 (a) presents the FAA values of Soft-Prompt with different 7 values. Surprisingly, Soft-Prompt achieves its best performance of 77.15 at == 1.0 instead of at lower values. While its performance is comparable to other prompt- based methods, Soft-Prompt falls short of "VQ-Prompt-S", which achieves an FAA value of 78 on 10-task ImageNet-R. Here, VQ-Prompt-S is simplified version of VQ-Prompt that does not use representation statistics. Our standard version VQ-Prompt further achieves an FAA value of 78.83. This observation underscores the effectiveness of our VQ design compared to the intuitive low-temperature soft prompt selection.
The rationale behind this is that reducing the temperature makes the softmax operation more sensitive to differences logits. While this heightened sensitivity acceptable when the model confident in its predictions, it can lead to more aggressive prompt choices at the beginning of the training when the model less fully trained. contrast, our VQ-Prompt utilizes standard softmax for prompt formation, substitutes the resulting prompt with the nearest one in the prompt pool, and enables
8

---

(n)
79
VG-Prompt
76.83
(c)
VG Prompt-S
77.75
78.09
78.67
78.66
78.77
78.79
78.76
75.78
FAA
77
70.00
77.15
77,12 70.00
70.55
78.01
78.71
78.69
70.81
78.76
78.63
76.29
76
76.13
78.68
78.64
78.56
78.80
78.62
78. 45
78,00
76.60
75
75.19
74
0.001
0,000
0.01
0.03
78.71
78.47
78.67
78.76
78.68
76. 65
78.72
77.35
-7.5
Softmax Temperature
(b)
N=10 N-30 N-50 N-100
79.50
78.63
78.58
78.20
78.73
78.90
78.80
77.49
-77.0
78.55
78.44
40
78. 70
78.62
78.
78
77.11
79.5
78.73
75
78.53
78. 41
78
78
78 53
77.86
78.6
78.0
5.
78.30
78.47
78.26 78.40
78:18
78.39
78.51
-75.0
12
20
24
0.5
FAA
Lp
Ag
Figure 3: Ablation study. (a) VQ Design. We show the performance of an alternative of VQ Design, "SSoft-rroppt", that generates the continuous prompt with low-temperature softmax operation only without using VQ. Here, "VQ-Prompt-S" is simplified version of VQ-Prompt without using representation statisties.
(b) Prompt Hyperparameters. The results of varying the size of the prompt pool N and the length of a single prompt Lp are displayed. (c) Loss Weights. The results of different combinations of N and d values are presented. See $5.3 for details.
Table 5: Effectiveness of classifier bias mitigation. Results for "S-task", "10-task", and "20-task" settings on ImageNer-R are included. "C.B.M." denotes "Classifier Bias Mitigation". Backbones are pre-trained on ImageNet-IK. denotes larger values are better. See $5.3 for details.
5-task
10-task
20-task
Method
C.B.M.
FAA (1)
CA)
(1)
CAA (f)
FAA (1)
CAA (f)
L,P++
662
No
70.83 + 0.58
78.34
69.29 + 0.73
78.30 = 0.69
65.89 # 1.30
77.15 = 0.65
L2P++ V2
Yes
74.11
72.93
0.27
78.63
0.80
70.99 0.26
77.65 = 0.79
EvoPrompt 26
No
0.18
82.22 + 0.54
76.83
+ 0.08
82.09 = 0.68
74.41 # 0.23
80.96 = 1.42
VQ-Prompt-S
No
78.52
82.64
0.68
78.00
0.39
82.83
0.69
76.19 = 0.26
81.68 1.02
VQ-Prompt
Yes
79.23 0.29
82.96
0.50
78.71
0.22 83.24 0.68
78.10 1: 0.22
82.70 1.16
end-to-end learning through gradient estimation and VQ regularization, which proves be more robust for task knowledge learning compared with Soft-Prompt.
Hyperparrameters for Prompting. There are two key hyperparameters: i) the size of the prompt pool N that represents the total capacity of the learnable prompts, and ii) the length of a single prompt Lp which determines the capacity of single prompt to encode certain aspects of task knowledge. The total size of the prompts to be prepended to the input of one MSA block is given by N x Lpp
Fig. (b) illustrates the impact of varying Lp and N on FAA performance. Across different parameter configurations, our method consistently outperforms existing approaches, demonstrating its robustness. Specifically, excessively small Lp value consistently yields sub-optimal results, as indicated by the lower FAA scores across different N values. Increasing can partially compensate for small Ip values, leading to improved performance. In contrast, increasing Lp generally enhances performance up to a certain threshold, beyond which an overly large Lp may cause knowledge overfitting, reflected by the stable slightly declining FAA scores for larger L, values. We selected L, and = 10 as our default configuration. This configuration achieves superior results with fewer parameters compared to other prompt-based methods (c.f., A.33
Our competitive performance is primarily attributed to the use of VQ, which offers several key benefits for prompt-based continual learning. First, VQ enables the encoding of task knowledge into discrete prompts, which provide a more compact representation than continuous prompts. This discrete nature helps in capturing essential task-specific features with the necessary level of abstraction. Second, integrating VQ within the prompt-based framework facilitates end-to-end optimization with task loss, ensuring that the selected prompts are highly relevant to the task at hand, thereby enhancing the task

---

knowledge learning of the prompts. This enables the use of shorter prompts while maintaining strong performance, making our approach more parameter-efficieet and effective.
Impact of X and A-- To further enhance the learning of prompt-related parameters, we introduce two regularization terms Lvς and LCommit guide the learning (c-f., $4.2) We investigate the impact
of various loss weights, as shown in Eq. (9). The outcomes are detailed in Fig. 3 (c). As can be
observed, these two terms can contribute good performance when assigned with a relatively broad
range of values. According to Fig. 3 (c), we set X 0.4 and A =0 in all of our experiments.
Effectiveness of Classifier Bias Mitigation. The classifier bias mitigation utilizes representation statistics to stabilize task knowledge learning (c.f.. $4.3), which can also be applied to other methods. To evaluate its potential advantage, integrated this component into L2P++. As shown in Table 5 1,2P++ with representation statistics ("1.2P+++ V2") achieves improved performance over the original 1,2++ across all three ImageNet-R settings, but remains inferior our method. Additionally, include the results of "VQ-Prompt-S", a simplified version of VQ-Prompt that omits classifier bias mitigation. Notably, VQ-Prompt-S still outperforms other methods such EvoPrompt, demonstrating the effectiveness of our approach. This indicates that the classifier bias mitigation process can contribute performance improvements, but the sole determinant of the final performance.
6 Discussion and Conclusion
This study focuses on one critical deficiency inherent in current prompt-based continual learning methodologies, specifically the end-to-end optimization of the prompt selection process with task loss while keeping its discrete nature as the representation of task knowledge. Our proposed Vector Quantization Prompting (VQ-Prompt) framework mitigates the challenge by substituting continuous prompts with their nearest counterparts from the prompt pool, thereby enhancing task accuracy through a more aligned and abstract representation of conceptual task knowledge. To overcome the non-diffeen din-dfrffeeniabiiity inherent in this process, we employed gradient estimation along with vector quantization regularization terms, which allows for optimizing prompt retrieval with task loss. Repre- sentation statistics are utilized to further stabilize task knowledge learning. Extensive experiments in class- mass-incrementll scenarios consistently demonstrate VQ-Prompt's superiority over SOTA methods.
Limitations and Future Work. One limitation of VQ-Prompt is its dependence on pre-trained models. While these models offer rich initial knowledge, enabling a more mature learning process akin to that of an adult, they also inherit the limitations of the pre-trained data distribution and the high computational costs associated with their use. This challenge is not unique VQ-Prompt but applies broadly to other continual learning methods that rely on pre-trained models. Another limitation is the absence of constraints calculating similarity scores and prompt keys, which can result in suboptimal prompt utilization, i.e., some prompts are more frequently selected for samples from different tasks while others are less frequently used. To alleviate this, one possible strategy is to introduce constraints on prompt selection, such as limiting the reuse of prompts that have already been heavily utilized by previous tasks to enhance the diversity and utility of the prompts. We leave more thorough exploration of such prompt selection constraints for future research.
Broader Impacts
This paper presents a prompt-based continual learning method to tackle the challenges of knowledge preservation in sequential task learning. Our approach facilitates continual adaptation and learning, thereby contributing to the advancement of intelligent, adaptive, and efficient technologies applicable to various domains, including autonomous vehicles and personalized Al agents. While VQ-Prompt advances Masss-ncremmnnal continual learning, its robustness could be compromised if poisonous samples are introduced after a concept has been learned. One possible mitigation strategy is to implement robust anomaly detection methods to identify and filter out suspicious input samples before they are introduced into the training process.
Acknowledgments and Disclosure of Funding
This work is supported by the National Natural Science Foundation of China (No. 62176241) and the Fundamental Research Funds for the Central Universities (No. CUC24QT06).
10

---

References
[1] R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuytelaars. Memory aware synapses: Learning what (not) forget. ECCV, pages 139-154, 2018.
[2] E. Belouadah and Popescu. I12m: Class incremental learning with dual memory. In ICCV, pages 583-592, 2019.
[3] Y. Bengio, N. Léonard, and A. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 2013.
[4] P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara. Dark experience for general continual learning: strong, simple baseline. Neur/PS, 33:1 1321550--1999,, 2020.
M. Caron, H. Touvron, Misra, H. Jégou, J. Mairal, Bojanowski, and A. Joulin. Emerging properties in self-suue vision transformers. CVPR, pages 9650-9660, 2021.
[6] S. Cha, S. Cho, Hwang, S. Hong, Lee, and Moon. Rebalancing batch normalization for exemplar- based class-incremental learning. In CVPR, pages 20127-20136, 2023.
[7] A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny. Efficient lifelong learning with a-gem. In ICLR, 2019.
[8] A. Chaudhry, M. Rohrbach, M. Elhoseiny, T. Ajanthan, P. K. Dokania, P. H. Torr, and M. Ranzato. On tiny episodic memories in continual learning. arXiv preprint
2019, 2019.
[9] K. Chen and C.-G. Lee. Incremental few-shot learning via vector quantization in deep emhedded space. In ICLR, 2021.
[10] M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE TPAMI, 44(7):336633385 2021.
[11] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenbom, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, Gelly, et al. An image is worth 16x words: Transformers for image recognition scale. ICLR, 2021.
[12] A. Douillard, A. Ramé, Couairon,
Cord. Dytox: Transformers for continual learning with dynamic token expansion. In CVPR, pages 9285-9295, 2022.
[13] P. Esser, R. Rombach, and B. Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873-12883, 2021.
[14] D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang. E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, et al. The many faces of robustness: A critical analysis of dui-o--distribution generalization. In ICCV, pages 8340-8349, 2021.
[15] D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song. Natural adversarial examples. CVPR, pages 15262-15271, 2021.
[16] S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, pages 831-839, 2019.
[17] C.-Y. Hung. C.-H. Tu, C.-E. Wu, C.-H. Chen, Y.-. Chan, and C.-S. Chen. Compacting, picking and growing for unforgetting continual learning. NewrIPS, 32, 2019.
[18] D. Isele and A. Cosgun. Selective experience replay for lifelong learning. In AAAI, volume 32, 2018.
[19] Kiefer and F. Pulvermüllerr Conceptual representations in mind and brain: Theoretical developments, current evidence and future directions. Cortex, 48(7):805-825, 2012.
[20] Kirkpatrick, Pascanu, Rabinowitz, J. Veness, Desjardins, A. A. Rusu, Milan, Quan, T. Ramalho, A. Baahska-Barwinska,, et al. Overcoming catastrophic forgetting in neural networks. PNAS, 114(13)3521-356,, 2017.
[21] T. Kohonen. Improved versions of learning vector quantization. In IJCNN, pages 550, 1990.
[22] T. Kohonen. The self-organizing map. Proceedings of the IEEE, 78(9):1464 1480, 1990.
11

---

[23] Y. Kong, L. Liu. Z. Wang, and D. Tao. Balancing stability and plasticity through advanced null space in continual learning. In ECCV, pages 219-236, 2022.
[24] T. Konishi, M. Kurokawa, C. Ono, Z. Ke, G. Kim, and B. Liu. Parameter-leve soft- masking for continual learning. In ICMI, 2023.
[25] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[26] R. Kurniawan, X. Song. Z. Ma, Y. He, Y. Gong, Qi, and X. Wei Evolving parameterized prompt memory for continual learning. In AAAI, volume 38, pages 13301-13309, 2024.
[27] S.-W. Lee, J.HH. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang. Overcoming catastrophic forgetting hy incremental moment matching. NeurIPS, 30, 2017.
[28] B. Lester, Al-Rfou, and N. Constant. The power of scale for parameter-effiiient prompt tuning. In EMNLP, pages 3045 3059, 2021.
[29] X. Li, Y. Zhou, T. Wu, R. Socher, and C. Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. ICML, pages 3925 -3344, 2019.
[30] X. L. Li and P. Liang. Prefix-tuning: Optimizing continuous prompts for generation. In ACL-IJCNLP. pages 1582-459) 2021.
[31] Z. Li and D. Hoiem. Learning without forgetting. JEEE TPAMI, 40(12) 294-299999-997,----9 2017.
[32] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.
[33] Y. Liu and Tuytelaars. Residual tuning: Toward novel category discovery without labels. IEEE TNNLS, 2022.
[34] D. Lopez-Paz and M. Ranzato. Gradient episodic memory for continual learning. NeurlPS, 30, 2017.
[35] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In ICLR, 2018.
[36] Z. Luo, Y. Liu, B. Schiele, and Q. Sun. innsemnncremnttl exemplar compression for class caas-incrrmmnnal learning. In CVPR, pages 11371 11371-1155 11380, 2023.
[37] T. Malepathirana, D. Senanayake, and S. Halgamuge. Napa-vq: Neighborrhood-aware prototype augmenta- tion with vector quantization for continual learning. In CVPR, pages I 1674-11684, 2023.
[38] T. Martinetz, K. Schulten, et al. "neural-gas" network learns topologies. ANN, pages 397-402, 1991.
[39] M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer. insremeanrl learning: Survey and performance evaluation image classification. IEEE TPAMI, 2022.
[40] M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. 1989.
[41] G. L. Murphy and D. L. Medin. The role of theories in conceptual coherence. Psychological Review, 92 (3):289, 1985.
[42] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
[43] T. Ridnik, E. Ben-Bane han-Baruch, A. Noy, and L. Zelnik Manoo. Imagenet-211 pretraining for the masses. In NewrIPS, 2021.
[44] M. Riemer, I. Cases, R. Ajemian, M. Liu, Rish, Y. Tu, and G. Tesauro. Learning to lear without forgetting by maximizing transfer and minimizing interference. In ICLR, 2019.
[45] D. Rolnick, Ahuja. J. Schwarz, T. Lillicrap, and G. Wayne. Experience replay continual learning. NewrIPS, 32, 2019.
[46] O. Russakovsky, Deng, Su, J. Krause, S. Sathcesh, S. Ma. Z. Huang. A. Karpathy, A. Khosla, Bernstein, al. Imagenet large scale visual recognition challenge. IJCV, 115())211-252 2015.
[47] G. Saha, I. Garg, and K. Roy. Gradient projection memory for continual learning. In ICLR, 2021.
[48] J. Serra, D. Suris. M. Miron, and A. Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In ICML, pages 4548 4557, 2018.
12

---

[49] Y. Shi, K. Zhou, J. Liang, Z. Jiang. J. Feng. P. H. Torr, S. Bai, and V. Y. Tan. Mimicking the oracle: an initial phase decorrelation approach for class incremental learning. In CVPR. pages 16722-16731, 2022.
[50] J. S. Smith, I., Karlinsky. V. Gutta, P. Boniilte-Boonll,, D. Kim, A. Arbelle, R. Panda, R. Feris, and Z. Kira. Coda-prompt: Continual decomposed attention-basee prompting for rehearsal-free continual learning. In CVPR. pages 11909-11919, 2023.
[51] Y.-M. Tang, Y.-.. Peng, and W.-S. Zheng. When prompt-based incremental learning does meet strong pretraining. ICCV, pages 10161171,,200 2023.
[52] X. Tao, X. Chang, X. Hong. X. Wei, and Y. Gong. gopology-preserving class-incremental learning. In ECCV, pages 254-270, 2020.
[53] X. Tao, X. Hong. X. Chang, Dong, X. Wei, and Y. Gong. Few-shot class-incremental learning. In CVPR, pages 11183-12192, 2020.
[54] G. M.. Van de Ven and A. S. Tolias. Three scenarios for continual learning. arXiv preprini 0r11v::901:::::0:0:0 2019.
[55] A. Van Den Oord, O. Vinyals, al. Neural discrete representation learning. In NeurlPS, 2017.
[56] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkorcit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. NeurlPS, 30, 2017.
[57] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
[58] F-Y. Wang. D.-W. Zhou, L. Liu, H.-J Ye, Y. Bian, D.-C. Zhan, and P. Zhao. Beef: Bi-compatible class-incremental learning via energy-based expansion and fusion. In ICLR, 2023.
[59] L. Wang, J. Xie, X. Zhang, M. Huang, H. Su, and J. Zhu. Hicrarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality, In NeurIPS, 2023.
[60] Y. Wang. Z. Huang, and X. Hong. S-prompts leamning with pre-trained transformers: An occam's razor for domain incremental learning. NeurlPS, 2022.
[61] Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun. H. Zhang, C.-Y. Lec, X. Ren, G. Su, V. Perot, J. Dy, et al. Dualprompt: Complementary prompting rehearsal free continual learning. ECCV, pages 631-648, 2022.
[62] Z. Wang, Z. Zhang, C.-Y. Loe, Zhang, Sun, X. Ren, G. Su, V. Perot, J. Dy, and Pfister. Leaming prompt for continual learning. In CVPR, pages 139-149, 2022.
[63] Y. Wu, Y. Chen, L. Wang. Y. Ye, Z. Liu, Y. Guo, and Fu. Large scale incremental learning. CVPR, pages 374-382, 2019.
[64] Yoon, E. Yang. J. Lee, and S. Hwang. Lifelong learning with dynamically expandable networks. ICLR, 2018.
[65] G. Zeng. Y. Chen, Cui, and Yu. Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence, 1(8):364-37,, 2019.
[66] Zhai, J. Puigcerver, A. Kolesnikov, P. Ruyssen, Riquelme, Lucie, Djolonga, A. Pinto, M. Neumann, A. Dosovitskiy, et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint
2019.
[67] G. Zhang, L. Wang, G. Kang, L. Chen, and Y. Wei. Slca: Slow learner with classifier alignment for continual learning on a pre-trained model. In ICCV, 2023.
[68] J. Zhou, C. Wei, H. Wang. W. Shen, C. Xie, A. Yuille, and T. Kong. Image bert pre-training with online tokenizer. In ICTRR 2022.
13

---

A Appendix supplemental material
In this section, we provide detailed supplementary information. $A.1 outlines the evaluation metrics used to assess performance, providing detailed descriptions and formulas for clarity. $A.3 elabbraaes on the configurations of prompt-bases methods compared in our experiments. of.2r additional implementation details, including model training procedures, detailed parameter settings, and method- ology for obtaining results other methods compared in the experiments. Finally, $A.4 provides more results on two challenging datasets.
A.1 Evaluation Metrics
To assess the performance of continual learning, we record the average classification accuracy of all seen classes at the end of each task training, and denote the average accuracy the i-th task after learning the j-th task as A.j. The formal definitions of FAA and CAA are introduced as follows.
i) Final Average Accuracy (FAA) refers to the last average accuracy after learning all the tasks:
FAA T E A.T
(10)
where A is the average accuracy of task after learning task T, and T is the number of tasks. Larger FAA indicates greater learning capacity and less forgetting. FAA is also denoted as "Last-Ace".
ii) Cumulative Average Accuracy (CAA) is the average of historical FAA values after learning each task, which is calculated as:
1 CAA
Auj.
(11)
T
CAA reflects the overall performance after learning each incremental task, which can also be denoted as "Inc-Ace".
A.2 More Implementation Details
We use Adam W 35
with 31 0.9 and B2 = 0.999. Our batch size 64 for ImageNet-R, and 128 for Split CIFAR- 100 and Split CUB-200. We resize the input images to 224 x 224 and perform data transform following [50]. including random horizontal flip and normalization.
Following DualPrompt [6 and CODA-Prompt [50], we use 20% of the training data validation data, and perform hyperparameters tuning on it. After hyperparameter searching, we use a learning the hyperparameters
rate of 0.0025 for our method. For all other prompt- based methods,
following (50)
We search the values of prompt length Lp from 4 to 24 with step of 4. We search the number of prompt elements N in (10, 30, 50, 100). We found that a prompt length of 8 and 10 prompt elements already work fine. We insert prompts at the same locations all other implemented prompt-based methods in this paper, namely, the first 5 MSA blocks. A detailed comparison of the prompt configurations can be found in A.3
Finally, we run FT, FT++ L2P++, Deep L2P++, DualPrompt, and CODA ProD--promptt by using the official implementation provided by CODA-Prompy [50). We set the predicted logits for past task classes to be 0 to prevent gradients from flowing to the linear heads these classes. This is recommended by CODA-Prompt to improve the performance of these methods during code reproduction, as it could alleviate the bias towards new classes CIL. for rehearsal-free methods. For Prompt (59) and EvoPrompt 26). we reproduce the results using their respective official implementations.
A.3 Configurations of Prompt-based Methods
Table presents the configurations of the prompt-based continual learning methods compared in our experiment. Here, "Pro-T" denotes Prompt Tuning (28), and "Pre-T" denotes Prefix Tuning strategy R0 "Locations" indicates the MSA blocks to insert the prompts, N is the number of prompts/components in the prompt pool, and Lp is the length of a single prompt/compoent. L2P+
14

---

Table 6: Prompt configurations for prompt-based approaches in our experiments. See
A.3
Approuches
Strategy
Locations
Datasets
Hyperparumeteers
L2P++
Pre-T
30, L,
Deep L2P++
62
Pre-T
[0 1234]
All
N 30, L, 20
Pre-T
[0 []
All
G: N=1,, L, =6
Pool-Prompt
Pre-T
[2 4]
All
==10, Lp =p=20 20
CODA-P 50
Pre-T
[0 [344]
All
N 1000, Lp
ImageNet-R
N = 10, Lp 40
PiDeePrommt 59
Pre-T
(0.122 2.4]
Split CIFAR-100
== 1 10
Split CUB-200
- L, 40
EvoPrompt 26
Pro-T
[0 0023456789100
All
Input-cond, =p
VQ-Prompt (Ours)
Pre-T
[0 (011244)
All
10, L,
and Deep L2P++ are two variants of 1.2P for fair comparison. Specifically, L,2++ uses Pre-T instead of Pro-T prompting, and inserts the prompts to the first MSA block. Deep L.P++ is an extension of L2P++ with prompts incorporated into the same 5 MSA blocks as DualPrompt. For DualPrompt, "G" denotes the general prompt shared by all the tasks, and "E" denotes the expert prompt pool where only one of the elements is selected for certain query. As can be observed, our method requires fewer prompting parameters while consistently achieving superior or comparable performance across the benchmarks in continual learning.
A.4 More Experiment Results
This section presents results on two challenging datasets, namely ImageNet-A [5| and VTAB [66]. for evaluating continual learning methods based on pre-trained models. ImageNet-A contains adversarial images that fool current ImageNet pre-trained classifiers, while VTAB includes 19 datasets with diverse classes that do not overlap with ImageNer-IK, For ImageNet-A, we split the 200 classes into tasks. For VTAB, we sample five 10-class datasets from to construct cross-domain CII. setting. We used batch size of 64 for ImageNet-A and 8 for VTAB., with other training hyperparameters consistent with those used on other datasets. As shown in Table 7 our VQ-Prompt outperforms other SOTA methods such as HiDe-Promp when evaluated using FAA.
Table 7: Results evaluated using the FAA metric on the ImageNet-A and VTAB datasets. Backbones pre-trained ImageNet-IK. Larger values are better.
Method
ImageNet-A (1S)
VTAB (66]
HiDe HiDe-Prompt
559
51.67
86.38
VQ-Prompt
52.96
90.46
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations of the work in Section 6.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Method details included in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
The
code is available at
VQ-Prompt
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
ssiicyolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental settings are included in Section 5. Full details are in the code. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports the mean and standard deviations in tables with proper explanations and references.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper indicates the type of GPU and an estimation of the total compute.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The authors have reviewed the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Social impacts are discussed at the end of the paper.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release data or models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper properly cites the original paper that produced the code package or dataset. The license and terms of use explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Model details are included in the paper and the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "ChLGbOXfKb", "reviewer": "Reviewer_xyoq", "summary": "Prompt-based continual learning has emerged to address catastrophic forgetting in sequential task learning by using a pre-trained Vision Transformer (ViT) enhanced with learnable \"prompts.\" These prompts contain task-specific knowledge and guide the model in generating task-relevant outputs. During inference, the most suitable prompt is selected based on the input image. However, current methods face challenges due to non-differentiable indexing, leading to suboptimal prompt selection and performance issues. Discrete prompts offer better task knowledge representation and prevent interference among different tasks. This paper introduces Vector Quantization Prompting (VQ-Prompt) for continual learning, which optimizes prompts using task loss while maintaining their discrete properties. The method generates a continuous prompt and replaces it with the nearest match from a prompt pool, using gradient estimation to handle non-differentiability and additional regularization to refine the prompt pool. Representation statistics are used to mitigate classification bias towards previous tasks, improving continual learning performance.", "questions": "-", "limitations": "The authors addressed the limitations and broader impacts.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper develops a SoTA prompt-based continual learning method that improves prompt design with a widely used idea of vector quantization. The proposed methodology is sound and rational in design and outperforms other recent strong baselines.\n\nThe paper is easy to follow and provides a meaningful analysis to better understand the key components of the proposed approach.", "weaknesses": "The performance improvements are meaningful, but not significant - particularly on specific tasks, such as CIFAR-100 and CUB-200 benchmarks. \n\nThe technical contribution of the proposed method is not solid. Indeed I didn't find unique insightful ideas/analyses/novelty that I can get only from this paper.  Most technical components used in this work are somewhat general in vector quantization (e.g., NN look-up and straight-through gradient estimation) and continual learning literature (e.g.,post-adjustment of representation based on statistics). And this proposed method is more like a good combination of vector quantization idea for prompt design during continual learning.\n\nThe question of \"Why is the vector quantization idea more beneficial for prompt-based continual learning against other baselines\" is not sufficiently discussed. It achieves competitive performance with a smaller prompt length than other continuous-prompt methods. However, the paper does not provide any analysis/evidence as to why this happens, but only gives typical ablation studies.\n\n\nIn Table 4, the proposed method seems to have pros and cons for different self-supervised pre-training paradigms. It shows better cumulative accuracy (CAA) for on-par final accuracy (FAA); in my understanding, this means the proposed method shows a higher degree of forgetting compared to baselines, while showing better adaptation performance to new tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable and insightful comments. \n\n**W1. The performance improvements are meaningful, but not significant - particularly on specific tasks, such as CIFAR-100 and CUB-200 benchmarks.**\n\nWe respectfully disagree with the assessment that the improvements are not significant. As demonstrated in Table 1, our method achieves a relative improvement of 2.80%, 2.45%, and 4.96% over the second-best method, EvoPrompt, on the 5-task, 10-task, and 20-task settings of ImageNet-R using the FAA metric. ImageNet-R is widely recognized as a critical benchmark for continual learning, especially for methods utilizing backbones pre-trained on the ImageNet dataset, as discussed in DualPrompt.\n\nAdditionally, while our improvements on CIFAR-100 (Table 2) and CUB-200 (Table 3) are not as large as those on ImageNet-R, they are still considerable. On CUB-200, although we slightly trail behind SLCA in the CAA metric, it is important to note that SLCA does not freeze the backbone and employs a higher number of trainable parameters, allowing for better adaptation to the downstream task of fine-grained classification. \n\nThe differences in performance improvements might be attributed to the inherent characteristics of the datasets. CIFAR-100 and CUB-200 have data distributions that are closer to the original ImageNet distribution compared to ImageNet-R, which contains artistic renditions of ImageNet categories that are discouraged by the collecting rules of the original ImageNet. This closer alignment with ImageNet may reduce the relative improvement margins on CIFAR-100 and CUB-200.\n\n**W2. The technical contribution of the proposed method is not solid. Indeed I didn't find unique insightful ideas/analyses/novelty that I can get only from this paper. Most technical components used in this work are somewhat general in vector quantization (e.g., NN look-up and straight-through gradient estimation) and continual learning literature (e.g.,post-adjustment of representation based on statistics). And this proposed method is more like a good combination of vector quantization idea for prompt design during continual learning.**\n\nThanks for the comment. Our method is motivated by identifying a clear disadvantage in current prompt-based methods, i.e., the unoptimized selection of prompts. We propose a simple yet effective approach to address this issue by integrating vector quantization with prompt-based continual learning, which has not been explored before. While our technical components are based on established principles in vector quantization and continual learning, the innovation lies in the unique application and combination of these techniques to solve the specific problem of prompt selection. By incorporating vector quantization, we optimize the prompt selection process with task loss to achieve a more effective abstraction of task knowledge, which is crucial for continual learning. Experimental results demonstrate the effectiveness of our approach. Thus, while we build upon existing techniques, our work provides a fresh perspective and a practical solution to a challenge in prompt-based continual learning. \n\n**W3. The question of \"Why is the vector quantization idea more beneficial for prompt-based continual learning against other baselines\" is not sufficiently discussed. It achieves competitive performance with a smaller prompt length than other continuous-prompt methods. However, the paper does not provide any analysis/evidence as to why this happens, but only gives typical ablation studies.**\n\nThanks for the insightful comment. The use of vector quantization (VQ) in our method provides several key benefits for prompt-based continual learning. First, VQ allows us to encode task knowledge into discrete prompts, which provide a more compact representation compared to continuous prompts, as discussed between lines 50 and 61 of the manuscript. This discrete nature helps in capturing essential task-specific features with necessary abstraction compared with continuous representations. Second, by integrating VQ into the prompt-based framework, we enable the optimization of prompt selection with task loss. This end-to-end optimization ensures that the selected prompts are highly relevant to the task at hand, thereby enhancing the task knowledge learning of the prompts. In summary, we can use a smaller prompt length to achieve competitive performance due to effective end-to-end knowledge learning. We will extend our analysis in the revised manuscript by providing detailed explanations. \n\n**W4. In Table 4, the proposed method seems to have pros and cons for different self-supervised pre-training paradigms. It shows better cumulative accuracy (CAA) for on-par final accuracy (FAA); in my understanding, this means the proposed method shows a higher degree of forgetting compared to baselines, while showing better adaptation performance to new tasks.**\n\nThanks for the detailed observation. The higher CAA suggests that VQ-Prompt is more efficient in leveraging past knowledge to aid current tasks, even though it might exhibit a higher degree of forgetting compared to some baselines. This trade-off between adaptation to new tasks and forgetting of previous ones is a common challenge in continual learning. With self-supervised pre-training paradigms, our approach tends to prioritize adaptability to new tasks to ensure that the model remains relevant and effective in dynamic environments. In our revised manuscript, we will provide additional analysis to elucidate this trade-off and further highlight the strengths and limitations of VQ-Prompt in various self-supervised pre-training scenarios.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oZIk2StdzE", "reviewer": "Reviewer_7MCa", "summary": "This paper introduces Vector Quantization Prompting (VQ-Prompt), a novel prompt-based method designed to mitigate catastrophic forgetting in the sequential learning scenario of Class Incremental Learning (CIL). VQ-Prompt utilizes Vector Quantization (VQ) to facilitate end-to-end learning with a set of discrete prompts. Initially, it computes the queries for the input images and determines similarity scores between the query and keys, similar to the Learning to Prompt (L2P) approach. This involves using a pretraining image method to generate queries. Subsequently, it aggregates prompts based on these similarity scores, resulting in what is called a continuous prompt. This aggregated prompt is then quantized by selecting the nearest neighbor (NN) from the prompt pool. Then, the pipeline to classify the instances remains constant compared to L2P. To ensure gradient propagation through the key and all prompts, VQ-Prompt employs the straight-through estimator along with similarity scores. Additionally, it incorporates representation statistics to address classification bias on the classifier. The authors demonstrate that VQ-Prompt outperforms state-of-the-art (SOTA) baselines across three datasets: ImageNet-R (5, 10, and 20 tasks), CIFAR-100 (10 tasks), and CUB-200 (10 tasks).", "questions": "Please refer to the weaknesses section, I left some questions for you.", "limitations": "The authors mention a limitation of this method: scaling up the prompt size does not increase performance. However, this is not necessarily a limitation. This could indicate that the model does not need significant parameters to learn the task because it is relatively easy due to its initial knowledge. (Please see point 1 of the weaknesses).", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper presents the following strengths: \n\n1. The paper is well-written and organized. It is easy to follow. \n2. The problem studied is highly relevant and valuable. \n3. The paper proposes a method that enables end-to-end training for architecture based on prompt learning for CIL and shows that this can also be beneficial in achieving state-of-the-art results in three well-known datasets.", "weaknesses": "1. Considering Prompt Learning for Continual Learning (CL) assumes that there is a good and generalizable pretraining that could be transferable (tuned) to the different downstream tasks. Moreover, considering the way that the queries are generated, it also presumes that this pretraining can accurately select prompts for all tasks. I would like to see results on more challenging tasks like out-of-domain. Although the authors perform the evaluation in different scenarios, two of them, ImageNet-R and CIFAR-100, are closely related to ImageNet. For instance, ImageNet-R is derived from ImageNet-1K, and also its classes overlap with ImageNet-1K.\n2. Additionally, it is worth noting that the authors use a larger pretraining dataset for the experiments in Table 3, where VQ-Prompt achieves nearly the same performance as HiDe-Prompt. This raises the question of whether this is related to the initialization issue mentioned earlier. I would like to see these results with a different initialization to understand this relationship better.\n3. There is no consistency in the evaluation of state-of-the-art (SOTA) methods across all tasks (Table 1, Table 2, Table 3), making comparison and analysis challenging. For instance, HiDe-Prompt shows a significant improvement over CODA-P and almost matches the performance of VQ-Prompt. So, why is it not evaluated on the other tasks? I would like to see how HiDe-Prompt performs on the other tasks presented in Table 1 and Table 2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable and insightful comments. \n\n**W1. Considering Prompt Learning for Continual Learning (CL) assumes that there is a good and generalizable pretraining that could be transferable (tuned) to the different downstream tasks. Moreover, considering the way that the queries are generated, it also presumes that this pretraining can accurately select prompts for all tasks. I would like to see results on more challenging tasks like out-of-domain. Although the authors perform the evaluation in different scenarios, two of them, ImageNet-R and CIFAR-100, are closely related to ImageNet. For instance, ImageNet-R is derived from ImageNet-1K, and also its classes overlap with ImageNet-1K.**\n\nPlease note that ImageNet-R is widely recognized as a critical benchmark for CL, especially for methods utilizing backbones pre-trained on the ImageNet, as discussed in DualPrompt. This is because ImageNet-R contains artistic renditions of ImageNet categories that are discouraged by the collecting rules of the original ImageNet. CIFAR-100 has also been widely used in previous prompt-based CL methods. \n\nFollowing your suggestion, we conducted experiments on two new datasets, namely ImageNet-A [1] and VTAB [2]. ImageNet-A contains adversarial images that fool current ImageNet pre-trained classiﬁers, while VTAB includes 19 datasets with diverse classes that do not overlap with ImageNet-1K. For ImageNet-A, we split the 200 classes into 20 tasks. For VTAB, we sample five 10-class datasets from it to construct the cross-domain CIL setting. As shown below, our VQ-Prompt outperforms other SOTA methods such as HiDe-Prompt on these challenging datasets on the FAA metric. \n\n\n||ImageNet-A|VTAB|\n|-|-|-| \n|HiDe-Prompt|51.67|86.38| \n|VQ-Prompt|52.96|90.46| \n\n\n[1] Natural adversarial examples. CVPR21\n\n[2] A large-scale study of representation learning with the visual task adaptation benchmark. arXiv19\n\n**W2. Additionally, it is worth noting that the authors use a larger pretraining dataset for the experiments in Table 3, where VQ-Prompt achieves nearly the same performance as HiDe-Prompt. This raises the question of whether this is related to the initialization issue mentioned earlier. I would like to see these results with a different initialization to understand this relationship better.**\n\nIt is important to clarify that the results in Table 3 for both VQ-Prompt and HiDe-Prompt use the SAME pre-training dataset, i.e., ImageNet-21K, as denoted in the caption. Our method achieves superior performance compared to HiDe-Prompt, particularly in the CAA metric. \n\nRegarding the initialization with different dataset sizes, besides the results of pre-training on ImageNet-21K shown in Table 3, we present additional comparisons with HiDe-Prompt on ImageNet-R with backbones pre-trained on ImageNet-1K. The results for HiDe-Prompt are obtained by running its official repository. As shown in the table below, our VQ-Prompt consistently outperforms HiDe-Prompt in the 5-task, 10-task, and 20-task settings on ImageNet-R.\n\n\n||5-task FAA|5-task CAA|10-task FAA|10-task CAA|20-task FAA|20-task CAA|\n|-|-|-|-|-|-|-|\n|CODA-P|76.51$\\pm$0.38|82.04$\\pm$0.54|75.45$\\pm$0.56|81.59$\\pm$0.82|72.37$\\pm$1.19|79.88$\\pm$1.06|\n|HiDe-Prompt|76.29$\\pm$0.10|78.77$\\pm$0.11|76.74$\\pm$0.18|78.76$\\pm$0.11|76.46$\\pm$0.06|78.76$\\pm$0.11|\n|VQ-Prompt|79.32$\\pm$0.29|82.96$\\pm$0.50|78.71$\\pm$0.22|83.24$\\pm$0.68|78.10$\\pm$0.22|82.70$\\pm$1.16|\n\n\nMoreover, we have conducted experiments with different initializations originating from various self-supervised pre-training paradigms, as shown in Section 5.2 and Table 4. The findings demonstrate that VQ-Prompt consistently achieves competitive performance across different initializations, highlighting its robustness and adaptability. \n\n**W3. There is no consistency in the evaluation of state-of-the-art (SOTA) methods across all tasks (Table 1, Table 2, Table 3), making comparison and analysis challenging. For instance, HiDe-Prompt shows a significant improvement over CODA-P and almost matches the performance of VQ-Prompt. So, why is it not evaluated on the other tasks? I would like to see how HiDe-Prompt performs on the other tasks presented in Table 1 and Table 2.**\n\nWe would like to clarify that results in Tables 1, 2 and 3 are taken from previous publications, which utilize backbones pre-trained on different datasets. For example, HiDe-Prompt in Table 3 uses weights pre-trained on ImageNet-21K, whereas methods in Tables 1 and 2 are pre-trained on ImageNet-1K. This difference in pre-training datasets impedes direct comparisons across these tables.\n\nFollowing your suggestion, we ran the official code of HiDe-Prompt and obtained its performance with a backbone pre-trained on ImageNet-1K. The results on ImageNet-R are shown in the **Table in the response to W2**. As can be observed, HiDe-Prompt achieves inferior results compared to our method on ImageNet-R. Unfortunately, we could not provide results for CIFAR-100 because we failed to obtain reasonable results with the settings in the paper (bs=128 and lr=0.005). We found that the actual learning rate changes with the batch size. We are exploring other training settings. \n\n**The authors mention a limitation of this method: scaling up the prompt size does not increase performance. However, this is not necessarily a limitation. This could indicate that the model does not need significant parameters to learn the task because it is relatively easy due to its initial knowledge. (Please see point 1 of the weaknesses).**\n\nWe agree with the reviewer's perspective that this could also be seen as a strength of our method. Our model does not require significant parameters to learn tasks due to its effective end-to-end knowledge learning achieved by integrating VQ into the prompt-based framework. We will revise our manuscript to provide a more in-depth discussion regarding the scalability of our approach.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D5Q5OalY4h", "reviewer": "Reviewer_GsNc", "summary": "The representations learned by large pre-trained models have led to many Continual Learning approaches based on these models. Specifically, prompt-based approaches train a small set of learnable parameters (prompts) to guide a fixed pre-trained model for a particular task. One key component in these approaches is the selection of relevant prompts, which helps to specialise the guidance depending on the need for the corresponding input. Most previous works select the relevant prompt based on the similarity of the input and a key in the representation space of the same visual pre-trained model. The key is commonly connected to the prompt via a key-value data structure, making it unfeasible to traceable optimisation sequence and infeasible to optimise with task loss. In this paper, the authors propose Vector Quantization Prompting, which incorporates vector quantisation into an end-to-end training process, including the keys and prompts. By using a look-up NN over a weighted sum of all the prompts and a straight-through estimator to approximate the gradient for the prompts and keys, the authors were able to optimise not only the prompts but also the key when selecting relevant prompts, which, as shown in the results, can help achieve better performance in diverse benchmarks.", "questions": "- In line 195, it is mentioned that this selection allows more updates in relevant prompts without disrupting less relevant ones. This idea is very interesting and crucial in CL. However, some questions remain:\n    - Concerning this point, how different is a traditional prompt-based approach from the proposal? In both cases, you only update the “relevant” prompt, but the meaning of relevant changes. Or is there something else?\n    - The look-up table selects the most relevant prompt; however, we should be able to compose concepts to learn more abstract ideas. Can this proposal be extended to this?\n- There is a second training phase when applying the component described in Section 4.3. How many epochs does this new phase have? Could it be an unfair advantage over other methods? For example, how does L2P behave with this component?\n- The experiments done in 5.3 concerning the temperature of alpha suggest that a lower value leads to a sharper distribution, which can lead to a more discrete selection; however, a higher value achieves better performance. How can we read this, as it seems contrary to the need for discrete prompts?", "limitations": "The authors present some limitations and concerns regarding the proposal. Another limitation that can be added is the intrinsic limitation in the pre-trained models concerning the limitations of the pre-trained data distribution and the computational cost of running these models.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- The authors motivate the proposal by pointing out a clear disadvantage of current prompt-based methods: the unoptimised selection of prompts. This paper presents a creative and simple approach to tackling this issue, which also performs better, as shown in the results.\n- The paper is well-written and structured to help readers understand the problem and the authors' proposed solution. Although the solutions lack some intuitions and explanations concerning the reasons for the comparisons and ablations, this could be because of the lack of space.", "weaknesses": "- One of the authors' motivations is learning discrete prompts, which they even mentioned as a contribution. However, even after reading the explanation between lines 50 and 61, the contribution of having discrete prompts is unclear. I understand the motivation of having concrete concepts represented as discrete vectors for human understanding, but a continual vector can easily represent the same concept and also be distinct categories and even linearly separable.\n    - Furthermore, the implementation of discrete prompts in the proposal must be clarified. While I assume it occurs in the NN look-up or the prompt formation, there is a lack of ablation on this point. For instance, how is prompt usage distributed (Are they uniformly distributed, or are some very specialised)? Are there specialised prompts? Is there a discernible relationship between a 'concept' and a prompt?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable and insightful comments. \n\n**W1. One of the authors' motivations is learning discrete prompts, which they even mentioned as a contribution. However, even after reading the explanation between lines 50 and 61, the contribution of having discrete prompts is unclear. I understand the motivation of having concrete concepts represented as discrete vectors for human understanding, but a continual vector can easily represent the same concept and also be distinct categories and even linearly separable.**\n\nWe would like to clarify that while we refer to them as discrete, the values of these discrete prompts are indeed float (continuous) during optimization. They are distinct from continuous prompts in their conceptual structure and function.\n\nContinuous prompts typically refer to input-conditioned prompts where each input sample has a unique prompt, even if the samples are instances of the same concept. This approach can lead to redundancies in concept representation. In contrast, discrete prompts use a limited set of prompts to represent concepts, which can serve as a more abstract representation of knowledge and align more closely with human cognitive structures. This form of knowledge representation can enhance the model's ability to learn and manage task-specific knowledge efficiently.\n\n**W2. The implementation of discrete prompts in the proposal must be clarified. While I assume it occurs in the NN look-up or the prompt formation, there is a lack of ablation on this point. For instance, how is prompt usage distributed (Are they uniformly distributed, or are some very specialised)? Are there specialised prompts? Is there a discernible relationship between a 'concept' and a prompt?**\n\nRegarding prompt usage distribution, we have observed some prompts are more frequently selected for samples from different tasks while others are less frequently used. Since the prompts are designed to learn task-specific knowledge, each task might not have a universal ‘concept’. However, different tasks might share some features that can be captured by certain prompts. We will further clarify these observations and provide a more detailed analysis in the revised paper.\n\n**Q1. In line 195, it is mentioned that this selection allows more updates in relevant prompts without disrupting less relevant ones. This idea is very interesting and crucial in CL. However, some questions remain:**\n- **How different is a traditional prompt-based approach from the proposal? In both cases, you only update the “relevant” prompt, but the meaning of relevant changes. Or is there something else?**\n\nThe discussion in line 195 aims to explain how our design can enable CL. How to effectively select and update relevant prompt(s) is crucial for achieving CL, and the meaning of “relevant” in our context shares the same insight as traditional prompt-based methods.  However, the primary difference lies in whether the prompt selection can be optimized with the task loss. \n\n- **The look-up table selects the most relevant prompt; however, we should be able to compose concepts to learn more abstract ideas. Can this proposal be extended to this?**\n\nOne possible way is to design a two-level prompt pool, with each level responsible for a different degree of abstraction. The model can first identify relevant prompts from the pool of lower abstraction. Next, it learns the combination weights to form a more abstract prompt. This newly composed prompt is then matched with a prompt from the higher abstraction pool, and the matched one is used for further processing.\n\n**Q2. There is a second training phase when applying the component described in Section 4.3. How many epochs does this new phase have? Could it be an unfair advantage over other methods? For example, how does L2P behave with this component?**\n\nTen epochs. This phase stabilizes task knowledge learning by leveraging representation statistics of old data, which can contribute to performance improvements but is not the sole determinant of the final performance. \n\nFollowing your suggestion, we trained the L2P++ with this component. As can be observed in the following table, L2P++ with representation statistics (“L2P++ V2”) performs better than the original L2P ++ in the three settings on ImageNet-R, but remains inferior to our method. Additionally, we list the performance of VQ-Prompt-S, a simplified version of VQ-Prompt that does not use representation statistics. VQ-Prompt-S still outperforms other methods such as EvoPrompt, demonstrating the effectiveness of our approach.\n||5-task FAA|5-task CAA|10-task FAA|10-task CAA|20-task FAA|20-task CAA|\n|-|-|-|-|-|-|-|\n|L2P++|70.83$\\pm$0.58|78.34$\\pm$0.47|69.29$\\pm$0.73|78.30$\\pm$0.69|65.89$\\pm$1.30|77.15$\\pm$0.65|\n|L2P++ V2|74.11$\\pm$0.08|78.44$\\pm$0.63|72.93$\\pm$0.27|78.63$\\pm$0.80|70.99$\\pm$0.26|77.65$\\pm$0.79|\n|EvoPrompt|77.16$\\pm$0.18|82.22$\\pm$0.54|76.83$\\pm$0.08|82.09$\\pm$0.68|74.41$\\pm$0.23|80.96$\\pm$1.42|\n|VQ-Prompt-S|78.52$\\pm$0.34|82.64$\\pm$0.68|78.00$\\pm$0.39|82.83$\\pm$0.69|76.19$\\pm$0.26|81.68$\\pm$1.02|\n\n**Q3. The experiments done in 5.3 concerning the temperature of alpha suggest that a lower value leads to a sharper distribution, which can lead to a more discrete selection; however, a higher value achieves better performance. How can we read this, as it seems contrary to the need for discrete prompts?**\n\nThis can be understood by considering the balance between discrete selection and robust learning. While a lower temperature leads to a sharper distribution and more discrete selections, it can lead to overly aggressive prompt choices early in training when the model is less fully trained. This can negatively impact the learning process and overall performance, but it does not negate the value of discrete prompts themselves.\n\n**Another limitation.**\n\nThanks for the insightful suggestion. We will include a discussion of the intrinsic limitations of pre-trained models in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8lJN5RqVSh", "reviewer": "Reviewer_EHaW", "summary": "This paper presents VQ-Prompt, a prompt-based continual learning method using Vector Quantization (VQ) to enhance task knowledge representation and overcome catastrophic forgetting. VQ-Prompt incorporates VQ into the end-to-end training of discrete prompts, optimizing the prompt selection process with task loss and achieving effective task knowledge abstraction. Extensive experiments demonstrate that VQ-Prompt outperforms state-of-the-art methods in various benchmarks, particularly in class-incremental settings.", "questions": "1. In Fig. 3(a), temperature $\\tau=1$ achieves the best performance, then what about the temperature larger than 1?", "limitations": "NA", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written and easy to understand.\n\n2. Introducing VQ to the prompt CL is intuitive and can end-to-end train these prompts.\n\n3. Experiments have demonstrated the effectiveness of the proposed method.", "weaknesses": "1. There is no constraint when calculating the similarity score \\alpha and prompt key K, which may lead to corrupt prompt learning, i.e., for most test samples from different tasks, the selected prompts are similar, and large parts of prompts are useless.\n\n2. Lack of ablation study on the fine-tuned classifier. Fine-tune classifier is a trick to improve the classifier performance, which can also be applied to other methods. It is unfair to directly compare with other methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable and insightful comments. \n\n**W1. There is no constraint when calculating the similarity score \\alpha and prompt key K, which may lead to corrupt prompt learning, i.e., for most test samples from different tasks, the selected prompts are similar, and large parts of prompts are useless.**\n\nThanks for the insightful comment. We have observed that at certain layers, several prompts are more frequently selected for test samples from different tasks, while others are less frequently used. To alleviate this, one possible strategy is to introduce constraints for prompt selection, such as excluding prompts that have already been frequently used by previously learned tasks to enhance the diversity and utility of the prompts. We will explore it as future work.\n\n**W2. Lack of ablation study on the fine-tuned classifier. Fine-tune classifier is a trick to improve the classifier performance, which can also be applied to other methods. It is unfair to directly compare with other methods.**\n\nIn Section 5.3, we show the performance of “VQ-Prompt-S”, a simplified version of VQ-Prompt that does not use representation statistics to mitigate the negative effect of classifier bias developed during continual learning. VQ-Prompt-S achieves an FAA value of 78.05 (single run) on 10-task ImageNet-R, which is superior than the second best method EvoPrompt with an FAA of 77.16. \n\nTo address your concerns, we trained another method L2P++ with this component. The results are shown in the following table. As can be observed, L2P++ with representation statistics (“L2P++ V2”) performs better than the original L2P ++ in the “5-task”, “10-task”, and “20-task” settings on ImageNet-R, but remains inferior to our method. Further, our VQ-Prompt-S outperforms other methods such as EvoPrompt (AAAI24), demonstrating the effectiveness of our approach. The results show that while representation statistics can contribute to performance improvements,  they are not the sole determinant of the final performance. \n\n||5-task FAA|5-task CAA|10-task FAA|10-task CAA|20-task FAA|20-task CAA|\n|-|-|-|-|-|-|-|\n| L2P++       | 70.83 $\\pm$ 0.58 | 78.34 $\\pm$ 0.47 | 69.29 $\\pm$ 0.73 | 78.30 $\\pm$ 0.69 | 65.89 $\\pm$ 1.30 | 77.15 $\\pm$ 0.65 |\n| L2P++ V2    | 74.11 $\\pm$ 0.08 | 78.44 $\\pm$ 0.63 | 72.93 $\\pm$ 0.27 | 78.63 $\\pm$ 0.80 | 70.99 $\\pm$ 0.26 | 77.65 $\\pm$ 0.79 |\n| EvoPrompt   | 77.16 $\\pm$ 0.18 | 82.22 $\\pm$ 0.54 | 76.83 $\\pm$ 0.08 | 82.09 $\\pm$ 0.68 | 74.41 $\\pm$ 0.23 | 80.96 $\\pm$ 1.42 |\n| VQ-Prompt-S | 78.52 $\\pm$ 0.34 | 82.64 $\\pm$ 0.68 | 78.00 $\\pm$ 0.39 | 82.83 $\\pm$ 0.69 | 76.19 $\\pm$ 0.26 | 81.68 $\\pm$ 1.02 |\n| VQ-Prompt   | 79.32 $\\pm$ 0.29 | 82.96 $\\pm$ 0.50 | 78.71 $\\pm$ 0.22 | 83.24 $\\pm$ 0.68 | 78.10 $\\pm$ 0.22 | 82.70 $\\pm$ 1.16 |\n\n**Q1.  In Fig. 3(a), temperature $\\tau=1$ achieves the best performance, then what about the temperature larger than 1?**\n\nFollowing the suggestion, we conducted additional experiments regarding temperatures larger than 1. The results are shown in the following table. As can be observed, $\\tau=5$ achieves a result comparable to $\\tau=1$. However, as $\\tau$ increases, the performance decreases. These findings indicate that a temperature of 1 yields the best performance for our method.\n\n|$\\tau=1$|$\\tau=5$|$\\tau=10$|$\\tau=50$|$\\tau=100$|\n|-|-|-|-|-|\n|77.15|77.12|76.89|76.13|76.06|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: I agree with the unanimous decision of the reviewers to accept the paper. However, there has been some discussion regarding its presentation. I strongly encourage the authors to incorporate the reviewers' feedback and include the new results in the camera-ready version of the paper.
Decision: Accept (poster)