Paper: Knowledge Accumulating Contrastive Prompt for Continual Learning (ICLR-2024)

Reviewer comment (Novelty):
Despite the clear motivation, the proposed method only marginally outperforms the relatively earlier baseline in this direction, i.e., DualPrompt (ECCV22). There have been many more recent work based on constructing a prompt pool, e.g., CODA-Prompt (CVPR’23) and HiDe-Prompt (NeurIPS’23). I'm concerned about whether there is enough room for further development of the core insight of this paper, i.e., learning all tasks in a single prompt. Although I appreciate that the proposed method uses less parameters than L2P and DualPrompt, the improvement seems to be less significant because the prompts are light-weight.

Author response:
>**Q1. The reviewer is concerned about the further improvement of the core insight of this paper, i.e., learning all tasks in a single prompt.**

As the reviewer comments, we suggest the view that the single prompt has enough capacity to capture the knowledge of all tasks, which is verified by the upper-bound prompt $p^*$. 

According to the reviewer's concern, we highlight the accuracy gap between the upper-bound model and the continual learning methods, as shown in the Table 1 of main paper. We believe that the single prompt can further tighten the gap, as the existence of the optimal point within the parameter space is already provided. Moreover, the accumulation of the task knowledge improves the generalizability of the model, as suggested in [1].

Moreover, our work is beneficial for task-agnostic setting, since using the single prompt allows the overlap between the tasks. In contrast, prompt pool stores the task-specific prompts, which requires the explicit task boundary.

[1]Wang, Qin, et al. "Continual test-time domain adaptation." in CVPR 2022.

Reviewer comment (Evaluation):
In addition to the results of parameter cost and inference cost, I would encourage the authors to further compare their training cost, as the use of contrastive learning usually requires more computation in training phase.

Author response:
>**Q2. Comparison for the training cost is required, as the method utilizes contrastive learning which may demand significant computation.**

Following to the Reviewer's comment, we compare the training time per iteration for the validation sets of all tasks, as shown in Table 9. 

For the training, our method has the similar cost with the prompt selection methods. The previous methods computes the token feature from the seperate frozen model, which is used to train the prompt key. Similarly, we compute the features from the previous model which is frozen during the current task. Moreover, our method utilizes the positive-only contrastive loss, which avoids the additional computational burden by the negative pairs.

**Table 9**. Comparison of training time for Split ImageNet-R.


|    | Ours|DualPrompt|L2P|
|----|----|----|----|
|Training(s/iter.)|0.979|0.981|1.293|

Reviewer comment (Experiments):
The formulation of $L_{ctr}$ and $L_{prev}$ are very similar except that different W are used. It is unclear to me which part plays a more important role in the method. It will be great more discussions and experiments on how $\lambda_{ctr}$ and $\lambda_{prev}$ affect the model performance on different benchmark are provided to help better understand these two losses.

Author response:
>**Q2.More discussions and experiments for each loss term, $L_{ctr}$ and  $L_{prev}$.**
 
As introduced in the main paper, $L_{ctr}$  captures the embedding space to pull the defined positive pairs, and $L_{prev}$ aligns the feature space between the old model and the current model.
Specifically for $L_{prev}$, we kindly remind the plasticity-stability tradeoff suggested in the section 6 of the main paper. $L_{prev}$ enhances the alignment of the feature space, which contributes to the stability of the model for the continual learning.

Following to the reviewer's comment, we present the additional ablation study for the loss term, as follows:

**Table8.** Additional ablation study for loss terms
| $\lambda_{ctr}$ | 1 | 1 | 1 | 1 | 2 | 2 | 2 | 0.5 | 0.5 | 0.5 |
|--- |--- |--- |--- |--- | --- | --- | --- | --- | --- |---|
| $\lambda_{prev}$ | 1 | X | 0.1 | 0.3 | X | 0.1 | 0.3 | X | 0.1 | 0.3 |
| **Acc.**  | 69.98 | 69.59 | 69.85 | 69.56 | 69.20 | 69.28 | 68.45 | 69.97 | 69.80 | 69.44 |
| **Forget.** | 4.24 | 5.35 | 4.88 | 3.77 | 4.11 | 3.97 | 3.96 | 5.34 | 5.46 | 4.30 |


The result show that increasing $L_{prev}$ supports that the loss term alleviates the forgetting by enhancing the stability of the model. In case of $L_{ctr}$, the loss term contributes to the alleviated forgetting, however, increased $\lambda_{ctr}$ lowers the accuracy, suggesting degraded stability-plasticity tradeoff.

Reviewer comment (Experiments):
The contrastive learning is novel but in compared to the recent work [1,5] paper does not shows the SOTA results. The recent prompting based baselines shows much better result but are missing in the paper.

Author response:
>**Comment 1. The reviewer suggested the detailed comparison with the recent related works, including S-Prompt and CODA-Prompt.**

Thanks to the reviewer's fruitful comment, we recognize the HiDe-prompt which is a very recent work (11 October, 2023 Arxiv and accepted for NeurIPS 2023) publicly released after the submission of our paper (28 September, 2023). We clarify that we could not recognize HiDe-prompt during our research. 

In response to the reviewer’s comment, We provide the comparison with the related work, as follows:

**Table 5.** Comparison with related works. 
|  | Ours | S-prompt | CODA-prompt | 
|---|----|---|---|
|Acc. |85.47|80.13|82.72|
|Forget.|4.16|9.19|10.26|

Reviewer comment (Experiments):
The approach in [1] seems to outperform the proposed approach. One justification can be the the approach requires two passes through the ViT during inference: one pass with the old prompt and another with the new prompt (referred to as ensemble in the paper). However, without the ensemble also, the approach seems to perform better as can be inferred from table 3 of their paper. Similarly, look the other work [4,5].

Author response:
>**Comment 1. The reviewer suggested the detailed comparison with the recent related works, including S-Prompt and CODA-Prompt.**

Thanks to the reviewer's fruitful comment, we recognize the HiDe-prompt which is a very recent work (11 October, 2023 Arxiv and accepted for NeurIPS 2023) publicly released after the submission of our paper (28 September, 2023). We clarify that we could not recognize HiDe-prompt during our research. 

In response to the reviewer’s comment, We provide the comparison with the related work, as follows:

**Table 5.** Comparison with related works. 
|  | Ours | S-prompt | CODA-prompt | 
|---|----|---|---|
|Acc. |85.47|80.13|82.72|
|Forget.|4.16|9.19|10.26|

Reviewer comment (Evaluation):
The paper mentions that the prompt selection mechanism in l2p and dual-prompt can introduce mis-alignment of prompts. I am a bit curious as to how much mis-alignment does each of the approaches have?

Author response:
>**Comment 4. The reviewer commented the change of performance by the prompt length**

We appreciate for the constructive discussion. The reviewer is kindly reminded that we have presented the effect of misalignment of the prompt in the Table 3 in the main paper. As DualPrompt predicts the task identity to select the task-specific prompt, we calculate the accuracy of the task prediction, denoted as `Task ID Accuracy'. The accuracy is below 60$\%$, and the accuracy degrades as the number of the tasks increases.
In case of L2P, the prompt index is trained by the diversity regularization, without the ground-truth index. Accordingly, the accuracy for task prediction cannot be defined. Instead, we suggest the result of task-agnostic setting(e.g. G-CIFAR100) in Table 1 of the main paper. The lower performance of L2P compared to our method indirectly indicates the misaligned prompt problem of L2P.

Reviewer comment (Experiments):
The comparison in this paper is insufficient, as it does not compare their method to the CODA-Prompt [1]. Different from most existing prompt-based incremental learning methods, the authors use the same prompt for different sessions. To support the technique selection contrary to most existing methods, more comparison is essential.

Author response:
>**Comment2. The comparisons with recent related works, including CODA-Prompt are required.**

In response to the reviewer's comment we present the result of the CODA-prompt by our implementation, as follows:

**Table2.** Comparison with related approaches by 10-split-CIFAR100. ViT-B/16 model pre-trained on ImageNet-21k is used. We set learning rate and the scheduler identical to the main paper, for fair comparison.

| Sup-21k   | Upper-bound | Ours |  CODA-Prompt |S-Prompt| HiDe-prompt | 
|----|----|----|---|---|---|
| Acc. | 90.85 | 85.47 | 82.72|80.13|92.61|
| Forget. |-|4.16|10.26|9.19|3.16|

Thanks to the reviewer's fruitful comment, we recognize the HiDe-prompt which is a very recent work(11 October, 2023 Arxiv and accepted for NeurIPS 2023) publicly released after the submission of our paper(28 September, 2023). We clarify that we could not recognize HiDe-prompt during our research. 

We provide the conceptual comparison with HiDe-prompt, which is summarized as follows:
- HiDe-prompt is based on the prompt selection process and prompt pool.
- Contrastive loss with negative pairs is suggested to remove the instance-wise and task-wise overlaps, which is different motivation with our method.
- Additional process and buffer memory are required to obtain old-task statistics.
- Not applicable for smooth transition of data distribution.

First, HiDe-prompt propose to improve the prompt selection process for the prompt pool, by the uninstructed features. Second, the method proposes the contrastive learning to reduce the task-wise overlap, which requires negative pairs. Positive pairs $(h, \mu_c)$ are presented, where $h$ is feature extracted by the prompt and $\mu_c$ is corresponding class prototype of the current task. Also, negative pairs $(h,h')$ and $ \lbrace \(h,\mu_{c,i} \) \rbrace_{i=1}^{t-1}$are defined to minimize the overlaps between the instances and tasks.

Third, the method utilizes the old-task statistics, which should be calculated for each training stage. It requires additional process and the memory. Lastly, the method requires the explicit task boundary which is not provided for the continual learning with smooth transition of the data distribution(i.e. G-CIFAR100 dataset). In contrast, our method is applicable for the task-agnostic dataset, as shown in Table 1 in the main paper.

Reviewer comment (Experiments):
The authors do not provide the performance of using the upper-bound prompt. It is insufficient to only prove that the prompt has been close to the upper-bound prompt. Whether is the performance of using the upper-bound prompt higher than the performance of state-of-the-art prompt-based incremental learning methods using task-specific prompts (including CODA-Prompt and HiDe-Prompt [2])?

Author response:
>**Comment4. Detailed explanation about the upper-bound prompt and its performance.**

The reviewer is kindly reminded that we have already provided the performance of the upper-bound prompt in the Table 1 in the main paper. In section 3.1 of main paper, we specified the upper-bound prompt $p^*$ which is optimized by the merged dataset of all tasks (i.e. $\mathcal{D}_{all}=\mathcal{D}_1 \cup \dots \cup \mathcal{D}_T$). Hence, the $p^*$  instructs the model to perform well for all tasks. In Table 1 in the main paper, we present the accuracy by the $p^*$ denoted as `Upper bound'.

Reviewer comment (Experiments):
This method restricts the model ability to learn new tasks, so I suspect that it may not work when there is a large gap between the pre-trained data and new-task data. It is essential to use other pre-trained models or conduct experiments on other datasets, e.g., using the model pretrained on ImageNet-1K with MoCo v3 [2,3]. For more datasets, the authors can refer to [4].

Author response:
>**Comment5. The reviewer commented to present more results using different pretrained models**

We provide the ablation study for different pretrained models in Table 3. The result shows that our method consistently outperforms DualPrompt and L2P for all models. The comparison with CODA-prompt shows dependency for model architecture.


**Table4.** Ablation study for pretrained model with related methods by 10-split-CIFAR100. We set learning rate and the scheduler identical to the main paper, for fair comparison
|  |  | Upper-bound |Ours|CODA-prompt|DualPrompt|L2P|
|--|--|--|--|--|--|--|
|Sup-21k|Acc.|90.85|85.47|82.72|85.87|84.50|
||Forget.|-|4.16|10.26|5.48|6.10|
|iBOT|Acc.|82.21|70.94|72.95|70.86|69.49|
||Forget.|-|9.74|14.84|10.50|10.07|
|iBOT-21k|Acc.|84.05|73.76|74.87|69.67|64.52|
||Forget.|-|12.63|13.20|16.01|18.38|

Reviewer comment (Theory):
The argument in this paper is not convincing. Theoretically, when different sessions use the same prompt, the prompt becomes a set of continuously updating model parameters. In incremental learning, it is challenging for the prompt to both avoid forgetting old tasks and adapt to new tasks. The reason that most existing works use task-specific prompts is to avoid this problem. This work is totally based on the premise that the same prompt is used for each session, but the author does not delve into why task-specific prompts should be abandoned.

Author response:
>**Comment1. Provide more evidence for supporting the advantage of single prompt compared to task-specific prompt.**

We appreciate for the constructive comments. We highlight that the task-specific prompts require the explicit task boundary, which is not provided for the task-agnostic setting. Specifically, G-CIFAR100 dataset introduces 200 different tasks, allowing the overlaps between tasks. According to reviewer's concern, we present the result of DualPrompt with 200 task-specific prompts.

**Table1.** Comparison with task-specific prompts for task-agnostic setting (G-CIFAR100)
|    | Ours | DualPrompt| L2P |
|----|----|----|----|
| Acc. | 86.00 | 79.37 | 84.17 |
|\# Prompt Params|0.11M|18.4M|0.92M|

As shown in the table, the methods with task-specific prompt require more parameters, which becomes redundant by the overlaps between the tasks.

Reviewer comment (Experiments):
Experimentally, the method mentioned in this paper has not been compared to state-of-the-art methods using task-specific prompts, e.g., CODA-Prompt. It is also unknown whether the upper-bound prompt can bring a significant performance increase. Based on these aspects, there is insufficient evidence to believe that using the same prompt for all sessions can outperform using task-specific prompts, and more evidence may be required to justify this claim.

Author response:
>**Comment2. The comparisons with recent related works, including CODA-Prompt are required.**

In response to the reviewer's comment we present the result of the CODA-prompt by our implementation, as follows:

**Table2.** Comparison with related approaches by 10-split-CIFAR100. ViT-B/16 model pre-trained on ImageNet-21k is used. We set learning rate and the scheduler identical to the main paper, for fair comparison.

| Sup-21k   | Upper-bound | Ours |  CODA-Prompt |S-Prompt| HiDe-prompt | 
|----|----|----|---|---|---|
| Acc. | 90.85 | 85.47 | 82.72|80.13|92.61|
| Forget. |-|4.16|10.26|9.19|3.16|

Thanks to the reviewer's fruitful comment, we recognize the HiDe-prompt which is a very recent work(11 October, 2023 Arxiv and accepted for NeurIPS 2023) publicly released after the submission of our paper(28 September, 2023). We clarify that we could not recognize HiDe-prompt during our research. 

We provide the conceptual comparison with HiDe-prompt, which is summarized as follows:
- HiDe-prompt is based on the prompt selection process and prompt pool.
- Contrastive loss with negative pairs is suggested to remove the instance-wise and task-wise overlaps, which is different motivation with our method.
- Additional process and buffer memory are required to obtain old-task statistics.
- Not applicable for smooth transition of data distribution.

First, HiDe-prompt propose to improve the prompt selection process for the prompt pool, by the uninstructed features. Second, the method proposes the contrastive learning to reduce the task-wise overlap, which requires negative pairs. Positive pairs $(h, \mu_c)$ are presented, where $h$ is feature extracted by the prompt and $\mu_c$ is corresponding class prototype of the current task. Also, negative pairs $(h,h')$ and $ \lbrace \(h,\mu_{c,i} \) \rbrace_{i=1}^{t-1}$are defined to minimize the overlaps between the instances and tasks.

Third, the method utilizes the old-task statistics, which should be calculated for each training stage. It requires additional process and the memory. Lastly, the method requires the explicit task boundary which is not provided for the continual learning with smooth transition of the data distribution(i.e. G-CIFAR100 dataset). In contrast, our method is applicable for the task-agnostic dataset, as shown in Table 1 in the main paper.