Paper: Towards Interpretable Continual Learning Through Controlling Concepts (ICLR-2024)

Reviewer comment (Experiments):
Since the paper utilize a pretrained backbone, there is not much difference between the proposed method and the baselines. Moreover, it is unclear whether the gain comes from its continual learning ability or just the concept bottleneck. It would be good to see whether the proposed GPT+Concept Bottleneck procedure works well for a non-incremental learning setting.

Author response:
**#1 Details of CC-CBM**

We describe the CC-CBM in Section 4 (p.5 -p.7). Even though it uses a pretrained backbone, we think it is very different from other baselines [2,3,4]. We combine CC with LF-CBM [1] to build CC-CBM. CC is for its continual learning ability. To know where performance gain comes from, we need to investigate CC-CBM in two steps:
1. *Finetune standard model v.s. Finetune CBM* : To understand the impact of the concept bottleneck.
2. *Finetune CBM v.s. CC-CBM* : To understand the impact of CC.

Both points have been done in our paper, please see the main results (Section 5.1 from p.7- p.9) and Appendix (p.23 - p.26):
* Overall, Finetune CBM outperforms Finetune standard model by up to 2.3% in $\bar{A}_T$ and up to 16.4% in $\bar{F}_T$. 
* CC-CBM makes CBM’s continual learning ability even better, as it outperforms Finetune CBM by up to 1.8% in $\bar{A}_T$ and up to 12.4% in $\bar{F}_T$.  

The non-incremental learning setting is studied in LF-CBM, which shows comparable performance to the standard model. In contrast, in this paper we have studied the GPT + concept bottleneck in the continual learning setting, which shows that this direction is promising and is a novel solution to address the catastrophic forgetting as shown in Section 5.1 of the main paper (p.7- p.9).

Reviewer comment (Reproducibility):
The paper lacks clarity on the GPT concept generation and filtering procedure. It would be helpful to give examples on what the concepts are (move some figures from Appendix to main text). It is also important to share the text prompts used to generate concepts.

Author response:
**#3 GPT-3 usage and example**

CC-CBM is builded upon LF-CBM [1]. It follows the procedure described in LF-CBM to use GPT-3. Example prompts:
```
• List the most important features for recognizing something as a {class}:
• List the things most commonly seen around a {class}
• Give superclasses for the word {class}:

```

Reviewer comment (Experiments):
There is no analysis provided on the order of the number concepts added to Ct for new tasks. This would affect how scalable the method is, especially as it is mentioned that repeated concepts in subsequent tasks are added to the concept set as well.

Author response:
**#1 Scalability**
1. Baseline papers [1, 2, 3] mostly focus on the CIFAR dataset. TinyImagenet is the largest dataset in all baseline papers we compared except SSRE [1] which does experiments in ImageNet-100. Due to the rebuttal constraint, we have tried to add additional experiments on large scale dataset (e.g. ImageNet-10), and the results are in Table R1 and R2. The experiment results are still under the same trend: CC outperforms other exemplar-free methods by up to 0.87% in $\bar{A}_T$ and up to 5.77% in $\bar{F}_T$. Meanwhile, can improve exemplar-based methods by combining them by up to 1.83% in $\bar{A}_T$ and up to 2.87% in $\bar{F}_T$.
2. In Table R3 and Table R4, we provide the number of concepts added in the concept bottleneck layer (CBL) for CC-CBM. 
3. For large-scale datasets in common continual learning benchmarks, our methods perform well, as shown Table 1 and Table 2 (p.8 -p.9) in the draft and the new ImageNet-10 experiments. Due to the expansion ability of CC-CBM, we believe that we can handle other large-scale datasets as well.
 
Table R1: Experiment results on ImageNet-10 in a 5-task scenario.
|                    | $\bar{A}_T$ | $\bar{F}_T$ |
| ------------------ | ------------ | ------------ |
| Finetune           | 23.33        | 81.20        |
| EWC                | 24.14        | 78.46        |
| CC_sol0 (Ours)     | **25.01**        | **77.20**        |
| GEM                | 31.34        | 61.71        |
| CC_sol0_GEM (Ours) | **33.17**        | **59.65**        |

Table R2: Experiment results on ImageNet-10 in a 5-task scenario, the backbones are pretrained on Place365 dataset.
|                   | $\bar{A}_T$ | $\bar{F}_T$ |
| ----------------- | ------------ | ------------ |
| Finetune          | 23.83        | 81.92        |
| EWC               | 25.40        | 78.98        |
| CC_CBM (Ours)     | **26.12**        | **73.21**        |
| GEM               | 32.66        | 60.27        |
| CC_CBM_GEM (Ours) | **34.37**        | **57.40**        |

Table R3: Average concepts added per task in a 5-task scenario.
|                   | CIFAR-10 | CIFAR-100 | CUB200 | ImageNet-10 | TinyImageNet |
| ----------------- | -------- | --------- | ------ | ----------- | ------------ |
| Number of Concept | 37.4     | 133.1     | 276.6  | 35.6        | 301.8        |

Table R4: Average concepts added per task in a 10-task scenario.
|                   | CIFAR-100 | TinyImageNet |
| ----------------- | --------- | ------------ |
| Number of Concept | 154.6     | 284.3        |

Reviewer comment (Novelty):
The method does not seem to necessitate having interpretable concepts to alleviate catastrophic forgetting. The same thing could have been carried out on the classification layer itself using a vision-language aligned model and backbone.

Author response:
**#3 Questions regarded to vision-language aligned (VL-aligned) model**
1. We build CC-CBM based on LF-CBM [4]. Following LF-CBM’s procedure, we use CLIP to calculate the activation matrix $P$ as described in Section 2.2 (p.3). We can replace CLIP with other VL-aligned models as long as they have a text encoder and an image encoder to calculate matrix $P$. We would like to point out that the choice of the VL-aligned model is related to LF-CBM’s design instead of the continual learning problem. We will add this discussion in the revision and leave it as a future work.
2. As we discussed in **#2**, controlling concepts provides interpretability and improves performance. Just using a VL-aligned model is not sufficient to understand the concepts inside itself.

Reviewer comment (Novelty):
The entire dissection and subnetwork search process could have been performed directly on the classes. How are concepts or rather interpretability in general helping here? The paper seems to be attempting to address two different albeit related things, although the motivation for doing so is not very clear.

Author response:
**#2 Motivation for the role of concepts and interpretability**

We agree that the subnetwork search process can be performed on the classification layer directly, just like DEN [2]. However, this gives us no idea about what knowledge and information is learned and preserved in the model, which lacks interpretability. This is the main motivation of our proposed method. In the detailed motivation (please see **#5** in the **Author Response (2/2)** to **Reviewer Hm1u**), we saw the catastrophic forgetting of concepts in other baselines, which motivated us to design the proposed Concept Controller.  As experiment results in the main results (Section 5.1 from p.7- p.9) and Appendix (p.23 - p.26) show, controlling concepts improves the model’s performance in continual learning. Moreover, our methods provide interpretability, which is different from other continual learning algorithms. Finally, We added a comparison between our method and DEN, please see **#2** in the **Author Response** to **Reviewer 6BcB**.

Reviewer comment (Experiments):
By design, it appears that the proposed method can only be used for CNN models and not transformer architectures. It would be nice to see how the proposed work can be more contemporary in its application.

Author response:
**#6 Generalization to transformers**

We would like to point out that whether concepts can be disentangled in transformers is still under research [6, 7, 8]. We believe this is an interesting future work after transformers’ interpretability has made solid progress.

Reviewer comment (Theory):
In the freeze-all variant, it is possible that classes in newer tasks may be based on concepts that were available earlier. How would the model learn these associations if the weights for old tasks are not allowed to change?

Author response:
**#5 Details for freeze concepts and subnetwork**
1. In CC, classes in new tasks may reuse old concepts learned in previous classes. CC freezes the subnetwork from the concept units to the bottom, but does not freeze the associations between the classification layer and concept units. Therefore, the new classes can reuse old concepts. We added new experiments on CUB200 [5], which is a bird species classification dataset and has many concepts overlapped between classes. We measured the forward transfer metric [9] (FWT) to understand whether old concepts help new classes. The experiment results are in Table R6, which shows our method can reuse concepts efficiently and improve a model’s performance.
2. We added the subnetwork size of different datasets in appendix A.14 (p.29). Overall, a more complicated dataset has a bigger subnetwork.

Table R6: FWT on comparing our methods with baselines.
| CUB200, 5 tasks            | FWT     |
| -------------------------- | ------- |
| Without memory buffer:     |
| Finetune                   | 0.76    |
| EWC                        | 3.83    |
| SI                         | 13.50   |
| LwF                        | 10.20   |
| CC-freeze-all (Ours)       | 28.79   |
| CC-freeze-part  (Ours)     | **30.07**   |
| With memory buffer:        |
| GEM                        | 1.35    |
| CC-freeze-all-GEM  (Ours)  | **26.76**   |
| CC-freeze-part-GEM  (Ours) | 26.68   |
| MIR                        | \-12.44 |
| CC-freeze-all-MIR  (Ours)  | 6.89    |
| CC-freeze-part-MIR  (Ours) | 8.22    |

Reviewer comment (Experiments):
The paper shows experiments on relatively small datasets. Related to the point on scalability, results on larger scale datasets would be valuable.

Author response:
**#1 Scalability**
1. Baseline papers [1, 2, 3] mostly focus on the CIFAR dataset. TinyImagenet is the largest dataset in all baseline papers we compared except SSRE [1] which does experiments in ImageNet-100. Due to the rebuttal constraint, we have tried to add additional experiments on large scale dataset (e.g. ImageNet-10), and the results are in Table R1 and R2. The experiment results are still under the same trend: CC outperforms other exemplar-free methods by up to 0.87% in $\bar{A}_T$ and up to 5.77% in $\bar{F}_T$. Meanwhile, can improve exemplar-based methods by combining them by up to 1.83% in $\bar{A}_T$ and up to 2.87% in $\bar{F}_T$.
2. In Table R3 and Table R4, we provide the number of concepts added in the concept bottleneck layer (CBL) for CC-CBM. 
3. For large-scale datasets in common continual learning benchmarks, our methods perform well, as shown Table 1 and Table 2 (p.8 -p.9) in the draft and the new ImageNet-10 experiments. Due to the expansion ability of CC-CBM, we believe that we can handle other large-scale datasets as well.
 
Table R1: Experiment results on ImageNet-10 in a 5-task scenario.
|                    | $\bar{A}_T$ | $\bar{F}_T$ |
| ------------------ | ------------ | ------------ |
| Finetune           | 23.33        | 81.20        |
| EWC                | 24.14        | 78.46        |
| CC_sol0 (Ours)     | **25.01**        | **77.20**        |
| GEM                | 31.34        | 61.71        |
| CC_sol0_GEM (Ours) | **33.17**        | **59.65**        |

Table R2: Experiment results on ImageNet-10 in a 5-task scenario, the backbones are pretrained on Place365 dataset.
|                   | $\bar{A}_T$ | $\bar{F}_T$ |
| ----------------- | ------------ | ------------ |
| Finetune          | 23.83        | 81.92        |
| EWC               | 25.40        | 78.98        |
| CC_CBM (Ours)     | **26.12**        | **73.21**        |
| GEM               | 32.66        | 60.27        |
| CC_CBM_GEM (Ours) | **34.37**        | **57.40**        |

Table R3: Average concepts added per task in a 5-task scenario.
|                   | CIFAR-10 | CIFAR-100 | CUB200 | ImageNet-10 | TinyImageNet |
| ----------------- | -------- | --------- | ------ | ----------- | ------------ |
| Number of Concept | 37.4     | 133.1     | 276.6  | 35.6        | 301.8        |

Table R4: Average concepts added per task in a 10-task scenario.
|                   | CIFAR-100 | TinyImageNet |
| ----------------- | --------- | ------------ |
| Number of Concept | 154.6     | 284.3        |

Reviewer comment (Experiments):
The baselines for continual learning (CL) are not contemporary – there have been several state-of-the-art baselines for CL in the last 2 years, which are not considered. Additionally, no existing continual interpretable baselines have been included like ICICLE (ICCV 2023). While ICCV happened after the ICLR deadline, this work was available on arXiv since March 2023 (https://arxiv.org/abs/2303.07811).

Author response:
**#4 Recent baselines in experiments**
1. We have compared the two baselines within two years: Adam-NSCL[3] and SSRE [1]. As you requested, we compare our methods with ICICLE, and the experiment results are in Table R5. Our method outperforms ICICLE by 2.02% in $\bar{A}_T$ and 31.68% in $\bar{F}_T$. 
2. ICICLE focuses on part-based prototype concepts. Our work focuses on text-based concepts instead, which allows more general interpretability. Meanwhile, it is only suitable for particular model architectures whereas our methods are suitable for any CNN-based models.

Table R5: Experiment results on CUB200 [5] in a 5-task scenario. The backbones are pretrained on Place365 dataset.
|               | $\bar{A}_T$ | $\bar{F}_T$ |
| ------------- | ------------ | ------------ |
| Finetune      | 10.16        | 60.73        |
| EWC           | 11.02        | 62.43        |
| LwF           | 12.50        | 64.50        |
| Adam-NSCL     | 13.27        | 40.73        |
| ICICLE        | 11.40        | 53.66        |
| CC-CBM (Ours) | **13.42**        | **21.98**        |

Reviewer comment (Experiments):
Additional analysis on other vision-language aligned models like FLAVA would be useful.

Author response:
**#3 Questions regarded to vision-language aligned (VL-aligned) model**
1. We build CC-CBM based on LF-CBM [4]. Following LF-CBM’s procedure, we use CLIP to calculate the activation matrix $P$ as described in Section 2.2 (p.3). We can replace CLIP with other VL-aligned models as long as they have a text encoder and an image encoder to calculate matrix $P$. We would like to point out that the choice of the VL-aligned model is related to LF-CBM’s design instead of the continual learning problem. We will add this discussion in the revision and leave it as a future work.
2. As we discussed in **#2**, controlling concepts provides interpretability and improves performance. Just using a VL-aligned model is not sufficient to understand the concepts inside itself.

Reviewer comment (Experiments):
There is no discussion on subnetworks beyond Section 3. It would have been helpful to see details such as how big the learned subnetworks were and how many weights were actually frozen on the different datasets.

Author response:
**#5 Details for freeze concepts and subnetwork**
1. In CC, classes in new tasks may reuse old concepts learned in previous classes. CC freezes the subnetwork from the concept units to the bottom, but does not freeze the associations between the classification layer and concept units. Therefore, the new classes can reuse old concepts. We added new experiments on CUB200 [5], which is a bird species classification dataset and has many concepts overlapped between classes. We measured the forward transfer metric [9] (FWT) to understand whether old concepts help new classes. The experiment results are in Table R6, which shows our method can reuse concepts efficiently and improve a model’s performance.
2. We added the subnetwork size of different datasets in appendix A.14 (p.29). Overall, a more complicated dataset has a bigger subnetwork.

Table R6: FWT on comparing our methods with baselines.
| CUB200, 5 tasks            | FWT     |
| -------------------------- | ------- |
| Without memory buffer:     |
| Finetune                   | 0.76    |
| EWC                        | 3.83    |
| SI                         | 13.50   |
| LwF                        | 10.20   |
| CC-freeze-all (Ours)       | 28.79   |
| CC-freeze-part  (Ours)     | **30.07**   |
| With memory buffer:        |
| GEM                        | 1.35    |
| CC-freeze-all-GEM  (Ours)  | **26.76**   |
| CC-freeze-part-GEM  (Ours) | 26.68   |
| MIR                        | \-12.44 |
| CC-freeze-all-MIR  (Ours)  | 6.89    |
| CC-freeze-part-MIR  (Ours) | 8.22    |

Reviewer comment (Writing):
The writing is unclear in several places. For example, the sentences “it’s not considering classification accuracy of CBM in continual learning setting, which is different than our goal.” (pg 5) and “the Concept Controller strategy follows the similar idea as CC’s in step 4” (pg 6) are difficult to understand.

Author response:
**#7 Presentation**
1. Thank you for pointing out the unclear part of our paper. We have revised them in the revision. 
2. In Section 3, the subnetwork of a concept unit is frozen. In Section 4, we freeze the concept unit itself. Concept unit is a neuron as we stated in Section 1 (p.1).
3. Due to the page limit, we are not able to put Figures 6 or 7 in the main paper. The discussion for CC’s interpretability is in Section 3 (p.5).

Reviewer comment (Writing):
In Section 3, the paper states that the subnetwork is frozen, while in Section 4, it states that the concepts are frozen. It is unclear whether a concept is a neuron or a sub-network.

Author response:
**#7 Presentation**
1. Thank you for pointing out the unclear part of our paper. We have revised them in the revision. 
2. In Section 3, the subnetwork of a concept unit is frozen. In Section 4, we freeze the concept unit itself. Concept unit is a neuron as we stated in Section 1 (p.1).
3. Due to the page limit, we are not able to put Figures 6 or 7 in the main paper. The discussion for CC’s interpretability is in Section 3 (p.5).

Reviewer comment (Evaluation):
Since the main premise of this work is on concepts and their subsequent use of interpretability, it would have been nice to see results such as Figures 6 and 7 in the main paper. The primary results in the main paper are all standard CL metrics. Tables 3 (and 8 in the Appendix) only study the concept consistency – they do not study interpretability.

Author response:
**#7 Presentation**
1. Thank you for pointing out the unclear part of our paper. We have revised them in the revision. 
2. In Section 3, the subnetwork of a concept unit is frozen. In Section 4, we freeze the concept unit itself. Concept unit is a neuron as we stated in Section 1 (p.1).
3. Due to the page limit, we are not able to put Figures 6 or 7 in the main paper. The discussion for CC’s interpretability is in Section 3 (p.5).

Reviewer comment (Theory):
How does the freezing strategy of concept controller account for old concepts occurring in new classes?

Author response:
**#5 Details for freeze concepts and subnetwork**
1. In CC, classes in new tasks may reuse old concepts learned in previous classes. CC freezes the subnetwork from the concept units to the bottom, but does not freeze the associations between the classification layer and concept units. Therefore, the new classes can reuse old concepts. We added new experiments on CUB200 [5], which is a bird species classification dataset and has many concepts overlapped between classes. We measured the forward transfer metric [9] (FWT) to understand whether old concepts help new classes. The experiment results are in Table R6, which shows our method can reuse concepts efficiently and improve a model’s performance.
2. We added the subnetwork size of different datasets in appendix A.14 (p.29). Overall, a more complicated dataset has a bigger subnetwork.

Table R6: FWT on comparing our methods with baselines.
| CUB200, 5 tasks            | FWT     |
| -------------------------- | ------- |
| Without memory buffer:     |
| Finetune                   | 0.76    |
| EWC                        | 3.83    |
| SI                         | 13.50   |
| LwF                        | 10.20   |
| CC-freeze-all (Ours)       | 28.79   |
| CC-freeze-part  (Ours)     | **30.07**   |
| With memory buffer:        |
| GEM                        | 1.35    |
| CC-freeze-all-GEM  (Ours)  | **26.76**   |
| CC-freeze-part-GEM  (Ours) | 26.68   |
| MIR                        | \-12.44 |
| CC-freeze-all-MIR  (Ours)  | 6.89    |
| CC-freeze-part-MIR  (Ours) | 8.22    |

Reviewer comment (Experiments):
How does the framework scale to large datasets?

Author response:
**#1 Scalability**
1. Baseline papers [1, 2, 3] mostly focus on the CIFAR dataset. TinyImagenet is the largest dataset in all baseline papers we compared except SSRE [1] which does experiments in ImageNet-100. Due to the rebuttal constraint, we have tried to add additional experiments on large scale dataset (e.g. ImageNet-10), and the results are in Table R1 and R2. The experiment results are still under the same trend: CC outperforms other exemplar-free methods by up to 0.87% in $\bar{A}_T$ and up to 5.77% in $\bar{F}_T$. Meanwhile, can improve exemplar-based methods by combining them by up to 1.83% in $\bar{A}_T$ and up to 2.87% in $\bar{F}_T$.
2. In Table R3 and Table R4, we provide the number of concepts added in the concept bottleneck layer (CBL) for CC-CBM. 
3. For large-scale datasets in common continual learning benchmarks, our methods perform well, as shown Table 1 and Table 2 (p.8 -p.9) in the draft and the new ImageNet-10 experiments. Due to the expansion ability of CC-CBM, we believe that we can handle other large-scale datasets as well.
 
Table R1: Experiment results on ImageNet-10 in a 5-task scenario.
|                    | $\bar{A}_T$ | $\bar{F}_T$ |
| ------------------ | ------------ | ------------ |
| Finetune           | 23.33        | 81.20        |
| EWC                | 24.14        | 78.46        |
| CC_sol0 (Ours)     | **25.01**        | **77.20**        |
| GEM                | 31.34        | 61.71        |
| CC_sol0_GEM (Ours) | **33.17**        | **59.65**        |

Table R2: Experiment results on ImageNet-10 in a 5-task scenario, the backbones are pretrained on Place365 dataset.
|                   | $\bar{A}_T$ | $\bar{F}_T$ |
| ----------------- | ------------ | ------------ |
| Finetune          | 23.83        | 81.92        |
| EWC               | 25.40        | 78.98        |
| CC_CBM (Ours)     | **26.12**        | **73.21**        |
| GEM               | 32.66        | 60.27        |
| CC_CBM_GEM (Ours) | **34.37**        | **57.40**        |

Table R3: Average concepts added per task in a 5-task scenario.
|                   | CIFAR-10 | CIFAR-100 | CUB200 | ImageNet-10 | TinyImageNet |
| ----------------- | -------- | --------- | ------ | ----------- | ------------ |
| Number of Concept | 37.4     | 133.1     | 276.6  | 35.6        | 301.8        |

Table R4: Average concepts added per task in a 10-task scenario.
|                   | CIFAR-100 | TinyImageNet |
| ----------------- | --------- | ------------ |
| Number of Concept | 154.6     | 284.3        |

Reviewer comment (Novelty):
Other than the fact that a neuron-level interpretable model is being used, is such a model even necessary to the problem the paper attempts to address? The same purpose could be served by using any vision-language aligned model.

Author response:
**#3 Questions regarded to vision-language aligned (VL-aligned) model**
1. We build CC-CBM based on LF-CBM [4]. Following LF-CBM’s procedure, we use CLIP to calculate the activation matrix $P$ as described in Section 2.2 (p.3). We can replace CLIP with other VL-aligned models as long as they have a text encoder and an image encoder to calculate matrix $P$. We would like to point out that the choice of the VL-aligned model is related to LF-CBM’s design instead of the continual learning problem. We will add this discussion in the revision and leave it as a future work.
2. As we discussed in **#2**, controlling concepts provides interpretability and improves performance. Just using a VL-aligned model is not sufficient to understand the concepts inside itself.

Reviewer comment (Experiments):
The introduction of label-free concept bottleneck to the proposed framework makes it even more complex and also difficult to find which part actually contributes to the performance gain. Therefore, an ablation study has to be performed. What if we combine label-free concept bottleneck to DER directly? How does it perform? The paper needs to study each added module carefully and show its advantages.

Author response:
**#2 Ablation Study for CC-CBM**

We have done ablation studies for CC-CBM in the main results and appendix:
1. When introducing label-free CBM (abbreviated as LF-CBM), we compare the Finetune strategy vs CC strategy in Step 2 (p.6), the experiment results are in the main results (Section 5.1 from p.7- p.9) and Appendix (p.23 - p.26).
2. Ablation studies for freezing $W_C$ and regularizing $W_F$ are in Appendix A.5 (p.17).

These ablation studies have shown the contribution of each part in CC-CBM. Meanwhile, we believe that combining LF-CBM with DER (CVPR 2021) may not be a fair ablation study for our CC-CBM for the following reasons:
1. DER requires a new backbone for each task, while CC-CBM only needs one.
2. DER official code is not complete, since the code to learn a new backbone is missing. The issues have been complained about in their official codebase (https://github.com/Rhyssiyan/DER-ClassIL.pytorch/issues/18). 
3. DER is an exemplar-based method, while CC-CBM is an exemplar-free method. We have compared with other exemplar-free methods (EWC, SI, LwF, Adam-NSCL, SSRE) and exemplar-based methods (GEM, MIR, DER-NeurIPS-2020 [11]). Please see Section 5.1 in p.7 for references. Note that the DER-NeurIPS-2020 is a different method published in [11] but happens to have the same name “DER” as the method you suggested (CVPR 2021) and hence we denote [11] as DER-NeurIPS-2020. Overall, our methods outperform other exemplar-free baselines and improve exemplar-based methods when combining them.
4. We have compared with baseline methods that are more recent than DER, e.g. SSRE (CVPR 2022) [10], and show that our results outperforms it in $L_T$ by up to 57.1% in Table 18 (p. 27, appendix A.12). Unfortunately, SSRE did not compare with DER either.  
 
We will add the above discussion into the revision. Thank you for your suggestions!

Reviewer comment (Experiments):
The motivation to design such a complex system is weak. The usage of label-free concept bottleneck will introduce additional information from GPT-3, which is questionable. One can easily achieve good performance if you store the text label and perform zero-shot classification on continual learning dataset (which can easily outperform your results). Even if you use label-free concept bottlenecks, the additional text information is still leaked to your model. I am not sure whether this is still a fair comparison.

Author response:
**#3 Additional Information from GPT-3**

We would like to highlight several points for the potential issue:
1. Using GPT-3 to general concept sets is firstly proposed from an earlier work LF-CBM [7], which has many concurrent and follow-up works that use language models to create concept bottleneck layers, e.g. see [8, 9]. In fact, introducing additional information is widely exploited when designing CBM [1, 2, 3], and is reasonable task-relevant information that we can leverage. 
2. We agree that the potential leaking of CBM is an interesting problem, but we would like to point out that this is not the focus of our work as this is an universal problem in CBM. We will leave it as a future work and add discussion in the revised draft.
3. Using additional information to enhance a model's performance is common in other fields as well [4, 5].
4. Our method is still under the setting of continual learning because during the training phase, the information for each class only appears in one task.

Thus, we believe that our methods are fair in the continual learning setting.

**#4 Further information for additional experiments**

Following your suggestion, we performed the zero-shot classification experiment. We found that zero-shot classification on continual learning will lead to bad performance. In the 5-task scenario, we pretrained the backbone only on the first task, and did zero-shot classification for the following tasks. The experiment results of zero-shot classification are much worse than our proposed CC-CBM as below Table R1 shows.

Table R1: Comparison between zero-shot classification and our methods, in the 5-task scenario of CIFAR-100. The experiment results shows that our methods (CC-CBM) outperforms the zero-shot verison in average accuracy $A_T$ and learning accuracy $L_T$ (ability to learn new tasks). 
|                  | $A_T$         | $L_T$         |
| ---------------- | ------------- | ------------- |
| Zero-shot CC-CBM | 4.34          | 15.36         |
| CC-CBM           | **16.37** | **62.33** |

**#5 Motivation**

We have briefly discussed our motivation in the Introduction section (p.1). Overall, our goal is to design continual learning algorithms which are interpretable. Previous work [6] points out the benefits of human-interpretable concepts for transfer learning.
  
Meanwhile, we measured the evolution of human-understandable concepts in the models when learning sequences of tasks. The experiment results in Table 3 in main draft (p.9) and Table 9 and Table 10 in Appendix p.19) show that existing continual learning algorithms don’t really preserve human-understand concepts. To prevent this behavior, we think it is reasonable to design an algorithm that aims to mitigate catastrophic forgetting of concepts, which is the main motivation for us to design the proposed Concept Controller. We will add this discussion to the introduction in revision to make the motivation more clear.