Paper: Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations (ICLR-2024)

Reviewer comment (Theory):
In page 1, under limitations of CBMs the paper mentioned interpretability and said "They cannot effectively quantify the intricate relationships between various concepts and class labels". I disagree, in the original CBM on top of the CBM layer there is a linear which quantifies how each concept affects the final output so if a model classifies an image with label "A" we can look at the weight matrix and see which concepts where used in this classification. Yes, at inference time (assuming we don't have the ground truth concepts) we don't know if the model correctly classified the concepts or not or the likelihood of a concept being correctly classified but we do know the relationship between the concepts and the final output.

Author response:
**Clarity: Final Linear Layer of CBM as Conditional Interpretation.** We apologize if our presentation has led to confusion. By 'intricate', we were referring to complex conditional dependencies such as $p(c|y)$, $p(c_{k}|y,c_{k^{\prime}})$, and $p(c_{-k}, y|x,c_{k})$. We agree that Post-hoc CBMs and vanilla CBMs, with their interpretative linear layers, provide a way to understand concept-label relationships. In fact, in PCBMs, the weights can indicate each concept's importance for a given label ($p(c_k|y)$).
However, these existing CBM variants cannot provide more complex conditional dependencies such as $p(c_{k}|y,c_{k^{\prime}})$ and $p(c_{-k}, y|x,c_{k})$. In contrast, our model, which relies on the Energy-Based Model (EBM) structure, can naturally provide such comprehensive conditional interpretations.

Reviewer comment (Writing):
The paper is not very clear, please see the questions section.

Author response:
**Clarity: Final Linear Layer of CBM as Conditional Interpretation.** We apologize if our presentation has led to confusion. By 'intricate', we were referring to complex conditional dependencies such as $p(c|y)$, $p(c_{k}|y,c_{k^{\prime}})$, and $p(c_{-k}, y|x,c_{k})$. We agree that Post-hoc CBMs and vanilla CBMs, with their interpretative linear layers, provide a way to understand concept-label relationships. In fact, in PCBMs, the weights can indicate each concept's importance for a given label ($p(c_k|y)$).
However, these existing CBM variants cannot provide more complex conditional dependencies such as $p(c_{k}|y,c_{k^{\prime}})$ and $p(c_{-k}, y|x,c_{k})$. In contrast, our model, which relies on the Energy-Based Model (EBM) structure, can naturally provide such comprehensive conditional interpretations.

Reviewer comment (Reproducibility):
ECBMs Training: Training is much more complicated than regular CBMs and probably much more expensive, particularly optimizing equation 10.

Author response:
**Significance: Training Cost.** Training the ECBMs is indeed more complex than training regular CBMs. To manage the training cost, we employ a negative sampling strategy, similar to traditional CBMs' cost.

Reviewer comment (Reproducibility):
ECBMs Inference requires optimization not just a forward pass through the network.

Author response:
**Significance: Inference Cost.** Indeed, due to the energy-based nature of our model, our ECBM's inference requires multiple optimization iterations rather than a single feedforward pass. There are strategies to expedite this inference process. One such approach is to use the predictions from a vanilla CBM as an initialization, thereby 'jump-starting' the optimization and substantially accelerating inference.

Reviewer comment (Reproducibility):
Usage and reproducibility: There was no code to reproduce the experiment. It is very difficult to imagine how all of this is trained together without the code.

Author response:
**Usage and Reproducibility: Code Release.** Thank you for your encouraging comments and interest in our work. We understand the importance of reproducibility in scientific research. So far, we have finished cleaning up the source code and will release it if the paper is accepted.

Reviewer comment (Experiments):
Results in figure 2 are from which dataset?

Author response:
**Q2. Results in Figure 2 are from which dataset?**

Our apologies for the oversight. The results displayed in Figure 2 are from the CUB dataset. We have updated the figure caption in the revised manuscript to include this information. Thank you for pointing this out.

Reviewer comment (Presentation):
Aren't $E_\theta^{class}$, $E_\theta^{concept}$, $E_\theta^{global}$ neural networks? If so, why don't they have trainable or frozen signs in figure 1?

Author response:
**Q3. Aren't $E_{\theta}^{class}$, $E_{\theta}^{concept}$, $E_{\theta}^{global}$ neural networks? If so why don't they have trainable or frozen signs in figure 1?**

Thanks for mentioning this. Indeed, $E_{\theta}^{class}$, $E_{\theta}^{concept}$, and $E_{\theta}^{global}$ are neural networks. The absence of trainable or frozen signs in Figure 1 was purely for aesthetic and clarity purposes. We apologize for any confusion this may have caused. According to your suggestion, we have updated Figure 1 to include these signs.

Reviewer comment (Writing):
Equation 9 is $[v_k]_{k=1}^{K}$ — is this the concatenation of all concepts embeddings?

Author response:
**Q4. Equation 9 is $[v_k]_{k=1}^{k}$ this the concatenation of all concepts embeddings?**

Yes. In Equation 9, $[v_k]_{k=1}^{k}$ denotes the concatenation of all concept embeddings. Thank you for pointing this out, and we have clarified this in our revision accordingly.

Reviewer comment (Reproducibility):
Please clearly state the output dimensions of each energy model.

Author response:
**Q5. Please clearly state the output dimensions of each energy model.**

We are sorry for the confusion. Each energy model (e.g., $E_{\theta}^{concept}$) outputs *a scalar value*, which measures the compatibility between the variables. We hope this addresses your question. If you require further clarification or have additional questions, we will be more than happy to follow up with any details needed.

Reviewer comment (Evaluation):
[Critical] In my opinion, the biggest weakness of this paper is its evaluation and the fairness of the evaluation itself. As discussed in the questions below, this paper seems to compare baselines with significantly varying capacities and fine-tuning processes ( e.g., ProbCBM is trained with a ResNet18 backbone in the source of the results reported in this paper vs ECBM trained with a ResNet101 backbone) as well as failing to evaluate key contributions across several tasks (e.g., interventions are evaluated on a single task). This yields a lot of the evaluation inconclusive as it makes the source of the improvements shown in the proposed model hard to identify.

Author response:
Thank you for mentioning this. 
Our preliminary results show that ProbCBM with ResNet101 underperforms ProbCBM with ResNet18. We suspect that probCBM may exhibit instability as the dimensionality of the feature embedding increases. Hence we have chosen to report the highest scores in the paper. The table below shows the results. 

|       Dataset       |         |       CUB       |       |         |     CelebA      |       |         |      AWA2       |       |
| :-----------------: | :-----: | :-------------: | :---: | :-----: | :-------------: | :---: | :-----: | :-------------: | :---: |
|       Metric        | Concept | Overall Concept | Class | Concept | Overall Concept | Class | Concept | Overall Concept | Class |
| ProbCBM (ResNet18)  |  0.946  |      0.360      | 0.718 |    -    |        -        |   -   |  0.959  |      0.719      | 0.880 |
| ProbCBM (ResNet101) |  0.932  |      0.236      | 0.650 |  0.888  |      0.511      | 0.330 |  0.930  |      0.553      | 0.739 |
|      **ECBM**       |  0.973  |      0.713      | 0.812 |  0.876  |      0.478      | 0.343 |  0.979  |      0.854      | 0.912 |


Furthermore, we expanded our evaluation to include more intervention experiments and adopted interventions across all datasets. This approach allows us to demonstrate the intervention performance under varying ratios (more details in the response to **Q1-Q3** below). We believe your suggestions have significantly enhanced the comprehensiveness of our experiment. Thank you for your valuable input.

Reviewer comment (Evaluation):
[Critical] Against common practices in the field, none of the results in the evaluation section include error bars. Without such error bars, it is very hard to judge for statistical significance. This is particularly important for the paper’s main evaluation in Table 1 where differences between ECBMs and other baselines are small for task (e.g., ”class”) accuracies on some tasks.

Author response:
This is a good suggestion. Accordingly, we have revised Table 1 to include standard deviations over five runs with different random seeds. These will help present a clearer perspective on the statistical significance of our results. Our standard deviations range from 0.000 to 0.006, except for the overall concept accuracy for CUB, which is 0.009. All values are represented up to three decimal digits. These results confirm the significance of the improvement brought by our ECBMs.

Reviewer comment (Experiments):
[Critical] Some of the key baseline results in this paper seem to contradict the results presented in previous literature. For example, CEM’s intervention performance in Figure 2 is in direct contradiction to those reported in the original paper and in subsequent works ([1, 2, 3]). Furthermore, the authors claim that ProbCBMs cannot be trained on CelebA due to the binary nature of the concept annotations. This however, to the best of my understanding, is not a requirement for ProbCBM (an example is CUB itself which is used as a baseline for this method even though it has not uncertainty labels as claimed by the manuscript). Similarly, results for Post-hoc CBMs are missing for CelebA and AWA2 as the authors claim that Post-hoc CBMs require "predefined concept activation vectors." This is not true as the CAVs needed for PCBMs can be easily learnt from a dataset with binary concept annotations as those in CelebA and AWA2. In fact, the authors of that paper do this exact process for CUB.

Author response:
We are sorry for the confusion. It seems the inconsistency with prior intervention performance might be due to our use of individual-level concept intervention. In light of your feedback, we re-conducted the intervention experiment using group-level intervention (more details in the response to **Q1-Q3** below). 

For the results of ProbCBM and Post-hoc CBM (PCBM), we acknowledge the misunderstandings regarding their requirements and capabilities. We revisited these models and retrained them accordingly. These revised results are now included in Table 1 of our revised paper. Your suggestions and guidance have been invaluable in addressing these issues. Thank you for drawing our attention to these points. The new results confirm that our ECBM outperforms these baselines (ProbCBM and PCBM) in both CeleA and AWA2 too.

Reviewer comment (Reproducibility):
[Critical] A lot of details required for reproducibility are missing. There is no code attached to this submission as part of the supplement. There are no details on the details and training procedures of any of the baselines. The hyperparameter selection is not discussed anywhere, which brings doubt to the very specific precision indicated for some of ECBM’s selected hyperparameters (e.g., $\lambda_c = 0.49$).

Author response:
Thank you for mentioning this. We understand the importance of transparency in research and are committed to making our work as accessible and replicable as possible. 

For the code release, so far, we have finished cleaning up the source code and will release it if the paper is accepted.

For the baselines, we assure you that we have strictly adhered to the original papers and their provided codebases for reproduction. 

Regarding hyperparameter $\lambda_{c}=0.49$, this is because in our implementation, we introduce $\lambda_l$ to Eqn. (12) in the paper, resulting in:
$\lambda_l E_{\theta}^{class}(x,y) + \lambda_c E_{\theta}^{concept}(x,c) + \lambda_g E_{\theta}^{global}(c,y)$

We then set $\lambda_l=1$, $\lambda_c=1$ and $\lambda_g=0.01$.

This is therefore equivalent to setting $\lambda_c=1/(1+1+0.01)=0.49$ for the original Eqn. (12) without $\lambda_l$:
$E_{\theta}^{class}(x,y) + \lambda_c E_{\theta}^{concept}(x,c) + \lambda_g E_{\theta}^{global}(c,y)$. 

We are grateful for your attention to these details and have revised the paper accordingly.

Reviewer comment (Experiments):
[Major] Related to the point above, there are no ablation studies showing the sensitivity of the proposed method to its hyperparameters ($\lambda_{\{c, g\}}$).

Author response:
In response to your feedback, we have conducted ablation studies to demonstrate the sensitivity of our proposed method to its hyperparameters. The results on the CUB dataset are available in the table below. These studies provide insights into the hyperparameter search process and show that our ECBM is stable over a wide range of hyperparameters. We appreciate your valuable suggestion and have included all additional results on CUB, AWA2, and CelebA datasets in our revised paper (Table 5 of Appendix C.2). 

|   Hyperparameter   | Concept | Overall Concept | Class |
| :----------------: | :-----: | :-------------: | :---: |
|  $\lambda_l=0.01$  |  0.971  |      0.679      | 0.756 |
|  $\lambda_l=0.1$   |  0.971  |      0.680      | 0.795 |
|   $\lambda_l=1$    |  0.973  |      0.713      | 0.812 |
|   $\lambda_l=2$    |  0.971  |      0.679      | 0.808 |
|   $\lambda_l=3$    |  0.971  |      0.679      | 0.808 |
|   $\lambda_l=4$    |  0.971  |      0.679      | 0.806 |
|  $\lambda_c=0.01$  |  0.971  |      0.679      | 0.799 |
|  $\lambda_c=0.1$   |  0.971  |      0.679      | 0.799 |
|   $\lambda_c=1$    |  0.973  |      0.713      | 0.812 |
|   $\lambda_c=2$    |  0.971  |      0.679      | 0.798 |
|   $\lambda_c=3$    |  0.971  |      0.679      | 0.798 |
|   $\lambda_c=4$    |  0.971  |      0.679      | 0.798 |
| $\lambda_g=0.0001$ |  0.971  |      0.679      | 0.805 |
| $\lambda_g=0.001$  |  0.971  |      0.679      | 0.805 |
|  $\lambda_g=0.01$  |  0.973  |      0.713      | 0.812 |
|  $\lambda_g=0.1$   |  0.971  |      0.680      | 0.795 |
|   $\lambda_g=1$    |  0.971  |      0.680      | 0.756 |

Reviewer comment (Novelty):
[Major] The claim that previous concept-based methods do not allow one to understand concept to label relationships is not entirely true. Post-hoc CBMs enable this via their sparse interpretable predictor and even vanilla CBMs tend to be trained with a linear layer on top of their concept bottleneck, rendering an easy mechanism to obtain concept importance for a given label via the linear layer's weights.

Author response:
We apologize if our presentation has led to confusion. We meant to say that previous concept-based methods do not allow one to understand ``"complex"`` conditional dependencies (as mentioned in the abstract) such as $p(c|y)$, $p(c_{k}|y,c_{k^{\prime}})$, and $p(c_{-k}, y|x,c_{k})$. We agree that Post-hoc CBMs and vanilla CBMs, with their interpretative linear layers, provide a way to understand concept-label relationships. In fact, in PCBMs, the weights can indicate each concept's importance for a given label ($p(c_k|y)$). However, these existing CBM variants cannot provide more complex conditional dependencies such as $p(c_{k}|y,c_{k^{\prime}})$ and $p(c_{-k}, y|x,c_{k})$. In contrast, our model, which relies on the Energy-Based Model (EBM) structure, can naturally provide such comprehensive conditional interpretations. Thank you for pointing this out, and we have clarified this in the revision as suggested.

Reviewer comment (Experiments):
[Major] The proposed evaluation is limited in the number and quality of the competing baselines. For example, several of the baselines are missing values in Table 1.

Author response:
Thank you for mentioning this. According to your feedback, we have updated Table 1 to provide a more comprehensive and fair comparison (see more discussion in the response to **W1-W3** above). We appreciate your suggestion, which has helped improve the quality of our evaluation.

Reviewer comment (Evaluation):
[Minor] Inference on ECBMs requires one to solve an optimization problem via a relaxation that is solved with gradient descent + back-propagation. This means that this proposed approach may not scale well to a setup where one is expected to run inference quickly or as part of a separate pipeline, which needs to be end-to-end differentiable.

Author response:
Indeed, due to the energy-based nature of our model, our ECBM's inference requires multiple optimization iterations rather than a single feedforward pass. There are strategies to expedite this inference process. One such approach is to use the predictions from a vanilla CBM as an initialization, thereby 'jump-starting' the optimization and substantially accelerating inference. We believe further research into accelerating energy-based CBMs would be interesting future work. We have included the discussion above in the revision.

Reviewer comment (Presentation):
[Minor] The paper lacks any discussion of the limitations of its method. The last section, which has “limitations” on its title, fails to discuss any of the limitations.

Author response:
We appreciate your feedback regarding the omission of a discussion on the limitations of our method in the last section of the paper. We are sorry for the oversight and agree that such a discussion is crucial for a balanced presentation of our work. In response to your observation, we will add a comprehensive discussion of the limitations of ECBMs to the final section. 

Your feedback has helped us identify several potential limitations, including the need for better solutions in the optimization process of our ECBM and the design of efficient initialization procedures. Additionally, our current model requires concept labeling, a process that could be improved with label-free approaches. As a potential future improvement, our model could be adapted to work with label-free CBMs. Thank you for your constructive feedback, it has been instrumental in helping us improve our work.

Reviewer comment (Experiments):
Does the architecture of the network affect the performance? Does it need to be shallow or deeper? What are the design considerations for these networks?

Author response:
Indeed, the architecture of the network plays a role in the model's performance. In our case, we've followed the approach implemented by all the baselines, utilizing the ResNet architecture. It is preferable for the network to be deeper. Our experiments show that the fusion of Energy-Based Models (EBM) and Concept Bottleneck Models (CBM) necessitates a deeper network structure; this allows for the extraction of higher-quality feature embeddings, which significantly enhances the model's performance. We also include more details on network architecture in Table 2 of the Appendix.

Reviewer comment (Experiments):
Is the model robust towards background shifts (ref. experiments in [1])?

Author response:
This is a good question. In response to your comment, we conducted an additional experiment, and the results are included in the table below. These additional results show that on background shift datasets derived from [1], our ECBM achieved a Class accuracy of 0.584 and a Concept accuracy of 0.945. These results are higher than the best Concept Bottleneck Model (CBM) results presented in Table 3 of [1] (i.e., Class accuracy of 0.518 and Concept accuracy of 0.931). We have included the discussion above in our revised paper (e.g., Table 6 of Appendix C) as suggested. 

| Methods | Concept | Overall Concept | Class |
| :--: | :--: | :--: | :--: |
|Standard | -       | - |   0.373  |
|Joint (CBM)  | 0.931       | - |   0.518  |
|Sequential (CBM)   | 0.928       | - |   0.504  |
|Independent (CBM)   | 0.928      | - |   0.518  |
|ECBM  | 0.945| 0.416 | 0.584 |

This suggests that our model demonstrates considerable robustness towards background shifts.

[1] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy Liang: Concept Bottleneck Models. ICML 2020

Reviewer comment (Evaluation):
What is the compute overhead in comparison to CBMs?

Author response:
During the training phase, the computational complexity of our energy-based model (ECBM) is similar to traditional CBMs. During inference, due to the energy-based nature of our model, it requires several iterations, typically between 20 and 100. There are strategies to expedite this inference process. One such approach is to use the predictions from a vanilla CBM as an initialization, thereby 'jump-starting' the optimization and substantially accelerating inference. We believe further research into accelerating energy-based CBMs would be interesting future work. We have included the discussion above in the revision.

Reviewer comment (Experiments):
One of the main shortcomings of CBMs is the need for concept annotations. There has been a lot of work on addressing it. Why do the authors build on top of vanilla CBMs, not the improved frameworks [3]?

Author response:
This is a good point. Indeed, the requirement for concept annotations is a known limitation of Concept Bottleneck Models (CBMs). 

+ We chose to build upon the vanilla CBM as our goal was to delve into the relationships between concepts and between concepts and labels.
+ Moreover, the baseline models we used for comparisons, including PCBM [1], CEM [2], and ProbCBM [3], are also based on vanilla CBM. To ensure a fair comparison, we chose this approach.
+ Our model is compatible with label-free CBMs. Therefore one could potentially use label-free CBMs to generate concept annotations automatically, and train our ECBM using these concepts, thereby addressing this limitation of concept annotation. This way, our ECBM can simultaneously enhance the interpretability, intervention capabilities, and performance of CBM, without manual concept annotations. 

Self-Explaining Neural Networks (SENNs) and their variants efficiently encode relationships between concepts (as mentioned in [4]) and between classes and concepts. However, they are not supervised concept bottleneck models and do not allow for direct concept intervention. Therefore they are not applicable to our setting. This is also why previous CBM papers (e.g., our baselines CEM and PCBM) do not include SENNs as baselines. Nonetheless, we acknowledge the importance of these models and have cited and discussed them accordingly in the revision. 


[1] Post-hoc Concept Bottleneck Models. ICLR, 2023.

[2] Concept Embedding Models. NIPS, 2022.

[3] Probabilistic Concept Bottleneck Models. ICML 2023.

[4] A Framework for Learning Ante-hoc Explainable Models via Concepts. CVPR 2022.

Reviewer comment (Experiments):
In addition to CBMs, SENNs also offer a way to develop models that can implicitly explain their predictions, and many of their variants can efficiently encode the relationships between concepts ([2]), as well as between class and concepts. How does the proposed framework compare against those?

Author response:
Self-Explaining Neural Networks (SENNs) and their variants efficiently encode relationships between concepts (as mentioned in [4]) and between classes and concepts. However, they are not supervised concept bottleneck models and do not allow for direct concept intervention. Therefore they are not applicable to our setting. This is also why previous CBM papers (e.g., our baselines CEM and PCBM) do not include SENNs as baselines. Nonetheless, we acknowledge the importance of these models and have cited and discussed them accordingly in the revision. 

[4] A Framework for Learning Ante-hoc Explainable Models via Concepts. CVPR 2022.

Reviewer comment (Experiments):
Many of the existing explanation methods aren't robust in practice. Are ECBMs robust to adversarial attacks?

Author response:
This is a good point. We believe our ECBMs do potentially enjoy stronger robustness to adversarial attacks compared to existing CBM variants. Specifically, our ECBMs are designed to understand the relationships between different concepts, as well as the relationships between concepts and labels. As a result, during inference, ECBMs can leverage these relationships to automatically correct concepts that may be influenced by adversarial attacks. Our preliminary results (e.g., the new experiments on background shifts for **Q2**) suggest that our ECBM can potentially improve the robustness against adversarial attacks compared to existing CBM variants. We believe this is interesting future work; we appreciate your comments and have included the discussion above in the revision.

Reviewer comment (Writing):
The second gap: Given the class label and one concept, the probability of predicting another concept? is not clear.

Author response:
**Q1.  The second gap: Given the class label and one concept, the probability of predicting another concept? is not clear**

We appreciate your question about the interpretation of the second gap. This scenario is an example of conditional interpretation in our model. In this context, given a class label $y$ and one concept $c$, the model predicts the probability of another concept $c'$. This process provides insights such as: "from the model's perspective, how relevant concepts $c$ and $c'$ are for images in class $y$". 

We understand that this explanation might not fully address your comment. If you could provide further details or specify the aspects that are unclear, we would be very glad to offer a more targeted explanation.

Reviewer comment (Novelty):
Incomplete literature review. The authors should add the following papers as each of them has a unique perspective to solve the problem.  
[1] Interpretable Neural-Symbolic Concept Reasoning. Barberio et al. ICML 2023  
[2] Entropy-based Logic Explanations of Neural Networks. Barberio et al. AAAI 2021  
[3] Addressing Leakage in Concept Bottleneck Models. Havasi et. al., Neurips 2022  
[4] Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat. Ghosh et al. ICML 2023  
[5] Distilling BlackBox to Interpretable models for Efficient Transfer Learning. Ghosh et al. MICCAI 2023  
[6] Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification. Yang et al. CVPR 2023  
[8] A Framework for Learning Ante-hoc Explainable Models via Concepts. Sarkar et al. CVPR 2022  
[9] Label-Free Concept Bottleneck Models. Oikarinen et al. ICLR 2023

Author response:
**Q2. Incomplete literature review.**

We appreciate your feedback about the comprehensiveness of our literature review. Your suggestions have indeed enhanced the breadth of our research and have been incorporated into the 'Related Work' section of our revised paper.

Reviewer comment (Theory):
How does each of the three conditional probabilities — p(y|x), p(c-k|x, ck), and p(ck|y, ck′) — mitigate the three limitations: Interpretability, Intervention, and Performance?

Author response:
**Q3. How does each of three conditional probabilities - p(y|x), p(c-k|x, ck), and p(ck|y, ck′)—mitigate the 3 limitations - Interpretability, Intervention, Performance?**

Each of the three conditional probabilities addresses different limitations related to Interpretability, Intervention, and Performance:

- $p(c, y|x)$ contributes to **performance** improvment. Unlike original CBMs, which predict concepts first and then predict labels using these predicted concepts, our model predicts concepts $c$ and class labels $y$ *simultaneously* while considering their *compatibility*. Our global energy network measures the dependency between $c$ and $y$, thereby improving the prediction accuracy. 
- $p(c_{-k}|x,c_k)$ is designed for **intervention**. For instance, given the input $x$, if an expert knows the correct concept $c_k$ and performs concept intervention on concept $k$, this can help correct other concepts $c_{-k}$. This is equivalent to computing the conditional probability $p(c_{-k}, y|x,c_k)$. 
- $p(c|y)$ and $p(c_{k}|y,c_{k^{\prime}})$ are used for **interpretability**. $p(c_{k}|y,c_{k^{\prime}})$ indicates the strength of the connection between $c_k$ and $c_{k^{\prime}}$ for a certain class $y$ of images, providing conditional interpretation. $p(c|y)$ highlights which concepts are important to the given class.

We hope that these explanations provide clarity on your question, and we are open to further discussions on this.

Reviewer comment (Experiments):
How does this method impact label-free CBM or Language in a Bottle kind of framework? These concepts are retrieved from the language model, and in more realistic settings like medical images, having ground truth concepts is expensive.

Author response:
**Q4. How does this method impact label free CBM or Language in a bottle kind of framework?**

This is a good question. Indeed, our ECBM is compatible with the label-free CBM method. One possible approach is to initially extract concepts using the label-free method and subsequently train our ECBM. This would be interesting future work. We appreciate your valuable insights and have incorporated this comment into the revision.

Reviewer comment (Evaluation):
I believe this is an inherently interpretable method. How to integrate it in a posthoc setting like PCBM? Also, if it can be integrated in a posthoc setting, I would like to see the comparison of completeness scores.

Author response:
**Q5.  I believe this is an inherently interpretable method. How to integrate it in a posthoc setting like PCBM?**

Yes, ECBM is an inherently interpretable method, not a post-hoc one. In this paper, we concentrate on inherent interpretability. Although integrating our method into a post hoc setting like PCBM is outside the scope of this study, we agree that it would be interesting and nontrivial future work to explore. We are grateful that you have accurately grasped the nature of our method, and will make this point clearer in the revision as suggested.

Reviewer comment (Experiments):
One limitation of concept bottleneck models is the limited applicability in real-life scenarios. I would like to see one experiment on an application — for example, shortcut identification as in [4] and “Discover and Cure: Concept-aware Mitigation of Spurious Correlation.” Wu et al. ICML 2023.

Author response:
**Q7. One limitation of concept bottleneck models is the limited applicability in a real-life scenario...shortcut identification...**

Thank you for pointing us to these interesting papers. We have carefully gone through them and find that our model is actually equipped to carry out such tasks. 


Specifically, $p(c|y)$ generated by our ECBM shows the importance of each concept for each class. By comparing our ECBM's estimated $p(c|y)$ with the ground-truth $p(c|y)$, we can identify any significant differences in concept importance to shed light on any spurious feature learned in the model. For example, if our ECBM estimates that $p(c_1=1|y=1) = 0.8$ while the groud-truth $p(c_1=1|y=1) = 0$, this indicates that the model learns concept $c_1$ as a spurious feature. 

Our experiments show that our ECBM is robust against spurious correlations. For example, Figure 3 in the paper shows that our ECBM's estimated $p(c|y)$ is very close to the ground truth. This is possibly due to our ECBM's joint inference of class labels and concepts, which helps mitigate spurious correlation. Furthermore, we conducted additional experiments on the TravelingBirds dataset following the robustness experiments of CBM concerning background shifts. The results (Appendix C's Table 6) reveal that our ECBM outperforms CBMs in this regard. These findings underscore our model’s superior robustness to spurious correlations.

We have included the discussion and the references in our revision as suggested.

Reviewer comment (Experiments):
There is a problem of leakage in CBM as in [3]. Will the ECBM be helpful in that regard? If the authors can show ECBM is better for the experiments depicted in Fig. 2 and 3 in [3], this paper will be very strong.

Author response:
**Q8. Also, there is a problem of leakage in CBM as in [3]. Will the ECBM be helpful in that regard?**

This is an insightful question, and thanks for pointing us to [3]. We believe that our ECBM model could indeed offer a solution to this problem. In response to your suggestion, we conducted an experiment similar to the ones depicted in Figures 2 and 3 in [3], but using our ECBM model. We found that when only a few concepts were present, the accuracy of our method did not significantly increase. However, there was a noticeable increase in the performance of the baseline model(like CEM) at this stage. Our model's accuracy gradually increased when more concepts were given during training. This suggests that *our ECBM model can help mitigate the leakage problem*; in contrast, baselines such as CBM and CEM still suffer from some degree of leakage due to their soft concepts. We have included these additional empirical results in Figure 11 of Appendix C.5 of our revised manuscript. Thank you for your valuable suggestion, which has helped us strengthen our evaluation of ECBM.

[3] Addressing Leakage in Concept Bottleneck Models. Havasi et. al., Neurips 2022.

Reviewer comment (Theory):
I find it difficult to fully embrace the idea of Concept Bottleneck Modeling (CBM) presented in this paper, as well as in prior research, due to the absence of a clear definition for the term "concepts." From my perspective, what is referred to as "concepts" in this context appears to be nothing more than additional labels or explicit features that can be readily discerned by humans. However, one of the notable advantages of deep learning networks lies in their ability to automatically extract features, even those that may not be easily identifiable by humans. Consequently, I believe that the primary reason for the observed decline in performance of CBMs, when compared to recent work such as [1] on CUB, is that the model is exclusively directed towards learning these so-called "concepts" and neglects the acquisition of features that can be explicitly identified.

Author response:
**W1.  "... due to the absence of a clear definition of the term "concepts" ... neglects the acquisition of features that can be explicitly identified."**

This is a good question.

**The Advantages of Concept Bottleneck Models (CBMs):** We agree that "concepts" can be seen as "explicit features that can be readily discerned by humans". However, we would like to emphasize that by using these concepts as the bottleneck, Concept Bottleneck Models (CBMs) and their variants have additional advantages: 

+ Humans can intervene in these predicted concepts, thereby influencing final predictions. This is particularly beneficial in real-world settings, where it is crucial not only to provide interpretability but also to incorporate human participation into the model's decision-making process, known as the human-in-the-loop process.

+ They predict interpretations that can be presented to humans for verification, creating a feedback loop for accuracy.


**Addressing Performance Decline:** Thanks for pointing us to the interesting paper [1], which we have cited and discussed in the revision. We agree that maintaining concept-based interpretability while ensuring model performance is an open question in the field. The baselines we compare in our work all strive to address this issue. 

Our proposed ECBM further pushes the limit by utilizing an energy network. Specifically, by enabling the learning of compatibility between input $x$, concept $c$, and class label $y$, we aim to create a network that can capture features that are not readily identifiable. Looking at the empirical results (refer to Appendix C, Table 4), our ECBM achieves performance very close to the black-box model on the CUB and AWA2 datasets; for example, on CUB, the accuracy of the black-box model and our ECBM is $0.826$ and $0.812$, respectively. In some cases, such as with the CelebA dataset, it even improves upon the black-box performance ($0.343$ versus $0.291$). This can be attributed to our joint inference process of concepts and class labels. 

[1] Progressive co-attention network for fine-grained visual classification. 2021 International Conference on Visual Communications and Image Processing (VCIP). IEEE, 2021.

Reviewer comment (Experiments):
There missing baseline models that are not based on CBM in the experiments.

Author response:
**W2. "...missing baseline models that are not based on CBM in the experiments."**

Thank you for mentioning this. Our current focus is primarily on concept-based models and, as such, our comparisons are chiefly made with standard CBMs, with an emphasis on both interpretability and model performance. 
In line with similar studies, such as those on PCBM [1] and CEM [2], we've chosen to include only CBM variants as baselines. This decision is based on the fact that non-CBM models do not possess both concept interpretability and direct concept intervention capabilities, which are crucial for our line of investigation. 

Nevertheless, we understand the potential value of incorporating traditional black-box models as baselines to provide a more comprehensive perspective. In addition to our primary findings, we have conducted tests on a black-box model with the same network architecture. Please kindly find these results below and in Table 4 of Appendix C in our revised paper. These results show that 

+ our ECBM achieves performance very close to the black-box model on the CUB and AWA2 datasets; for example, on CUB, the accuracy of the black-box model and our ECBM is $0.826$ and $0.812$, respectively.
+ In some cases, such as with the CelebA dataset, it even improves upon the black-box performance ($0.343$ versus $0.291$). This can be attributed to our joint inference process of concepts and class labels.
+ our ECBM outperforms all baseline models, including CBM and CEM. 



| Model/Dataset |         |       CUB       |       |         |     CelebA      |       |         |      AWA2       |       |
| :------------: | :-----: | :-------------: | :---: | :-----: | :-------------: | :---: | :-----: | :-------------: | :---: |
|     Metric     | Concept | Overall Concept | Class | Concept | Overall Concept | Class | Concept | Overall Concept | Class |
|    **ECBM**    |  0.973  |      0.713      | 0.812 |  0.876  |      0.478      | 0.343 |  0.979  |      0.854      | 0.912 |
|      CBM       |  0.964  |      0.364      | 0.759 |  0.837  |      0.381      | 0.246 |  0.979  |      0.803      | 0.907 |
|      CEM       |  0.965  |      0.396      | 0.796 |  0.876  |      0.457      | 0.330 |  0.978  |      0.796      | 0.908 |
|    Blackbox    |    -    |        -        | 0.826 |    -    |        -        | 0.291 |    -    |        -        | 0.929 |

If you have any specific suggestions for such baselines, please do not hesitate to share them. We would be more than willing to conduct additional comparative experiments accordingly.

[1] Post-hoc Concept Bottleneck Models. ICLR, 2023.

[2] Concept Embedding Models. NIPS, 2022.

Reviewer comment (Experiments):
There missing ablation study on the proposed method with only a single branch, i.e, the 'concepts' branch and the 'class' branch as shown in Figure 1.

Author response:
**W3. "There missing ablation study on the proposed method with only a single branch, i.e, the 'concepts' branch and the 'class' branch as shown in Figure 1."**

Thank you for your astute observation regarding the absence of an ablation study focused on a single branch method. In response to your comment, we have conducted an additional ablation study and incorporated the results into our work. 

Specifically, we include two baselines for this ablation study:

+ x-y: This baseline involves a single branch that utilizes the class energy network to directly predict the class label.

+ x-c-y: This baseline, on the other hand, employs a single branch that first uses the concept energy network to predict the concepts. Following this, it uses the global energy network to predict the class label.

The findings are as follows:

| Branch/Dataset |         |       CUB       |       |         |     CelebA      |       |         |      AWA2       |       |
| :------------: | :-----: | :-------------: | :---: | :-----: | :-------------: | :---: | :-----: | :-------------: | :---: |
|     Metric     | Concept | Overall Concept | Class | Concept | Overall Concept | Class | Concept | Overall Concept | Class |
|      x-y       |    -    |        -        | 0.825 |    -    |        -        | 0.265 |    -    |        -        | 0.909 |
|     x-c-y      |  0.968  |      0.680      | 0.726 |  0.870  |      0.464      | 0.175 |  0.979  |      0.864      | 0.905 |
|    **ECBM**    |  0.973  |      0.713      | 0.812 |  0.876  |      0.478      | 0.343 |  0.979  |      0.854      | 0.912 |
|      CBM       |  0.964  |      0.364      | 0.759 |  0.837  |      0.381      | 0.246 |  0.979  |      0.803      | 0.907 |
|      CEM       |  0.965  |      0.396      | 0.796 |  0.876  |      0.457      | 0.330 |  0.978  |      0.796      | 0.908 |


We have also included this table detailing these results in Table 4 of Appendix C for further reference.

From this new ablation study, we have the following observations: 

+ Our full model demonstrates superior performance when tested on the CelebA and AWA2 datasets, outperforming all other baseline models, and verifying the effectiveness of each component of our ECBM.

+ For the CUB dataset, our ECBM $x-y$ single branch surprisingly approaches the performance of the black-box model. However, it lacks concept interpretability. In contrast, our full ECBM can provide such concept interpretability, albeit with a slight decrease in class accuracy. Note that our full ECBM still outperforms all baseline models, including "x-y", "x-c-y", CBM, and CEM, verifying the effectiveness of each component of our ECBM.

Reviewer comment (Novelty):
The "related work" section about energy-based models is not complete. It misses some pioneering works and representative works to build a full history of the advancement of EBMs. For example, [2] [3] are pioneering EBM works, [4] also use an EBM for the distribution of labels (or other information) and data, which also captures high-level concepts and data. It is better to have a comparative discussion on both models if they share a similar concept.

Author response:
**W4. The "related work" section about energy-based models is not complete.**

We appreciate your insightful feedback on the 'Related Work' section of our paper. We agree that a more comprehensive review of the historical development and pioneering works of Energy-Based Models (EBMs) would enhance our discussion. We recognize the importance of the works you have cited, [2], [3], and [4], in the field of EBMs. Particularly, the use of an EBM for the distribution of labels (or other information) and data in [4] aligns closely with our research as it also captures high-level concepts and data. In light of your suggestions, we have updated our paper to include these seminal works. 

[2] A Theory of Generative ConvNet. ICML 2016

[3] Cooperative Training of Descriptor and Generator Networks. TPAMI 2018.

[4] Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning. TPAMI 2021.

Reviewer comment (Experiments):
Is it possible for the model to detect undefined 'concepts,' or is it capable of exploring new 'concepts'?

Author response:
**Q3. Is it possible for the model to detect undefined 'concepts,' or is it capable of exploring new 'concepts'?**

Thank you for this insightful question. Indeed, our model does have the potential to explore and identify new 'concepts'. Specifically, we can "reserve" additional "undefined" concept embeddings to allow the model to learn them, and introduce an inductive bias to ensure these embeddings are as distinct as possible from each other and from existing concepts. We believe this capability for concept discovery presents a promising direction for future research. In response to your question, we have incorporated a discussion on this aspect into the revision. 

We hope this response addresses your concerns adequately. We appreciate your insightful comments and look forward to further discussions on improving our work.