Paper: Faithful Vision-Language Interpretation via Concept Bottleneck Models (ICLR-2024)

Reviewer comment (Experiments):
I have one concern for the generalizability of the proposed solution. Although the authors claimed that they proposed a solution to deal with the stability issue of the label-free concept bottleneck model, the authors only focused on the prediction task over the vision dataset. I am wondering whether the proposed solution could be generalized to other settings such as the classification tasks over NLP data or tabular data.

Author response:
Briefly speaking, the main idea and our faithful concept can be directly extended to other forms of data such as text since they do not rely on the specific form of images. However, we are not sure about the performance of other data and this could left for future work. In fact, we think this should not be a reason that our paper will be rejected as to the best of our knowledge, most of the previous work on concept bottleneck models only considers image data and there is less study on other data types [1,2,3]. But it is certain that our method can migrate data to the remaining modes. Because we are based on the output of the feature layer for subsequent operations.
[1] Yang, Yue, et al. "Language in a bottle: Language model guided concept bottlenecks for interpretable image classification." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
[2] Chauhan, Kushal, et al. "Interactive concept bottleneck models." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 5. 2023.
[3] Yuksekgonul, Mert, Maggie Wang, and James Zou. "Post-hoc concept bottleneck models." arXiv preprint arXiv:2205.15480 (2022).

Reviewer comment (Evaluation):
The metrics of evaluating the model stability are also concerning to me. I feel that this metric should be $\epsilon$-invariant, which means that ideally the perturbation $\epsilon$ should occur in the denominator of the metric. Otherwise, with larger epsilon, you can get larger stability, which seems to be counter-intuitive. Also, by looking at the other similar metrics in literature, such as the stability metric proposed in the model interpretability literature (Alvarez Melis, David, and Tommi Jaakkola. "Towards robust interpretability with self-explaining neural networks." Advances in neural information processing systems 31 (2018).), they are also calculated by putting the input perturbation in the denominator. So it would be better if the authors could justify why they didn't choose to design their stability metric in this manner and ideally reference some other papers.

Author response:
Thanks for your question. There are mainly two reasons we do not use invariant metrics: (1) Note that unlike the previous paper (such as the one you mentioned) which only considers adding noises to image inputs as the perturbation, in our experiments, we consider (a combination of) three different perturbations. It is hard to normalize these three perturbations on the same scale, such as the word perturbation.  (2) Note that in our experiments, due to different combinations of perturbations, we do not intend to discuss the change of stability w.r.t. perturbation scale. Our main goal is to show that under the same scale of perturbation, our FVLC is more stable than baselines (Table 2). Thus, there is no need to consider the invariant metrics.

Reviewer comment (Presentation):
The presentation could be further improved. For example, for Figure 3, the text in the figure is very tiny and it is very hard to tell what concepts they are. For Figure 2, I guess the box mixed with black and white dots should represent the noise but I cannot find any explanations for this. It would be better if the authors could clearly illustrate this figure in the caption.

Author response:
Thanks for your suggestion. We will make the text in Figure 3 larger in the final version if additional pages are allowed. Moreover, in the revised version, we enlarge the fonts in Figure 3, see Figure 5-8 in Appendix for details. 
In the revised version, we also added more details to all figures (including Figure 2) in our paper.

Reviewer comment (Novelty):
The related work is not complete. The following important papers should have been included in the related work:

[1] Concept Embedding models. Zarlenga et al. Neurips 2022  
[2] Interpretable Neural-Symbolic Concept Reasoning. Barberio et al. ICML 2023  
[3] Entropy-based Logic Explanations of Neural Networks. Barberio et al. AAAI 2021  
[4] Addressing Leakage in Concept Bottleneck Models. Havasi et. al., Neurips 2022  
[5] Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat. Ghosh et al. ICML 2023  
[6] Distilling BlackBox to Interpretable models for Efficient Transfer Learning. Ghosh et al. MICCAI 2023  
[7] Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification Yang et al. CVPR 2023  
[8] A Framework for Learning Ante-hoc Explainable Models via Concepts Sarkar et al. CVPR 2022

Author response:
Thanks, we have added these references to the related work section in the revised version.

Reviewer comment (Writing):
What is the difference b/w faithful and original concept in definition 1?

Author response:
**Generally speaking, our faithful concept is more stable on prediction and interpretability against different perturbations than the original concept in Label-free CBM, while it also keeps the prediction performance and interpretability of the original concept.** To give a better illustration, in Appendix A of the revised version, we provided more details on the intuition of our definition 2. Next, we will show more of the difference between faithful and original concepts in definition 1. 

(1) The first condition ensures that the faithful concept keeps almost the same interpretability as in the original concept for each input. This is modeled via the top-k indices overlap between these two vectors is large. 

(2) The second condition ensures that our faithful concept is stable for explainability w.r.t. different perturbations, especially input perturbation and word perturbation. This is significantly different from the original concept. As illustrated in Figure 1, the interpretability of the original concept is very unstable. 

(3) The third condition ensures that the faithful concept keeps almost the same prediction output as in the original concept for each input. This is modeled via the distance or divergence between the prediction distributions of the faithful concept and the original concept. 

(4) The last condition guarantees that our faithful concept is stable for the prediction w.r.t. different perturbations. This is also motivated by the fact that the original concept may have wrong predictions with perturbations.

Reviewer comment (Writing):
Label free cbm step 1, page 3. I think it will be "they propose" instead of "we propose" as label free cbm is proposed by Oikarinen et al., 2023. All the in step 1, 2, 3, 4 in sect 3 "we" should be replaced by "they".

Author response:
We have corrected it in section 3.

Reviewer comment (Presentation):
I don't understand anything from fig 2. The caption should clearly state the objective of the figure.

Author response:
In the revised version, we have added more details on all Figures including Figure 2.

Reviewer comment (Writing):
The terms in the loss function are not at all clear (Page 6). What is D? Is this some distance? Also how will the two constraints \delta and \phro affect the loss and the optimization? And its connection with Fig.2 is completely unresolved to me.

Author response:
$D$ should be a divergence distance, which is defined in Definition 2. In our experiments, we use KL divergence.  For the role of two constraints on $\delta$ and $\rho$, please refer to Stability and Prediction on Page 5. In the revised version, we have added more details for our optimization loss. Moreover, we polished Appendix Section B with details on the optimization procedure for the relaxed problem.

Reviewer comment (Experiments):
Can the authors use any difficult datasets like HAM10k or MIMIC-CXR to bolster their claims. Medical images will make the claims stronger because if the explanations are not stable, it will raise trust issues among the users.

Author response:
We have added some experiments on the HAM10000 dataset to improve the reliability of our model, as detailed in Appendix G, Table 12 and Figure 9 in the revised version. We hope these changes will make you satisfied. If not, please feel free to reply to us.

Reviewer comment (Experiments):
I am confused with table 1. If the main message is to showcase that their method is more superior compared to the baselines, they should have the best configuration of their method and the baselines. Also they should have another table where they should conduct an ablation study to include different configurations of their method with various perturbations and compare the performance. Also, only PCBM and LCBM baselines are not enough. I would like to see the performances for LCBM and language in a bottle especially when the inputs and the concepts will be perturbed. Also thorough comparison is needed with basic CBM and some other methods for datasets where concepts are annotated. Also missing variances in table 1.

Author response:
(1) For more details of the experiment, such as baseline, etc., we have made additional supplements, which are detailed in Table 5 of Appendix D. We hope these changes will make you satisfied. If not, please feel free to reply to us.

(2) We conducted a set of ablation experiments to compare the performance, and we did not seem to find that our optimization had a significant impact on the experimental image classification performance, but it could effectively improve the stability of the output concept (see in Table 2 and Table 3). Due to time constraints, we only conducted supplementary ablation experiments on the CIFAR10 dataset (WP1(10%)+WP2+IP - FVLC). 
| L2   | L3   | L4   | CIFAR10 |
| ---- | ---- | ---- | ------- |
|      |      |      | 86.69%  |
| √    |      |      | 87.70%  |
|      | √    |      | 87.73%  |
|      |      | √    | 86.67%  |
| √    | √    |      | 86.73%  |
| √    |      | √    | 86.70%  |
|      | √    | √    | 86.71%  |
| √    | √    | √    | 86.70%  |

(3) Although they are both concept based models, their frameworks are very different, as are the output concepts and the forms of concept weights. It is meaningless to discuss the anti-interference ability of the concept that interpretability outputs differ between models. So we only compared the performance of the model, and we also compared the performance of P-CBM, LCBM and our model.

(3) It is notable that it is impossible to compare our method with previous methods with annotated concepts: (1) Our concept set is generated by LLM with previous work needs to use concept sets given by experts. Since the concept sets are different, they are incomparable. (2) It is notable that the main motivation of our methods is label-free CBM so there is no ground truth of concepts. And it is meaningless to compare with previous methods. 

(4) We repeated the last group of experiments in Table 1 for 5 times, and calculated the mean and variance of the output data in this group as shown below. The experimental results show that the variances of our experimental data is very small. Because of time, we did not repeat all the experiments to calculate the variances. We hope these changes will be satisfactory to you. If not, please feel free to reply us with more experiments and explanations.
| CIFAR10 | CIFAR100 | CUB | Places365 |
| ------- | ------- | ------- | ------- |
| 86.82%(2.15E-06) | 65.15%(1.11E-06) | 74.30%(5.03E-07) | 43.67%(2.19E-05) |

Reviewer comment (Presentation):
For table 2, nothing is highlighted or bolded. With so many numbers, it is difficult for me to see which is higher and which is lower.

Author response:
In the revised version, we have highlighted all the results of our method in grey color in all tables. Please check the revision.

Reviewer comment (Writing):
The quality of the explanations for some parts of the paper could be enhanced.

Author response:
In the revised version, we have added more details to all figures (including Figure 2) in our paper.

Reviewer comment (Writing):
It took me a significant amount of time to follow the flow of data through the network. Perhaps adding concise explanations detailing how input images traverse the network during inference would make the explanation more accessible.

Author response:
In the revised version, we have added a notation Table 4 in Appendix (page 13). We will move it to the main context if additional pages are allowed.

Reviewer comment (Experiments):
Can the authors explain the choice of architecture for the text and image encoder and its impact on the model's performance, particularly in terms of accuracy?

Author response:
Our text encoder and image encoder make up CLIP. The text encoder of CLIP is used to convert natural language text into vector representations. It is based on the Transformer model architecture, which learns to embed text into a high-dimensional vector space, where text with similar semantics are closer in the vector space. The text encoder transforms input text sequences into fixed-dimensional vector representations, capturing the semantic and contextual information of the text. The image encoder of CLIP is used to convert images into vector representations. It employs a convolutional neural network (CNN) as the backbone model to extract image feature representations. These feature representations are mapped to a fixed-dimensional vector space for comparison with the vectors from the text encoder.
By using the pre-trained CLIP text encoder and image encoder, our model is capable of understanding both images and text (concepts). By encoding images and text into the same vector space, CLIP quantifies the semantic similarity between images and text, enabling joint encoding and understanding across modalities. CLIP leverages large-scale pre-training datasets, allowing it to learn rich semantic representations. The advantage of pre-training is the ability to learn general-purpose feature representations from large-scale data. It is important to emphasize that we did not train the image encoder and text encoder ourselves.
We use the encoder to obtain the image-concept matrix to complete the concept-neuron (feature) mapping. In fact, this is the process that affects the interpretability of the model, and whether the choice of encoder affects accuracy is something we have not discussed. Because we couldn't find another image-text encoder model that was pre-trained on large-scale datasets for comparison.