Paper: Interpretable Concept Discovery and Learning from Pretrained Vision-Language Models (ICLR-2024)

Reviewer comment (Evaluation):
Table 1 is used as a support of the paper's claim about the misalignment of concepts and classes in CLIP. However, there is not a similar table for CLIP+CDL using the same evaluation to show that it does improve the alignment between classes and concepts. Current experiments only show that VLM+CDL becomes a good concept detector but does not say that it effectively relates concepts with classes.

Author response:
In order to demonstrate the ability of our CDL model to align concepts with classes, we add the experiment to compare our CDL model with VDES on zero-shot classification in Table 1 (last row, “CDL + Concept”). The results show that our fine-tuned CLIP model can achieve competitive performance while using only class-agnostic concepts. Notably, when VDES is modified to rely only on its concepts, the performance drops significantly (the row “CLIP + Concept”). The results illustrate that our CDL method can learn correct concept and class association and provide interpretability on how VLMs utilize primitive concepts for recognition, and that CDL can indeed improve alignment between classes and concepts.

Reviewer comment (Novelty):
Mutual information-based concept selection is correct but not new. Moreover, I do not see the necessity of introducing MI here. The paper should better rationalise the choice of this criterion.

Author response:
As illustrated in Figure 1 (fifth column, “Labo”), we observe that previous methods include many non-visual concepts (e.g. “keep the panda warm”) in their discovered concepts. We hypothesize that the selection of such concepts would make the concept-based representation less visually discriminative when the concept vocabulary size is fixed. Hence, we introduce Mutual Information to select visually discriminative concepts. 

The Mutual Information between the image-concept similarity and the ground-truth containment of the image for the concept can measure how well the concept can be recognized by the VLM and can select **visually salient** concepts. If we use other methods like normalized co-occurrence frequency between categories and classes to select concepts, it still cannot distinguish visual and non-visual concepts. For example, the non-visual concepts like “long life span” and “a very rare animal” also usually co-occur with “giant panda”, and we need to introduce the vision information to filter them.  

To better showcase the effectiveness of the Mutual Information based concept selection, we add some examples of the concepts selected and excluded by the method in Table 9 of the Appendix. From the examples we can see that our method can effectively select visually discriminative concepts and exclude non-visual ones.

Reviewer comment (Presentation):
Is there any example showing the concepts selected for different classes?

Author response:
To better showcase the effectiveness of the Mutual Information based concept selection, we add some examples of the concepts selected and excluded by the method in Table 9 of the Appendix. From the examples we can see that our method can effectively select visually discriminative concepts and exclude non-visual ones.

Reviewer comment (Reproducibility):
Missing details about the similarity threshold ``$\texttt{th}$'' (page 5). There is no further discussion on it except in Section 3.2.

Author response:
We fully agree that adding more details of the similarity threshold is necessary. The similarity threshold is 0.9 for our current method. Intuitively, a lower threshold will select fewer concepts and a higher threshold will include more similar concepts. We add the details of the similarity threshold selection in the experiment section and show the performance of different thresholds on the CUB dataset in Table 6 of the Appendix. The results show that the performance of the CDL model is reasonably robust against the choice of the threshold.

Reviewer comment (Experiments):
Is the framework sensitive to the similarity threshold ``$\texttt{th}$'' parameter?

Author response:
We fully agree that adding more details of the similarity threshold is necessary. The similarity threshold is 0.9 for our current method. Intuitively, a lower threshold will select fewer concepts and a higher threshold will include more similar concepts. We add the details of the similarity threshold selection in the experiment section and show the performance of different thresholds on the CUB dataset in Table 6 of the Appendix. The results show that the performance of the CDL model is reasonably robust against the choice of the threshold.

Reviewer comment (Experiments):
Missing details about human study.

Author response:
Thank you for your suggestions on the presentation of the human study.We agree that more details about human study need to be included. 
- Error bars about human evaluation: we conduct the Students’ T-test to evaluate the statistical significance of the human evaluation results. The results are shown in Table 8 of the Appendix. From the results we can observe that both our concept learning and concept discovery method significantly outperform the baseline methods regarding the intervention, factuality and groundability metrics. We also report the pairwise annotator agreement score of our human evaluation to validate the effectiveness of our human evaluation.
- Potential biases in the annotation process: for the annotation, we randomly shuffled the order of instances to remove possible biases. We add the examples of human annotation interfaces in Figure 5 of the Appendix. Since the annotator receives one image each time, the position will not bring biases to the result.

Reviewer comment (Evaluation):
There are no error bars in Figure 4, making it difficult to tell how well the human annotators agree and how significantly CDL improved over prior methods. Since each data point has been annotated by three human annotators (as described in the appendix), there should be at least an error bar for each result in the figure.

Author response:
Thank you for your suggestions on the presentation of the human study.We agree that more details about human study need to be included. 
- Error bars about human evaluation: we conduct the Students’ T-test to evaluate the statistical significance of the human evaluation results. The results are shown in Table 8 of the Appendix. From the results we can observe that both our concept learning and concept discovery method significantly outperform the baseline methods regarding the intervention, factuality and groundability metrics. We also report the pairwise annotator agreement score of our human evaluation to validate the effectiveness of our human evaluation.
- Potential biases in the annotation process: for the annotation, we randomly shuffled the order of instances to remove possible biases. We add the examples of human annotation interfaces in Figure 5 of the Appendix. Since the annotator receives one image each time, the position will not bring biases to the result.

Reviewer comment (Experiments):
There is no information about the interface viewed by the annotators. For example, are samples fairly displayed to the annotators, removing the possible bias caused by the order or the position of display?

Author response:
Thank you for your suggestions on the presentation of the human study.We agree that more details about human study need to be included. 
- Error bars about human evaluation: we conduct the Students’ T-test to evaluate the statistical significance of the human evaluation results. The results are shown in Table 8 of the Appendix. From the results we can observe that both our concept learning and concept discovery method significantly outperform the baseline methods regarding the intervention, factuality and groundability metrics. We also report the pairwise annotator agreement score of our human evaluation to validate the effectiveness of our human evaluation.
- Potential biases in the annotation process: for the annotation, we randomly shuffled the order of instances to remove possible biases. We add the examples of human annotation interfaces in Figure 5 of the Appendix. Since the annotator receives one image each time, the position will not bring biases to the result.

Reviewer comment (Writing):
Clarity issue: what is the "CDL+CDL" setting in Table 2?

Author response:
For the notations `A+B` in Table 2, `A` represents the CLIP model and `B` represents the concept discovery method. `CLIP+LaBo` represents the original CLIP model with LaBo concepts. `CLIP+CDL` represents the original CLIP model with our discovered concepts. `CDL+CDL` represents our fine-tuned CLIP model with our discovered concepts.

Reviewer comment (Evaluation):
This paper found out that the classification accuracy drops significantly when the input prompt for the text encoder is without the class name. However, not using the class names may not be a large problem. The key concept of using CLIP is based on the contrastive learning between image-text pairs, which is powerful. As such, it is reasonable that the class names associated with the descriptions improvement the classification. Moreover, the model with class names can provide the correct description, e.g., the examples in Fig. 1.

Author response:
This is a great point! We agree with the reviewer that class names are helpful for classification and can be used to improve empirical performance on zero-shot classification benchmarks. We would like to clarify that our main point is that the use of class names may bias the analysis on whether VLMs (e.g. CLIP) are able to learn and use concepts (e.g. black patches around the eyes). Specifically, the concepts retrieved by VDES may be correct for two reasons: First, CLIP captures the concept and uses it for zero-shot classification. Alternatively, CLIP uses the class name for zero-shot classification, while the concepts are only retrieved because GPT-3 indicates the concepts are correlated with the class names. 

To better illustrate the point, we revise Figure 1 and show a scenario where VDES is applied on class names with randomly shuffled concepts (e.g. giant panda, which has black wings), and we can observe that the class names are correctly retrieved despite having completely irrelevant concepts. This observation indicates that the class names instead of the descriptive concepts make the decisive contribution to the recognition. We further validate this observation in Table 1 (fourth row, `CLIP + Name w/ Random Concept`), which shows that pairing the class names with randomly shuffled irrelevant concepts do not hurt the zero-shot classification performance.

Reviewer comment (Experiments):
The method for alleviating the problem is to use different prompts, e.g., “What are useful visual features for distinguishing a {category name} in a photo?,” which is heuristic. Another alternatives should be considered and compared.

Author response:
We fully agree that the comparison between different prompts is important to show. We have conducted comparisons between different alternatives for the prompts for concept discovery. The results are in Table 5 of the Appendix. From the results we can observe that different prompts provide similar performance, which demonstrates the robustness of our observations.

Reviewer comment (Experiments):
The experiments should follow the similar setting with previous work. It is suggested to compare with VDES (and following works) and show the results with different backbones.

Author response:
We have added this suggested experiment in the revised Table 1 (last row). The row `CLIP + Name w/ Concept` represents the framework of VDES and the row `CDL + Concept` represents our fine-tuned CLIP with the discovered concepts. The results show that our fine-tuned CLIP model can achieve competitive performance while using only class-agnostic concepts. Notably, when VDES is modified to rely only on its concepts, the performance drops significantly (the row `CLIP + Concept`). The results illustrate that our CDL method can learn correct concept and class association and provide interpretability on how VLMs utilize primitive concepts for recognition.

Reviewer comment (Novelty):
The references are out-of-date. Only two papers published in 2023 are cited. Moreover, VDES is published in ICLR 2023, while the reference in the paper is still an arxiv paper.

Author response:
Thank you for pointing it out. We have referred to more recently published works in the related work section and corrected the wrong references. Now we have 10 references published in 2023.

Reviewer comment (Writing):
It is suggested to provide a little bit more details about previous work (Menon & Vondrick, 2022) to improve the readability.

Author response:
Thank you for your valuable advice on our presentation. We have added more details for VDES in Section 4.1 of our revision.