Extracting Interpretable Textual Descriptions for Image Classification by Vision-Language Models
Authors: Pingchuan Ma, Lennart Rietdorf, Dmytro Kotovenko, Vincent Tao Hu, Björn Ommer


Reviewer paper summaries: The paper presents a novel method to improve CLIP-based image classification performance by selecting (generating) semantically meaningful, class-name-free descriptions. By utilizing distinctive class-name-free textual attributes, the paper could ensure that performance gains are mostly due to semantic understanding. In addition, this approach enhances the efficiency of description utilization, achieving higher accuracy with fewer descriptions.
Summary of strengths: * The motivation for using class-name-free descriptions is convincing and has not been addressed in previous studies. It is impressive that the paper achieves high performance and solid motivation while maintaining human interpretability.
* The paper provides a comprehensive literature review and detailed methodology.
Summary of weaknesses: * There is no significant weakness.
Comments, suggestions and typos: * Please consider providing examples of accepted and rejected (filtered out) descriptions from DCLIP or FUDD for better readers’ understanding.
* For the contrastive setup, it is assumed that all target class names are known (so that the proposed algorithm can identify distinctive ones). It would be interesting to see the robustness of descriptions for unseen classes.
* (typo) In Table 4, Food101, one cell has an error.
Overall assessment scores: 4.5