Pre-trained Vision-Language Models Learn Discoverable Visual Concepts
Authors: Yuan Zang
Tian Yun
Hao Tan
Trung Bui
Chen Sun
Venue: Accepted by TMLR ()

Abstract: Do vision-language models (VLMs) pre-trained to caption an image of a durian learn visual
concepts such as brown (color) and spiky (texture) at the same time? We aim to answer
this question as visual concepts learned “for free” would enable wide applications such as
neuro-symbolic reasoning or human-interpretable object classification. We assume that the
visual concepts, if captured by pre-trained VLMs, can be extracted by their vision-language
interface with text-based concept prompts. We observe that recent works prompting VLMs
with concepts often differ in their strategies to define and evaluate the visual concepts,
leading to conflicting conclusions. We propose a new concept definition strategy based on
two observations: First, certain concept prompts include shortcuts that recognize correct
concepts for wrong reasons; Second, multimodal information (e.g. visual discriminativeness,
and textual knowledge) should be leveraged when selecting the concepts. Our proposed
concept discovery and learning (CDL) framework is thus designed to identify a diverse list
of generic visual concepts (e.g. spiky as opposed to spiky durian), which are ranked and
selected based on visual and language mutual information. We carefully design quantitative
and human evaluations of the discovered concepts on nine diverse visual recognition datasets,
which confirm that pre-trained VLMs do learn visual concepts that provide accurate and
thorough descriptions for the recognized objects. All code and models are publicly released.

Published in Transactions on Machine Learning Research (01/2025)
Pre-trained Vision- Models Learn Discoverable Visual Concepts
Yuan Zang
yuan munn_=aag@@bruum.c
Deportment of Computer Science
Brown University
Tian Yun
tian jom___yn@@broom.m edu
Department Computer Science Brown University
Hao Tan Adobe Research
haten@adobe..c com
Trung Bui Adobe Research
bui@adabe. com
Chen Sun
chensun@broun.cc edu
Department of Computer Science Brown University
Reviewed on OpenReview: https: // o/peneveepperreveee. net/ forumm id- Jg0uMFB
Abstract
Do vision-language models (VLMs) pre-trained to caption an image of durian learn visual concepts such brown (color) and spiky (texture) at the same time? We aim to answer this question visual concepts learned "for free" would enable wide applications such as neuro-symbolic reasoning human-interpretall object classification. We assume that the visual concepts, if captured by pre-trained VLMs, can be extracted by their vision-language interface with text-based concept prompts. We observe that recent works prompting VLMs with concepts often differ in their stralegies define and evaluate the visual concepts, leading to conflicting conclusions. We propose a new concept definition strategy based on two observations: First, certain concept prompts include shortcuts that recognize correct concepts for wrong reasons; Second, multimodal information (e.g. visual discriminativeness. and textual knowledge) should be leveraged when selecting the concepts. Our proposed concept discovery and learning (CDL) framework is thus designed to identify diverse list of generic visual concepts (e.g. spiky as opposed to spiky durian), which are ranked and selected based on visual and language mutual information. We carefully design quantitative and human evaluations the discovered concepts on nine diverse visual recognition datasets, which confirm that pre-trained VLMs do learn visual concepts that provide accurate and thorough descriptions for the recognized objects. Code and models are publicly released.
1 Introduction
Can a vision-
vision-language language model (VLM) pre-trained on images of California seagull learn visual concepts (Lake
et al. 2015 2011 Lee et al. 2023 Sun et. al. 2015),, such as yellow legs and white belly, to describe an
image a Gentoo penguin, which the model might not see during training? Visual concepts such as color,
shape, and texture help models generalize compositionally (Farhadi et al. 2009 Nagarajan & Grauman
Prooeec page and cade: htthss//moneeeddssseerytthhhhhhh

---

Published in Transactions on Machine Learning Research (01/2025)
VOES, CLIP aligns image with concepts:
removing category names, CLIP eligns the image with concepts:
randomly combine category nemes with concepts, CLIP aligns the image with:
COL, aligns the image with:
California Seagull
California Gull, which has a yellow ring around its black eyes
fish-eating
California Gull, which has a black cap with white feather streaks
yellow legs
California Gull, which 1s a fish-eating bird
long.narrow wings
California Gull, which 1s a colorful bird
black, tapered tell
California Gull, which has a long, black tail
necturnal of prey gray belly
California Gull, which has yellow eyes
white belly
California Gull, which has gray back wings
California Gull, which is four-1inbed
white bird with short neck
Figure The top-sclected text prompts by CLIP given a query image, VDES proposed by Menon & Vondrick (2022), whereas CDL is our proposed approach. The design of concept prompts plays critical role understanding whether VLMs learn visual concepts. We can observe that concept-angumented prompt predict correct visual concepts (e.g., gray back and wings) when the prompt is associated with the category name (California seagull). When the category name is removed from the prompt (Column 2), the retrieved concepts are either non-visual or incorrect. We altribute this to the calegory name bias (Column 3), as the correct category can be retrieved by CLIP even when the paired descriptions are randomly shuffled and thus irrelevant. We propose concept discovery and learning (CDL) framework and demonstrate that pre-trained VLMs can indeed learn visual concepts (e.g., Column 4). Correctly predicted concepts are in green wrong concepts are in red, and non-visual concepts are in violet. Category names are in orange
2018 Hsieh et al. 2024 Thrush et al. 2022 Stein et al. 2024), and can be incorporated into neuro- symbolic frameworks (Mao et al. 2019 Yi al. 2018) or offer concept-basod explanations for classification
decisions (Koh et al. 2020
Our paper aims to investigate if VLMs, such as CLIP (Radford et al. 2021), learn visual concepts auto- matically when pre-trained on image and text pairs with contrastive learning objectives. We hypothesize that the visual concepts, if captured by the pre-trained VLMs, can be directly extracted by prompling their vision-language interface, without needing to probe their internal representations. Our research question can thus be formulated as discovering the visual concepts encoded by pre-trained VLMs, and evaluating the quality of the extracted concepts.
Interestingly, several recent works (Menon & Vondrick 2022 Yang et al. 2023b Yun et al.
2023 reached Yun et al. (2023) observed that CLIP does not appear to recognize fine-grained visual attributes for birds Wah et al.
different conclusions on the encoding of visual concepts in pre-trained VLMs. For example,
2011 where the list of visual attributes is pre-defined by bird experts. In contrast, Menon & Vondrick
(2022) demonstrated that object prompts with visual concepts proposed by a large language model (LLM) appear to provide interpretable object classification, as the concept descriptions are nicely correlated with the recognized object categories. As illustrated in Figure [ we attribute these discrepancies to their different strategics for extracting visual concepts in a pro-trained VLM: First, according to the first and second columns of Figure D certain shortcuts (e.g., the object category California seagull) may dominate the text prompts when recognizing visual concepts, reinforcing the object-concept correlations as given by a prior knowledge base, such as LLMs. A possible remedy is to discover general visual concepts shared by multiple objects, without including the shortcuts in a text prompt. Second, according to the third column in Figure I some of concepts proposed by LLMs are not visually discriminative (e.g., "a nocturnal bird") or cannot be reliably recognized by VLMs (e.g. concept prompts marked red, or the expert prompts used by Yun et al. (2023)). They should be excluded from the list of visual concepts. More generally, in order to draw convincing conclusion on whether pre-traine pre-trained VLMs learn to encode visual concepts, one should discover the list of visually discriminative concepts and use them to prompt VLMs in a shortcut-free manner, and then measure the quality and thoroughness of the discovered concepts, when they are tasked to recognize fine-grained objects and generate interpretable explanations with visual concepts.
In this paper, we propose novel concept discovery and learning framework that extracts shortcut-free and visually discriminative concepts from pre-trained VLMs. To discover general concepts that do not include category specific shortculs, we propose to utilize large and diverse image captioning dataset to discover
2

---

Published in Transactions on Machine Learning Research (01/2025)
general concepts shared by multiple objects. To select concepts that are well-recognizable by VLMs, we propose to jointly consider LLM and VLM information during concept discovery. We prefer the visual concepts that can be both reliably recognized from the images by a VLM, and deemed as suitable descriptions for the objects of interest in the images by an LLM. Furthermore, although visual concepts can be discoverod from frozen pre-trained VLMs, they are not always perfectly aligned with images (see Column 2 in Figure] I To improve the quality of selected concepts, we propose a self-supervised method to fine-tune the last projection layer of VLMs by leveraging the knowledge already encoded in pre-trained VLMs and LLMs. We name our overall framework as Concept Discovery and Learning (CDL).
We design a suite of evaluation protocols for the quality of the extracted concepts. Intuitively, the desirable visual concepts should be precise, to faithfully reflect the visual concepts presented in an image; They should also be thorough, so that the concepts can provide most of the distinctive visual features for an object of interest. Furthermore, these concepts should be generalizable, to effectively describe new and unseen objects. According to the experimental results, the proposed CDL framework effectively discover and learn precise, thorough and generalizable concepts. The discovered concepts lead to accurate and interpretable object recognition on nine visually diverse benchmarks. The results demonstrate that pre-trained VLMs can indeed learn visual concepts via its vision-language interface, and these visual concepts can lead to performant and interpretable visual recognition.
2 Related Work
Vision-and sision-and-laat aud-language models (VLMs) pretrained on unlabeled pairs of images and texts from the internet have shown great success on multi-modal benchmarks. Basod on the pre-training objectives, VLMs can be broadly categorized into two primary types: contrastive VLMs and generative VLMs. Contrastive VLMs (Radford et al. of images and texts by leveraging contrastive learning techniques CChen et al. 2020j. Generative VLMs
2021 Alayrac et al.) 2022 Jia et al., 2021 Yao et al. 2021 focus on learning joint representations
Lu et al. (2019); Wang et al. (2023); Chen et al.
(0023); Liu et al. (2024) aim to generate coherent text
descriptions from images. Representations learned by
these VLMs can be transferred to a wide range of
tasks, such as image classification (Pratt et al. 2023 visual question answering (Li et al. 2023 Bai et al.
2023), and image and video captioning (Zhang et al. 2021 Yang et al., 2023a).
Visual concepts, which represent the fundamental factors of variations (e.g. colors, shapes) (Hu et al. 2018
in the visual world, have been widely utilized to develop intepretable visual models that can generalize lize
compositionally (Farbadi et al. 2009: Lampert et al. 2009
Nagarajan & Grauman 2018 Stein et al. 2024]
Previous works (Sun et al. 2015 Hernandez et al. 2021 Lee et al.. 2023) have proposed to discover textual terms associated to visual concepts to interpret visual systems. These concepts can be integrated into ncuro-symbolic frameworks [Ma et al. 2019 Yi ct al. 2018) and offer concept-based explanations for visual recognition (Koh et al. (2020). Contrastive VLMs, learning joint representation for images and lexts, provide a natural interface for discovering such visual concepts. However, it remains unclear whether VLMs pre- trained with contrastive objectives can learn discoverable visual concepts. Previous studics have explored the capability of VLMs to capture and compose primitive concepts (Yun et al.
2023
Yuksekgonul et al.
2022a) and bind visual concepts with objects (Lewis et al., 2022: Yamada et al. demonstrate that VLMs do not capture composable primitive concepts by intervening a linear classifier learned from VLM-predicted concepts. Mcanwhile, previous works (Pratt et al. 2023
2022
Yun et al. (2023)
Menon & Vondrick
2022) show that concept-augmented prompts do provide improvements for VLM-basee image recognition. In this paper, we aim to address this discrepency and understand pretrained VLMs' true capability of encoding interpretable visual concepts.
To interpret the decision of visual models with visual concepts, Koh et al. 2020 proposed Concept Bottleneck Model (CBM) to decompose the visual recognition into concept classification and concept-category mapping. While an alternative approach (Bau et al. 2017 Kim el al. 2018 Hernandez el al. 2021 aims to develop host-hoc interpretation for visual models by directly analyzing the internal representation of the models within the concopt space, the CBM does not rely on the model having already learned concopts, instead, it simultancously trains the concept classifier and linear concept-category mapper upon the model being analyzed with the image classification objective. As result, the CBM can evaluate the concept learning
3

---

Published in Transactions on Machine Learning Research (01/2025)
ability of a model through the concept-category map, regardless of whether the model was previously trained to learn those concepts. Recent works have developed various methods to enhance the performance and adaptability of CBMs (Yuksekgomul et al., 2022b Hu et al. 2024) and have applied them to machine learning tasks that require high transparency, such as medical diagnosis (Yan et al.) 2023b Chauhan et al., 2023 However, previous CBM-based approaches lypically rely on concept classifiers trained with supervision. this paper, we propose to build CBMs based on the pre-trained VLMs to evaluate their concept learning ability. Assuming that pre-trained VLMs inherently learn visual concepts, we directly extract concept-image similarities from the VLM encoders to serve as the concept classifier and only train one-layer concept-to- category mapper.
3
Method
We first describe a framework for extracting visual concepts from pre-trained VLMs and its application for object recognition in Section 3.1 We show in Section 3.2 that concepts which are non-visual or include certain shortculs might lead to wrong correlations, where the concept activations do not correspond to how likely the visual concepts are actually present in an image. In Section 3.3 we propose a concept discovery method to select shortcut-free visual concepts from a large image captioning dataset by utilizing VLM and LLM knowledge to evaluate the visual discrimination. We also propose a self-supervised learning framework to re-align the image-text interfaces of VLMs to improve the quality of selected concepts. In Section 3.4 we describe the method to select a compact set of concepts for specific object recognition tasks. In Section 3.5 we propose a suite of quantitative and human evaluations on the interpretability, precision, thoroughness, and generalizability of the discovered visual concepts. Together they help us understand if pre-trained VLMs learn to encode visual concepts.
3.1 Object Recognition via Visual Concepts
Vision-language models such as CLIP (Radford et al. 2021 jointly learn an image encoder &g and text encoder & to align images and texts in shared embedding space. Thanks to the flexibility of the image- language interface, several recent works (Pratt et al.) 2023 Menon & Vondrick 2022 Yun et al.) 2023; Yang et al. 2023b: Yan et al.] 2023a) attempted to construct semantically interpretable representations by projecting an encoded visual embedding with basis defined by encoded text embeddings. The text embeddings are obtained by encoding manually designed or automatically generated text "prompts" that are likely to correspond to visual concepts. Concretely, given an image and a set of concept prompts F of N concepts, one can project the visual features Ez( into the space of concepts as an n-dimensional concept activation vector a = [41, .2,..., a.)- Each concept activation is computed as a = sim(Ez(T), (p()) (1), where p; is the i-th concept prompt, and sim(-) is similarity function, such as cosine similarity, that measures the similarity between the encoded visual and text embeddings. The activalions are standardized with s- score normalization to ensure comparability across objects. Our paper assumes that the visual concepts, if well learned by a VLM, can be extracted as concept activations via the text prompts.
The concept activations can be utilized for multi-modal object recognition with function f JR RM that predicts object categories, where M is the total number of categories. When is a linear function f(a) a. W. the learned N x M weight matrix W allows us to interpret how the visual concepts are utilized for object recognition. A higher positive weight Wiy indicates the i-th concept is deemed as important positive evidence for recognizing the j-th object category, and a near-zero weight indicates the concept is deemed as irrelevant. The linear classifier that maps concept activations to categorical predictions is referred to as Concept Bottleneck Models (Koh et al.) 2020) (CBM), which have been adopted to study concept learning with VLMs in Yang et al. (2023b); Yun et al. (2023): Yan et al. (2023a). For zero-shot object recognition when f(-) cannot be learned from data, some works (Pratt ct al. 2023 Menon & Vondrick) 2022) assume the visual concepts are category-specifie (hence the object names are included in the prompls), and simplify s(-) to be linear function fa = |e Eaje, a;, where C2 is the set of visual concepts for the j-th category, and C, nC+ = 0. We observe that this assumption does not hold for real-world datasets as many visual concepts are often shared by several objects, as illustrated in Figure]
4

---

Published in Transactions on Machine Learning Research (01/2025)
3.2 Do Prompted Activations Correspond to Visual Concepts?
Large language models (LLMs) are often utilized as the knowledge source to propose the relevant visual concopt prompts given an object category (Pratt et al. 2023 Menon & Vondrick 2022 Yang et al.] 2023b; Yan el al. 2023a One example is illustrated in Figure 1 where for California Gull, concept prompts such "California. Gull, which has a long, black tail" are proposed. As discussed in Section 3. the linear function (() allows us to identify the most important visual concepts for object recognition by picking the highest weighted Wij concepts i's for object j. One can then consider two proxy evaluations to measure whether the prompted concept activation ag actually corresponds to the visual concept i that is present in an image: First, by measuring the object classification accuracy, and assuming that the higher the accuracy is, the more precise the concept activations are. Second, by comparing the top ranked concepts for an image of a certain object category and those identified by human experts, which allows us to understand not only the precision but also the thoroughness of the concepts.
We observe that these proxy evaluations, while intuitive, require careful design of concept prompting strategy in order to draw conclusions on whether the concepts are actually encoded by the pre-trained VLMs. The first issue we identify is the existence of certain shortcuts in the text prompts, loading to "false positive" conclusions. For example, Figure [ illustrates the concepts with the highest activations according to CLIP. Although the first column appears to indicate that most of the selected concepts are semantically correlated with the input image of a California Gull, it remains unclear whether the concepts are retrieved because they are recognized by CLIP the class name is utilized investigate this potential shortcut: In the second column, we observe that when we combine the calegory names with randomly shuffled descriptors, CLIP tends to align images with concepts that contain correct category names and wrong descriptors. This phenomenon indicates that the category names, instead of the visual concepts, are used to generate the concept activations. We validate these qualitative observations in Section 4.2 where we demonstrate consistently across nine datasets that the zero-shot classification accuracy remains similar when LLM--enerated concepts or randomly shuffled concepts are paired with category names, and that the accuracy drops significantly when category names are removed from the text prompts.
shortcut. We perform simple ablation to
A second issue is the existence of sub-optimal text prompts, leading to "false-negative" conclusions. For example, the concepts marked with purple background in Figure | are not visually recognizable (e.g. "noe- turnal bird"), and hence should not be considered as visual concepts. Similarly, we observe that the text prompts used by Yun et al. (2023) are designed by birding experts, which may not be in a friendly format to serve as text prompts (c.g. "crested head pattern").
In order to address both issues, we propose discover category-generic (hence no shortcuts) and visually discriminative concepts directly from pre-trained VLM (hence the text prompts are more friendly to the VLM). We then propose a suite of evaluations in order to draw robust conclusions on the precision and thoroughness of the discovered concepts.
3.3 Visual Concept Discovery and Learning
We propose to discover visual concepts from a large pool of diverse objects, so that the discovered concepts would be shared by multiple objects and generalizable to different domains. Towards this goal, we adopt the Conceptual Captions 3M (Sharma et al. 2018) dataset (CC-3M). CC-3M contains three million images and their captions, hence offering a huge repertoire of objects and their characteristics rendered both images and text. As illustrated in Figure 2 our overall goal to identify the key objects and their associated visual concepts from the image captions, with the help of a large language model as the knowledge source. The candidate visual concepts are then checked for their visual discriminativeness, by verifying if candidate visual concept can be reliably recognized from the corresponding image. Thus, we can select the concepts that can both be recognized by the VLM in the image and be deemed by the LLM as a relevant attribute for the object described in the caption.
As objects captions often hold specific dependencics, such as serving as the subject or object of an action, we employ Dependency Parsing (De Marneffe et al., (2006) along with set of designed rules (see Appendix for details) to retrieve the words and phrases that potentially correspond to objects in the captions (e.g..
5

---

Published in Transactions on Machine Learning Research (01/202))
Selected Concepts
LLM-gennrated Concepts
Muiual Information
VLA
"orange ear patches" "Pogular among kids"
"orange ear patches" "white belly"
"white belly" "social birds"
group of king penguin walking the snow"
Figure 2: Illustration of our proposed concept discovery method. Given image-caption pairs, we first identify objects from the captions and utilize large language model to propose candidate concepts for the objects. The concepts are then ranked by the agreement between VLM knowledge (concept recognition from the image) and LLM knowledge (concepl proposed based the caption) based on mutual information.
Selected Coucepis
YLM
Concept
Conerp--ategoryy
Avociation
Predicted
Soageli
Robin
"yellow loge"
conceptts
yullow logs
III categoriis
"white belly"
"crange threat"
white belly
Grasggo tlreat
Prgiistion
yellow loga
whilk belly
umange
Suroot
X
O
Seaguil
Robin
Joy
Seaqul3
0.66
0.30
0,05
1.46
0.05
0.66
Finooker
citegories
Loss & Rack-Prapapaiin
Figure 3: Illustration of the concept-based object recognition framework and our proposed concept learning method. We map the concept activations to categories with the concept-category association matrix W. For object recognition, only W is optimized based object classification supervision. For concept learning, we assume W is binary matrix given by LLM knowledge, and learns to update a by fine-tuning the last layers of visual and text encoders in the VLM. We rely on the VLM recognized object labels as opposed to ground truth object labels, hence the process self-supervisod.
"king penguin" from a sentence "a group of king penguins walking in the snow"). We then leverage the LLM as an external knowledge base obtain visual concepts for recognizing the objects. We design a set of prompts (c.g., "What are useful visual features for distinguishing an {object} in photo?") to query the LLM and retrieve relevant concepts for the object of interest. We take the union of the concepts discovered for all objects the preliminary list of generic visual concepts.
We then propose to filter the concepts so that the selected ones are also visually discriminative. Intuitively, a visual concept should be ranked higher when the VLM can recognize it from the image if and only if when the same concept. is proposed by the LLM based on the image caption. Otherwise, the concept either is not specific the object of interest (i.e. can be recognized from the image but not proposed to describe any object mentioned in the caption), or likely to correspond to non-visual concepts (e.g. "popular among kids" and "social birds" in Figure 2). Specifically, given concept and a list of image-caption pairs, define two variables X and Y, where X corresponds to the imagc-coneept similarity as measured by VLM, and the Y is a binary indicator on the caption-concept correspondence according to an LLM. A high X value x indicates that the VLM can recognize the concept from the image well, and a higher Y value indicates the concopt is dcemed by the LLM as relevant attribute for the object described in the caption. In practice, we compute a as the cosine similarity between the CLIP embedding of the concept and the image, and define y as boolean indicating whether the concept is relevant to the object in the image, determined by the

---

Published in Transactions on Machine Learning Research (01/2025)
LLM. We adopt the Mutual Information (MI) to determine the agreement between the VLM and the LLM:
Σ Σ (a,)(,,)) v) log P2.0(a. P2(2))
I(c)
(2)
xEY.
which measures the amount information gain of one variable by knowing another variable. A higher (c)) means that X, and Y, are in agreement.
Although we can discover general and visually discriminative concepts from pre-trained VLMs, these concepts are not always perfectly-align with images as shown in Column 2 of Figure I and Row 4 of Table [ We propose self-supervised fine-tuning method to refine the image-concept alignment of VLMs by leveraging the knowledge alrcady encoded in pre-trained VLMs and LLMs. We first perform the zcro-shot classification with pre-trained VLMs to generale pseudo labels for training. We then construct a "ground-truth" concept- category association matrix WLLM with binary weights, where each weight is obtained by querying the LLM with prompts like "Does the {category __m_c]_mmm usually have the attribute (concept)?". As illustrated Figure 3 we project the image and concept embeddings into concept activations using the linear projection. The prediction of image category is obtained by the matrix multiplication, where f(a) = a WLLM- Assuming that the WLLM provides ground-truth concept-category association, the loss derived from the predicted labels is utilized to optimize the image-concept alignment. The whole framework is self-supervised as it requires no additional human annotations.
3.4 Visual Concept Applications
Our concept discovery and learning framework aims to obtain list of generic and visually discriminative concepts. We propose to apply the discovered concepts on fine-grained object recognition benchmarks such as bird classification (Wah et al. 2011) and food classification (Bossard et al.) 2014) to evaluate their quality. For such applications, it. often desirable to obtain a compact and performant set of visual concepts (e.g. "four-wheeleds useful for recognizing cars, but not so much for birds). Towards this goal, we first construct the concept-category association matrix WLLM and perform concept learning only for the object categorics in the larget dalaset, so that irrelevant concepts not used by any objects are automatically discarded.
The list of remaining visual concepts can be further optimized based on their usefulness and generalizability. We first try to identify the visual concepts that can be reliably recognized from the target dataset. Wo re-purpose (c) where Y. is now obtained by looking up the concept association matrix WLLM knowing the ground truth object label for an image. A higher I(c) means the concept useful to recognize a subset of object categories in the dataset (according to the LLM knowledge used to construct WLLM), and can be reliably recognized from the images when the concept is expected to appear.
We also expect the selected concepts to be generalizable, that is, to benefit the recognition of unseen objects. We employ a heuristics where a visual concept is more likely to generalize if it is already used by many known object categories. The "generalizability"
a concept G(c) can hence estimated by the ratio of object categories that contain the concept c over the total number of object categories.
There exists natural trade-off between the usefulness and generalizability of visual concept, we hence compute the weighted average T(c) + (1 (11-00 a) G(c) to rank the concepts and apply fixed budget the number of visual concepts to use for each downstream benchmark. We select based classification performance on the validation set (a lower a, namely more general concepts, is preferred whenever the accuracy remains high).
3.5 Visual Concept Evaluation
Finally, we propose suite of evaluation protocols to measure the quality of VLM-discovered concepts in terms of Interpretability, Precision, Thoroughness and Generalizability.
Interpretability metric from Yun et al. (2033) quantitatively measures how well a VLM learns concept- category associations. The discovered concepts are interpretable they are associated with images in a human-undlesstaddabl manner. For example, we expect the model to recognize an image of "Giant
7

---

Published in Transactions on Machine Learning Research (01/2025)
Panda" according to the concepts "Rounded head", "Black eye patches" and "Black and white fur" instead of irrelevant concepts like "long wings". In this way, the discovered concepts should lead to a concept- category association matrix that is similar to WLLM- Given trained concept-category matrix Werained we can predict the category for an image with its concept activations, that is, f(a) = a Wrrained- Following Koh et al. (2020); Yun el al. (2023), we intervene the classification with ground-truth concept activations g. where gi
the ground-truth category of the image contains the i-th concept according to the LLM knowledge and g: = 0 otherwise. We calculate the accuracy of predictions made by f'g) g Werained the intervention accuracy. High intervention accuracy indicates high interpretability of visual recognition since the concept-category association is consistent with human knowledge.
Precision is measured with human evaluation on how well the discovered concepts provide correct reasoning clues for visual recognition. As illustrated in Section 3.1 the contribution of a certain concept to the predicted calegory j of an image can be measured by the dol product between the concept activation a and Wj from a CBM weight matrix W. We expect the top-weighted concepts to be accurate depictions of the object of interest in the image when the model prediction is correct. For cach correctly-classife annolators to determine whether the top-k weighted concepts actually describe the image and calculate the proportion as the precision score. The selection of the parameter & is contingent upon the scale of the concept space within each dataset. For datasets with large amounts of concepts, we can find more descriptive concepts for a certain category. Hence, we set larger value for k. Conversely, datasets with relatively few concepts have a diminished pool of descriptive elements for individual categories. Consequently, a smaller value for k is deemed appropriate in such scenarios.
image, ask human
Thoroughness measures how well the discovered concepts cover important features to recognize the objects in downstream domains. Given image correctly predicted for its category, first leverage the LLM knowledge to obtain complete list of concepts that are potentially related to the category. We then ask human annotators to select from the complete list of concepts that can be inferred visually from the image of interest. We call these the important visual concepts for an image. We then select the top-k weighted concepts and calculate the thoroughness score as the percentage important visual concepts covered by the top-k weighted concepts.
Generalizability measures whether the discovered and learned concepts can benefit the recognition of unseen objects. We consider in-domain and cross-domain generalization of the discovered concepts. For the in-domain generalization, we first randomly split the calegory list of a given dataset into seen categories and unseen categorios. We select from the discovered concepts separately for the seen and unseen categories (as in Section 3.4/, we then perform concept learning only the examples of seen categories. These concepts are transferred train cancept-based classifiers for the unseen categories, and we report classification accuracy the proxy to measure in-domain generalization. Cross-domain generalization is measured following the same procedure, but for seen dataset and different dataset.
4 Experiments
We conduct quantitative and human evaluations to evaluate the discovered and learned concepts. In Sec- tion 4.2 we demonstrate why we need short-cut free and visually discriminative concept discovery and learning by showing that concept prompts in previous works might be category-biased. In Section 4.3 and Section 4.4 we aim to prove that pre-trained VLMs do learn visual concepts by demonstrating that the discovered concepls can lead to accurate classification and exhibit high qualities including interpretability, precision, thoroughness and generalizability. In Section 4.5 aim to explore how specific components in our proposed framework contribute to the quality of discovered concepts.
4.1 Experimental Setup
Datascts: We conduct experiments on several challenging fine-grained image classification datasots, in- cluding ImageNet (Deng et al.) 2009). Food (Bossard et al. (2014). CIFAR-100 (Krizhevsky et al. 2209). CIFAR-10, CUB (ah et al. 2011) Flowers (Nilsback & Zisserman, 2008) Stanford Cars (Krause et. al.
8

---

Published in Transactions on Machine Learning Research (01/2025)
Prompt Design
ImageNet
Food
CIFAR-100
CIFAR-11
CUB
Flowers
Stanford Cazs
Airerafts
Oxford Pets
Category Name
71.6
91.8
75.9
96.2
63.1
77.4
77.3
36.1
93.5
Name LLM Concepis Menon Vondrick, 2012
75.0
92.4
77.7
96.0
63.5
78.9
77.6
37.4
92.2
Name w/ Randem Concepás
70.1
91.8
75.4
96.0
62.1
78.7
78.5
36.0
92.4
LLM Coneepts
22.1
3.6
30.9
70.7
5.3
3.2
1.6
6.4
Table 1: Zero-shot classification on nine object recognition benchmarks. We observe that although aug- menting calegory names with LLM-generaled concepls improves classification accuracy (e.g. VDES (Menon) & Vondrick 2022)), the method still mainly relies on category names in the text prompts, and it remains unclear if the concepts included in the text prompts are properly recognized by pre-trained VLMs. When paired with randomly shuffled concept in the prompts, the accuracy drops only moderately; when the cate- gory names are removed, the accuracy drops significantly.
2015), Aircralts (Maj et al. 2013) and Oxford Pets Parkhi et al., 2012). The statistics and split of the datasets are shown in the appendix.
Baselines: We compare with LaBo (Yang et al. 2023b and LM4CV Yan al. 2023a which are the state-of-the-art works on concept-based visual recognition. Following the sctting in LM4CV, we control the bottleneck sizes (mumber of concepts) to be the same for the baselines and our model for fair comparison. Besides full-shot classification, we also compare with LaBo on few-shot settings, where we select concepts and train the model on limited data. is infeasible for LM4CV to conduct few-shot classification since it requires the whole training set on concept generation.
Implementation Details: We use the same LLM 1er---textt-davnni--ee to obtain descriptors as pre- vious works. We also use the same CLIP backbone (ViT-L-14) to compare with baseline models. Following (Yun et al. 2023), we use logistic regression to train the concept bottleneck models. We observe that the performance of CBM is robust to the choice of hyperparameters and use the default Scikit-learn hyperpa- rameters for all datasets. For concept learning, we use the AdamW optimizer with 5e-4 learning rate and lc-4 weight decay to fine-tune the CLIP model, and we use the validation loss select checkpoints. For human evaluation experiments, we hire annotators from Amazon Mechanical Turk. For each example, we ask three annotators to annotate and the majority vote to obtain the result. We conduct Students' T-test (Student 1908) to evaluate the statistical significance of the human evaluation results. The detailed design of human evaluation and the statistical significance results are shown in the appendix.
4.2 Lancept-Augmented Prompts Are Eategory-Biaseed
As discussed in Section 3.2 we conduct ablation experiments to understand if the concepts used in the prompts of previous zero-shot classification methods lead to improved classification accuracy. The results in Table are consistent with Figure [ both of which show that the concept-augmented text prompts do not offer conclusive evidence whether pre-trained VLMs learn to encode concepts. Even random concepts have minimal impact on zcro-shot performance (as shown in Row and Row 3). In contrast, the removal of calegory names results in catastrophic decline in zero-shot performance (as shown in Row 2 and Row 4). These observations suggest that category names act critical shortcuts in concept-augmenteed prompts.
Besides category biasos, the discovered concepts in previous works contain many non-visual concepts as illustrated in Figure [ We conduct human evaluation to compare our discovered concepts with previous works by the proportion of non-visual concepts and concepts containing class names. For both baselines and our CDL, we randomly select 100 concepts for each dataset for human evaluation. The results are shown in Table D. We can observe that CDL offers visual concepts that are more category-agnostic and visually discriminative.
4.3 Classification Performance with Visual Concepts
Following the settings in LaBo and LM4CV, we set the bottleneck size be the same or times of number of classes in given dataset. The same VIT-I-114 CLIP model is used across all methods. To ensure
9

---

Published in Transactions on Machine Learning Research (01/2025)
/CCaeegoy-agnnoticc
%Visual
LaBo
65.50
66.83
ML4CV
82.33
73.83
CDL
92.33
85.50
Table 2: Human evaluation results on the proportion of eeveegory-agnosiie and visually-discimmiattve con- cepts. The results are averaged over all datasets. The results show that our concepts discovered from general corpora are less biased by category names and more visually discriminative.
ImaggeNet
Food
CIFAR-100
CIPAR-10
CUB
Flowers
Stanford Cars
Airerafts
Oxford Pets
#Concepls
1000
2000
202
100
200
10
20
200
400
102
204
196
392
102
204
37
74
LaBo
83.2
83.6
89.8
91.1
80.5
81.1
77.8
92.2
79.7
81.3
95.5
94.9
81.7
85.6
59.3
60.7
88.3
90.7
IM4CV
83.4
83.6
90.0
91.1
81.7
84.2
80.4
92.3
77.5
80.4
94.5
93.8
84.9
85.7
59.9
61.0
88.5
90.4
CDL
83.6
84.0
94.2
94.9
83.8
85.8
82.1
93.5
83.2
83.4
96.3
95.7
86.1
86.7
60.8
62.4
90.6
92.3
Table 3:
Comparison with baselines on classification accuracy with different bottleneck sizes. All methods
are based on ViT-L-14 CLIP checkpoint.
ImapeNet
CIFAR-100
Lafe:
lafo Dur
Laflo
CIFAR-10
CUB
Flowers
LaBc
Lado Dups
Lado
2000
Stanford
Aircrafts
Oxford
Late Ours
Lado
Lade
Durs
Figure 4: Fow-shot classification evaluation with LaBo and our method.
fair comparison, we conduct the classification using our concept discovery and selection methods, without concopt learning. Table g shows that our method consistently outperforms the bascline methods on all datasets.
We then adopt the few-shot learning setting, where we select the concepts (as described in 3.4 and train the model with the few training examples. Figure shows that CDL consistently outperforms LaBo, especially when the number of training examples is smaller.
4.4 Evaluation of the Discovered Concepts
Figure 5 shows the evaluation results of the Interpretability, Precision, and Thoroughness the baselines and our method. For human evaluation of Precision, and Thoroughness, we randomly select 100 images from each dataset and ask human workers annotate the Precision and Thoroughness of top-weighted concepts for each image described in 3.5 More details are shown in Appendix Sec. D The Precision
10

---

Published in Transactions on Machine Learning Research (01/202))
ImageNet
Food
CIFAR 100
LEPACY
Ineepesailiy
Thinightes
Insyroatiiity
Thooggoss
Ernegeesbillt
Thooogganii
CIFAR
CUB
Flowers
Iriepesshitty
Prooeint
Thoragghooo
Inseynsshiiy
Froteon
Thoorighno
Inkegreseribt
Prizioen
Threoggheoo
Soed Clus
Oxtond Pes
Aircrads
Dhoooggtoos
Inorgnnshiiitt
Througgnss
Thonaggrass
Figure 5: Human evaluation of the discovered concepts for their interpretability, precision, and thoroughness. We evaluate the concepts selected without and with concept lcarning (denoted as "CD" and "CDL").
Domain
Method
AClassification
Alnterpretability
A Prrcision
AThoroughness
LaBo
0.2
16.1
5.8
7.6
In-domain
LM4CV
0.4
14.2
4.5
6.4
CDL
1.0
18.3
7.9
10.4
LaBo
-1.5
0.1
-3.7
-4.5
Cross-domain
LM4CV
-0.3
-0.6
-6.4
-8.2
CDL
0.2
7.0
10.3
13.9
Table 4: Generalization evaluation for discovered concepts. The In-domain results refer to improvements on unseen calegories in the same dataset, while the Cross-domain results refer to improvements on unseen domains. A denotes the improvement of fine-tuned CLIP compared to the original CLIP. Higher improvement mcans better generalization.
and Thoroughness of each dataset are calculated the averages of these evaluations on the 100 images, with the results displayed the Y-axis of Figure 5 We observe that despite having high classification performance, the baseline models discover concepts that exhibit unsatisfactory interpretability, precision, and thoroughness. Our concept discovery method provides significant improvements on all three metrics, and the self-supervised concept learning can further improve the concept quality.
Table 4 shows the in-domain and cross-domain generalization results. Our proposed CDL outperforms both baselines for generalizability, especially for the cross-domain scenario when transferring from ImageNet CUB. We observe that LaBo and LM4CV struggle with cross-domain generalization as they select completely different concepts for different datasets and few common patterns can be learned with their methods.
4.5 Ablation Study
this subsection, we perform ablation studies to analyze the performance of different sub-modules in the proposed CDL framework.
Multi-modal Visual Concept Discovery To evaluate the effectiveness of the proposed concept discovery method, build baseline which only uses the LLM to generate concepts and perform random concept
11

---

Published in Transactions on Machine Learning Research (01/2025)
Accuracy
Interpretability
Precision
Thoroughness
LLM-only 68.5
3.6
24.2
27.0
Multi-modal
83.4
44.6
67.3
71.0
Table 5: Comparison of our proposed multi-modal concept discovery method with the LLM-only concept discovery method on CUB dataset. Our method siguificantly outperforms LLM-only method on both clas- sification accuracy and concept quality.
Domain
Source
A AClassffication
Alnterpretability
APrecision
AThoroughness
CUB
0.2
12.5
2.8
3.7
In-domain
CC-3M
1.0
18.3
7.9
10.4
ImageNet
-1.1
-0.2
-5.5
-4.6
Cross-domain
CC-3M
0.2
7.0
10.3
13.9
Table 6: Comparison of the concept discovered from CC-3M and other datasets on both in-domain and cross-domain generalization evaluation for the CUB task. The concepts discovered from CC-3M have better generalizability in both cases.
selection for downstream tasks. As shown in Table 5 the concepts discovered by the LLM-only method are of unsatisfactory classification accuracy, interpretability, precision and thoroughness, which demonstrates the necessity of utilizing multi-modal information to select visually discriminative concepts. The results show that our proposed multi-modal concept discovery method can effectively improve the classification performance and the interpretability, precision and thoroughness of the discovered concepts.
Utilization of the CC-3M dataset To analyze the effectiveness of the utilization of the CC-3M dataset during concept discovery, we perform the proposed concept discovery method on the CC-3M (Sharma et al. 2018) dataset and the downstream datasets. The performances of both methods are similar on CUB (details are shown in Appendix), which indicates that our proposed concept discovery method can still provide signif- icant improvement on the classification and concept quality with only downstream datasets. Considering the cost of human evaluation for the Precision and Thoroughness, we select the CUB dataset the ream benchmark for the generalization evaluation. The in-domain and cross-domain generalization results in Table i indicate that the concepts discovered from CC-3M are more generalizable than the concepts discovered from downstream datasets. In conclusion, the proposed concept discovery and learning method provides improvement on classification performance and concept quality regardless of the utilization of datasets, and discovering concepts from the CC-3M datasets can enhance the generalization of concepts.
5 Conclusion and Future Work
In this paper, we investigate the question of whether pre-trained Vision-Language Models (VLMs) can encode primitive visual concepts. We first illustrate that category- biased concepts extracted from specific datasets do not offer conclusive evidence of the concept learning capacity VLMss. To resolve this issue, we design novel framework to discover category-agnostic and visually discriminative concepts from large image- caption dataset with the help of VLMs and LLMs. To make use of the discovered concepts for downstream tasks, propose novel method to build concept bottlenecks with compact and performant set of concepts for specific domain. We also propose a self-supervised concept lcarning framcwork to re-align the concept knowledge in VLMs for the category classification in specific domains. To prove that VLMs do learn useful and interpretable concepts, we propose a suite of comprehensive protocols to evaluate the quality of the discovered concepts and perform exhaustive experiments including human evaluation. The experimental results demonstrate that VLMs do capture primitive concepts that can lead to effective, interpretable, and generalizable visual recognition.
12

---

Published in Transactions on Machine Learning Research (01/2025)
Giant Panda
LaBo
LMACV
Panda Bear
large, samsl
black and white fur
silver case with black strop
Bong. Black white
black eye patches
keeps the pande NarE
rounded ear
Bany 2008 have pandas
rounded head
Ped Thai
national dish of thailand
onions, peppers, and herbs
fried or bolled rice neodler
with noodles, tofu, and vegetables
the coup is usually served 62th race or noodles
pontains vegetables, proteii., and a sauce
atir-fried quickly
citrus fruits such as lenon or line
vegetables (beans, sprouts, strr))s.ato.)
Black-footed Albatress
#ost abundant albatross spectss
black and white plunage
black water bard
one of the largest albatrosses
gray body with darker head and sangs:
long, hooked bill
black sings
yellos bill with black tip
webbed Feet
only albatross species that has completely black legs and feet
shite head and nack, often streaked 6ith brown or black
brown plusage
Figure 6: Examples of the top weighted concepts correlated with the given images.
While we illustrate that VLMs do learn discoverable concepts, still significant to understand what kind of concept and compositionality knowledge cannot. be learned in the contrastive learning- based pre-training of VLMs. In future work, plan to explore whether VLMs capture the semantic and spatial relationships between concepts and utilize these relationships to perform complex multi-modal reasoning.
6 Acknowledgement
This work is in part supported by a gift from Adobe Research, a seed grant from NASA, and a Richard B. Salomon award for Chen Sun. We thank helpful discussions with Professors Ellie Pavlick and Stephen Bach. We thank Yue Yang, An Yang, Yu Wang for providing the open-sourced code for the basclines.
References
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in Neural Information Procossing Systems. 35:23716-23736, 2022.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile dision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.1200 3, 2023.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network disscction: Quantifying interpretability of deep visual representations. Proceedings of the TEEE conference on computer vision and pattern recognition, pp. 6541 6549, 2017.
Lukas Bossard, Matthieu Guillaumin, and Lue Van Gool. Food-101 mining discriminative components with random forests. In Computer Vision-ECCV 2014 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Procoedings, Part VI 13, pp. 446-461 Springer, 2014.
Kushal Chauhan, Rishabh Tiwari, Jan Freyberg, Pradcep Shenoy, and Krishnamurthy Dvijotham. Inter- aclive concept bottleneck models. In Proceedings of the AAAT Conference on Artificial Intelligence, pp. 5948 5955, 2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
13

---

Published in Transactions on Machine Learning Research (01/2025)
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Good- man, Adam Gryoner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. In The Eleventh International Conference on Learning Representations, 2023.
Marie-Catherine De Marneffe, Bill MacCartney, Christopher Manning, et al. Generating typed dependency parses from phrase structure parses. In Lrec, volume 6, pp. 449 454, 2006.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largo-scale hicrarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248 255. IEEE, 2009.
Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing objects by their attributes. In 2009 IEEE conference on computer vision and pattern recognition, pp. 1778 1785. IEEE, 2009.
Evan Hernandez, Sarah Schwettmann, David Bau, Toona Bagashvili, Antonio Torralba, and Jacob An- dreas. Natural language descriptions of deep visual features. International Conference on Learning Representations, 2021.
Cheng- Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. Sugarcrepe: Fixing hackable benchmarks for dision-languag compositionality. Advances in Neural Information Processing Systems, 36, 2024.
Lijie Hu, Chenyang Ren, Zhengyu Hu, Cheng-Long Wang, and Di Wang. Editable concept bottleneck models. arXiv preprint 2xXiv22405115472 2024.
Qiyang Hu, Attila Szabó, Tiziano Portenier, Paolo Favaro, and Matthias Zwicker. Disentangling factors of variation by mixing them. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pp. 3399 3407, 2018
Chao Jia, Yinfci Yang, Ye Xia, Yi-Ting Chen, Zarana Parckh, Hieu Pham, Quoe Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pp. 4904-4916. PMLR, 2021.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, el al. Inter- pretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference machine learning, pp. 2668-2677. PMLR, 2018.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pp. 5348. PMLR, 2020.
Jonathan Krause, Hailin Jin, Jianchao Yang, and Li Fei-Fei. Fine-grained recognition without part annota- tions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5546 5555, 2015.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Brenden Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua Tenenbaum. One shot learning of simple visual concepts. In Proceedings of the annual meeting of the cognitive science society, volume 33, 2011.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaur. Human-level concept learning through probabilistic program induction. Science, 350(6666):1332-1333 13388 2015.
Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In 2009 IEEE conference on computer vision and pattern recognition, pp. 951 958. IEEE, 2009.
Sharon Lee, Yunzhi Zhang, Shangzhe Wu, and Jiajun Wu. Language-informed visual concept learning. In The Twelfth International Conference on Learning Representations, 2023.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Martha Lewis, Qinan Yu, Jack Merullo, and Ellic Pavlick. Does clip bind concepts? probing compositionality in large image models. arXiv preprint arXiv:2212. 10537. 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, ICML, JMLR.org, 2023.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 36, 2024.
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task- visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32, 2019.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthow Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013.
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In International Conference on Learning Representations, 2019.
Sachit Menon and Carl Vondrick. Visual classification via description from large language models. In The Eleventh International Conference on Learning Representations, 2022.
Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Doso- vitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al. Simple open- vocabulary object detection. In European Conference on Computer Vision, pp.
Springer, 2022.
Tushar Nagarajan and Kristen Grauman. Attributes operators: factorizing unseen attribute-object com- positions. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 169-185 2018.
Maria-Elena Nilsback and Andrew Zisserman. Aulomated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pp. 722-729. IEEE, 2008.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pp. 3498 3505. IEEE, 2012.
Sarah Pratt, Ian Covert, Rosanne Liu, and Ali Farhadi. What does a platypus look like? generating customized prompts for zero-shot image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15691-15701, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Piyush Sharma, Nan Ding, Scbastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hyper- nymed, image alt-texi dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2556-2565, 2018.
Adam Stein, Aadilya Naik, Yinjun Wu, Mayur Naik, and Eric Wong. Towards compositionality in concept learning. arXiv preprint 4rXiv:2406.18534 2024.
Student. The probable error of mean. Biometrika, pp. 1 1-25, 1908.
15

---

Published in Transactions on Machine Learning Research (01/2025)
Chen Sun, Chuang Gan, and Ram Nevatia. Automatic concept discovery from parallel text and visual corpora. In Proceedings of the IEEE international conference on computer vision, pp. 2604, 2015.
Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Can dace Ross. Winoground: Probing vision and language models for visio-linguistic compositionality. Tn Procoodings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5238-5248, 2022.
Catherine Wah, Steve Branson, Poter Welinder, Pictro Perona, and Serge Belongie. The caltech-uesd birds- 200-2011 dataset. 2011.
Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. Cogylm: Visual expert for pretrained language models. arXiv preprint arXiv:231.00009,, 2023.
Yutaro Yamada, Yingtian Tang, and llker Yildirim. When are lemons purple? the concept association bias of clip. arXiv preprint MrXiv:2212.12443 2022.
An Yan, Yu Wang, Yiwn Zhong, Chengyu Dong, Zexue He, Yujie Lu, William Yang Wang, Jingbo Shang, and Julian McAuley. Learning concise and descriplive attribules for visual recognition. In 2023 IEEE/CVE International Conference on Computer Vision (ICCV), pp. 3077. IEEE Computer Society, 2023a.
An Yan, Yu Wang, Yiwu Zhong, Zexue He, Petros Karypis, Zihan Wang, Chengyu Dong, Amilcare Gentili, Chun-Nan Hsu, Jingbo Shang, al. Robust and interpretable medical image classifiers via concept bottleneck models. arXiv preprint arXiv:2310.0311 2023b.
Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, and Cordelia Schmid. Vid2seq: Largo-scale pretraining of visual language model for dense video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10714 10726, 2023a.
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 19187-19197)
2023b.
Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. Filip: Fine-grained interactive languago-image pro-training. arXiv preprint arXiv:2111.07783, 2021.
Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmect Kohli, and Josh Tonenbaum. Neural- symbolic vqa: Disentangling reasoning from vision and language understanding. Advances in neural information processing systems, 31, 2018.
Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-langnage models behave like ofgs-of-words; and what to do about it? In The Eleventh International Conference on Learning Representations, 2022a.
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205.15448 2022b.
Tian Yun, Usha Bhalla, Ellie Pavlick, and Chen Sun. Do vision-language pretrained models learn composable primitive concepts? Transactions on Machine Learning Research, 2023.
Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. Vinvl: Revisiting visual representations in vision-language models. In Proceedings of the TEEE/CVF conference on computer vision and pattern recognition, pp. 5579 5588, 2021.
16

---

Published in Transactions on Machine Learning Research (01/2025)
A Hyperparameters
We adjust hyperparameterss according to the performance of the models on the validation dataset (For ImageNet, we randomly select 10% of the training set as the validation set). For a in Equation 2, we set it to 0.7 for ImageNet dataset, 0.8 for Food-101, CIFAR-100 100, CUB-200 and Flowers-102 datasets and 0.9 Section 3.4, a smaller a will increase the generalizability
for CIFAR-10 dataset. According to Equation but decrease the discriminativeness of the selected concepts, and thus decrease classification performance. To achieve a trade-off between generalizability of the selected concepts and classification performance, we monitor the classification performance when gradually decreasing a. We pick the right before classification performance drops significantly.
B Additional Implementation Details
B.1 Dependency Parsing Rules
Given a caption, we perform Dependency Parsing to extract the grammatical relations from the caption. We extract the nouns or phrases from the following dependencies as potential objects in an image caption:
nsubj / nsubjpass: The nsubj (nominal subject) is the noun that performs the main action. For example, in the sentence "the horse is cating grass", "horse" is the nominal subject. The nsubjpass (passive nominal subject) is the noun that performs the main action in sentence of passive voice. For example, in the sentence "The dog is led by leash", "dog" is the passive nominal subject. We extract the nominal and passive nominal subject as a word potentially corresponding to an object in the image.
dobj / iobj: The dobj (direct object) is the noun noun phrase that receives the action of the verb. For example, in the sentence "the horse is eating grass", "grass" is the direct object. The iobj (indirect object) is the noun or noun phrase that indicates to or for whom the action of the verb is done. For example, in the sentence "The man gives the girl a flower", "girl" is the indirect object. We extract the direct and indirect object as word potentially corresponding an object in the image.
amod: An amod (adjectival modifier) an adjective that describes a noun (e.g. "black dog", "white bird"). We extract the amod and its object as a phrase potentially corresponding to an object in the image.
The compound label indicates the word is part of a compound phrase like "king penguin". Once select a word following above rules, we check whether it is part of a compound phrase. If so, we extract. the whole phrase as the object.
compound:
B.2 Mutual Information Implementation
The Mutual Information (MT) measures the information gain of one variable by knowing another variable. Given two variable X and Y, it measures the KL divergence between the joint distribution P(X,Y) and the product of the marginal distribution P(X))(Y):
(I, Pz.y (x, y) log P,(x))
MT Σ
Σ DEY #EX
The MI is high when X and Y are positively or nogatively related (e.g. high æ value indicates high y value and low y value indicates low value or vice versa). Given a concept and an image-caption dataset, the X variable corresponds to the image-concept similarity as measured by a VLM, and the Y variable is a binary indicator on the caption-concept corrospondence according to an LLM. Namely, a higher x indicates that VLM can recognize the concept from the image well, and a higher y indicates the concept is deemed by LLM relevant "attribute" for the object described in the caption. As such, we prefer concepts with higher MI
17

---

Published in Transactions on Machine Learning Research (01/2025)
since it indicates that if concept can be recognized from the image, the same concept should be relevant for the paired caption, and vice versa. This helps filter out concepts that are non-visual (relevant according to LLM, but cannot be recognized visually by VLM, such as "fish-eating" and "endangered animal"), irrelevant to the objects interest (can be recognized by VLM, but irrelevant according to LLM, such as concept "yellow-leg" for recognizing food). Figure A1 shows some examples for the MI calculation.
Since the image-concept similarity is continuous, we utilize K-nearest-neighhors- based entropy estimation toolkit in Scikit-learn perform discretization and calculate the MI.
C Dataset Details
The table shows the statistical details of the datasets we choose.
Dataset
# Class
# #Taainn
# Valid
# Test
ImageNet
1000
128,1167
50,000
Food-101
101
60,600
15,150
25,250
CIFAR-100
100
40,000
10,000
10,000
CTFAR-10
10
40,000
10,000
10,000
CUB-200
200
4,794
1,200
5,794
Flowers- 102
102
4,093
1,633
2,463
Stanford Cars
196
6,515
1,628
8041
Airerafts
102
3,400
3,400
3,400
Oxford Pets
37
1,846
1,834
3669
Table A1: Statistical details of datasets. "Caas"" means the number of classifications. "#Train", Vaald""
and "Tesst" denote the instance numbers of each dataset respectively. For ImageNet, we randomly select 10% of the training set as the validation set and regard the validation set as the test set.
D Human Evaluation Details
We hire workers on lutpp::/www.mtuut http://wwwwwwk.ooo/m to conduct human evaluation. To make our human evaluation more robust, for one data point we ask three human workers to annotate. For the precision and thoroughness metric, we randomly select 100 images on each dataset to evaluate. For CIFAR-10, CIFAR-100, Food-101, Flowers-102 we choose the top concepts to annotate. For ImageNet and CUB-200 we choose the top concepts to annotate, We annotate 2,200 data points for precision and around 3,000 data points for thoroughness. For the visual discriminability and category name containing, we utilize different methods to select 100 concepts for each dataset to evaluate and report the average score. Hence annotate 1,800 data points for those two task. In the annotation, we randomly shuflle the order of instances to remove possible biases.
In order to validate the effectiveness of our human evaluation, we calculate the pairwise annotator agroement score following LaBo. The average pairwise annotator agreement proportion on all datasets 69.4%, which is comparable the 69.8% proportion in LaBo.
We show examples of human evaluation interface in Figure A2 The examples shown for measuring precision of the concepts, and similar interface is used to measure thoroughness, where the humans are asked to build a complete concept list for an image.
E Statistical Significance
We conduct Students' T-test to evaluate the statistical significance of our main experiments including classifi- cation accuracy and human evaluation of concept quality. We set the threshold of p-value to be 0.05 following
18

---

Published in Transactions on Machine Learning Research (01/2025)
White Belly
bimb 3.52
Imaege? (1LM))
MI Score: 0.76
Fish-eating
Simvlarity (VLM)
Imaege? (LLM)
MI Score: 0.07
Endangered Animal
Sirvlartty (VLM)
3:45
Imagel (LLM)
MI Score: 0.02
Figure Al: Illustrations of Mutual Information calculation. We can observe that visually discriminative concepts such as "white belly" have high MI score because they can be reliably recognized from images that supposed to contain the concept according to I.M,, and vice versa. The non-visual concepts like "fish-eating" and "endangered animal" have low MI scores because VLMs cannot recognize these concepts from the images that are supposed to contain these concepts according to LLM.
previous works. When p-value is lower than 0.05, the mull hypothesis is rejected and method performs significantly better than the baseline method. From the results in Table A2 we observe that both
19

---

Published in Transactions on Machine Learning Research (01/2025)
Intrutions
Mertca
Does the bediy regee comtes necotrrettt
"Grge, weeti heudet
Instractione
Shortous
Dons the
whit....
Dree the otyert tn image atsibunee 'undwiih compose bun
slaectia imge
Yes
No
Intraction
Startous
Doe' the eipect the condiaali cage the atrtutt
"aage Shory
Buck tare
aitrbuue bare,
with long, sarved paralt??
Yes
No
Subreit
Figure A2: Examples of the annotation interface.
proposed method significantly outperform the bascline methods regarding the classification performance and concept quality.
20

---

Published in Transactions on Machine Learning Research (01/2025)
Precision
Thoronghness
Category Agnostic
Visual
Dataset
Method
p-value
significance
p-value
significance
p-value
significance
p-value
significance
CDL v.s.
LaBo
4.59e-65
2.67e-62
3.94e-11
v
2.00e-3
TetgeNet
CDL v.s.
LM4CV
2.19e-68
7.02e-66
3.87e-01
x
4.54e-2
CDL v.s.
LaBo
2G
8.73e-25
V
1.32e-05
V
1.30e-4
v
Food-101
CDL V.s.
LM4CV
1.59e-21
4.97e-20
V
9.31e-05
V
2.78e-2
v.s. LaBo
2.85c-10
3.44o-40
8,59e-10
7.26e-3
/
CIFAR-100
v.s.
LM4CV
1.27e-51
4.99e-51
1.56e-02
1.78e-2
CDL v.s. LaBo
4.28c-9
9.58c-8
<
5.35c-06
2.36c-3
CIPAR-10
CDL v.s. LM4CV
5.28c-12
1.30e-10
7.90c-01
x
2.27c-2
CDL v-s. LaBo
1.31e-50
4.54e-52
3.57e-04
V
6.18es4
CUB-200
<
<
CDL v.s. LM4CV
5.12e442
2.95e-42
5.09e-03
1.40c-1
CDL v.s. LaBo
7.44e445
3.10e-42
1.68e-09
8.44e-3
V
Flowers- 102
CDL v.s. LM4CV
1.21c400
4.58c-37
3,17c-02
4.00c-2 x
The statistical significance of the human evaluation results. Smaller p-value indicates higher sig- nificance. The results show that out human evaluation results statistically significant and our discovered concepts are consistently better than concepts in previous works on all datasets.
Table A2:
ImageNet
Food
CIFAR-100
CIFAR-10
CUB
Flowers
Stanford Cars
Airemafts
Oxford Pets
CDL v.s. LaBo
9e-02 (X)
2--74 (V)
1e-09 (V)
3e-14 (V)
le-06 (v)
3e-03 (V)
1e-02 (V)
2e-01 (X)
1e-02 (V)
CDL v.s. LM4CV
4c-01 (X)
1o-68 (V)
9c-05 (V)
2e-03 (V)
1c-14
2o-01 (X)
3e-02
4o-01 (X)
3c-02 (V)
Table A3: The statistical significance of the classification accuracy. Smaller p-value indicates higher signifi- cance. The performance gain is significant when p-value is lower than 5e-02.
ImageNet
Food-101
CIFAR-100
CIFAR-10
CUB-200
Flowers- 102
# #Concepts
1000
2000
101
202
100
200
10
20
200
400
102
204
W/o Concept Learning
83.6
84.0
94.2
94.9
83.8
85.8
82.1
93.5
83.2
83.4
96.3
95.7
W Concept Learning
83.8
83.9
94.4
94.9
83.6
85.3
96.1
96.5
82.5
82.1
96.6
96.2
Table A4: Comparison of classification performance with our discovered concepts before and after concept learning. The results show that is the concept discovery that mainly contribute to the improvement of classification performance.
ImageNet
Food-101
CIFAR-100
CIFAR-10
CUB-200
Flowers- 102
W/o Concept Learning W Concept Learning
21.9
55.7
62.8
60.0
25.5
27.6
32.3
73.4
78.2
80.0
44.6
48.5
Table A5: Comparison of interpretability of the discovered concepts before and after concept learning. The results the intervention accuracy show that both concept discovery and learning parts provide significant improvement for the interpretability of the concepts.
F Ablation Study on Effectiveness of CDL Framework
In this section, conduct ablation studies the effect of different phases of our CDL framework. We compare the classification performance and interpretability of the concept-based inage-recognition before and after concept learning to illustrate the effects of concept discovery and concept learning.
The results Table A4 A5 and A6 indicate that (1) it is the concept discovery that mainly contribute to the improvement of classification performance and the concept learning would not affect the classification accu- racy; (2) both concept discovery and learning parts provide significant improvement for the interpretability of the concepts according to the intervention accuracy resutls.
21

---

Published in Transactions on Machine Learning Research (01/2025)
Interpretability
Precision
Thoroughness
Wo Concept Learning
21.9
44.0
49.6
W Concept Learning
32.3
65.7
71.9
Table A6: Human evaluation of the discovered concepts before and after concept learning on ImageNet.
In-domain Generalization
Cross-domain Generalization
LaB.
LaBc
Avcuracy
Figure A3: Few-shot classification performance of in-domain and cross-domain generalization, where denotes the improvement of fine-tuned CLIP compared to the original CLIP. The comparison between our concepts and LaBo concepts suggest that our concepts can provide better generalization and lead to better performance for few-shot recognition on different categories and domains.
Method
AP
AP50
OWT
43.5
64.7
OWT + CDT.
46.2
68.
Table A7: Object detection performance on MS-COCO dataset. The baseline results are quoted from Minderer et al. (2022). Visual concept discovery and learning can enhance the object detection performance of the existing framework.
G Few-shot Performance about Generalization
To further measure the in-domain and cross-domain generalization of the discovered concepts, perform few-shot classification with the fine-tuned CLIP model on the unseen categories. The results in Figure A3 show that our discovered concepts can enable much better few-shot classification performance, which suggests that the encoded knowledge of our discovered concepts are more gencralizable to unscon categorics and domains.
H Broader Application of Visual Concepts
Besides image classification tasks, the discovered and learned visual concepts can also benefit tasks that require object localization by localizing visual concepts images. We integrate the visual concepts into the existing object detection framework OWL proposed by Minderer al. (2022) and conduct evaluations on the MS-COCO dataset (Lin et al.) 2014). We first perform concept discovery and learning the training data, and then replace the category names (e.g. "dog") with selected visual concepts (e.g. "four-legged mammal" and 'floppy cars") ) to train and evaluate the object detection model. The results are shown in Table A7 The visual concepts can enhance the object detection performance of the existing framework.
22


Reviews:
[{"review_id": "A3ud6PlGwp", "reviewer": "Reviewer_4BJ9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. Using mutual information to uncover category-independent visual concepts is an interesting way to understand VLMs' internal representations.\n2. Evaluation is done quite extensively on 9 diverse datasets. It's also good to see that human-evaluated metrics like precision and thoroughness are used - this improves the transparency in the VLMs' decision-making process(es).\n3. The \"self-supervised concept learning framework\" is also interesting which enables CDL's scalability by aligning concepts to categories without additional annotations.\n4. Performance-wise, CDL outperforms LaBo and LM4CV, which are SOTA in terms of accuracy and interoperability.\n\n## Weaknesses\n1. This point has also been mentioned in the Future Work section. I feel that this work focuses too much on individual concepts which has been explored significantly by prior work. What would have been more interesting is to see how VLMs capture spatial relationships. For example, techniques from scene graph generation, which model objects and their relationships, could be integrated into CDL.\n2. Experiments mostly focused on the task of classification. But I wonder if concept learning can also benefit image segmentation where we are interested in localizing such concepts in an image?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work introduces a Concept Discovery and Learning (CDL) framework, aiming to determine whether pre-trained vision-language models (VLMs) inherently capture visual concepts such as color, texture, and shape. By leveraging mutual information between visual and textual modalities, CDL identifies category-agnostic, visually discriminative concepts. The framework is demonstrated on 9 different datasets, showing its potential for interpretable and generalizable visual recognition (without the need for supervised fine-tuning).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "As mentioned in the weaknesses above, the experiments have mostly focused on the classification task, but concept learning could also potentially benefit image segmentation performance. Additional experiments exploring this could benefit the scope of the paper.", "broader_impact_concerns": "This paper proposes a general methodology, so I don't think it needs a broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eRKIbcCQAI", "reviewer": "Reviewer_yQ8g", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This work offers a valuable framework for automated concept discovery, with strong potential for application in downstream object detection tasks. The results are impressive; however, I believe the paper would benefit from a clearer presentation of the motivation and a more structured approach to experiments and results. My major and minor comments are detailed below.\n\nMajor Comments \n\nFraming and introduction \n\n- The motivation and proposed approach in this work is strong, but the contributions could be made more explicitly stated earlier in the paper. In its current form, it was not clear to me that the paper has three focuses: (1) exploring if pretrained vision-language models automatically learn visual concepts, (2) proposing a framework for extracting such visual concepts, and (3) training concept bottleneck models on the extracted concepts. Instead, the motivation (and title) highlight the idea of concept discovery, and minimize the contribution of learning. Thus, as I read through the manuscript, I was confused about the motivation behind the learning aspect of the framework and whether or not it was a contribution. Furthermore, this made it difficult to understand the motivation behind some of the experiments.   \n- The introduction includes method-specific details in paragraphs 4 and 5 that are difficult to follow without background and problem setup. It could be strengthened by staying high-level, and leaving specific details about the method to the methods section. \n- The setup to Figure 1 can be further clarified. For example, it wasn’t clear if the experiment conducted image or text retrieval. Additionally, how are the shown concept prompts chosen (i.e., are they the top-k prompts, randomly selected, etc)? \n\nMethods \n- The methods section is challenging to follow due to the lack of a clear, cohesive description of the proposed approach. While aspects of the method are introduced briefly in the introduction and then described in various parts of the methods section, it’s difficult to grasp the exact steps of the entire process. Including an overview of the method that outlines each component in a concise manner would make the methods section much more accessible and easier to follow.\n- Section 3.1 discusses the interpretation of weight matrix $W$. Typically, interpreting such weights as importance values relies on the assumption that the data for which W is trained on are standardized. Are the activation vectors used to train weight matrix W standardized? If not, one would need to calculate standardized regression weights [1] to interpret concept importance. Does the manuscript do this? If not, how does doing this affect the results? This is particularly important for evaluation around precision. \n- In section 3.3, there are conflicting statements between the motivation of the paper and assumptions made about VLMs. The paper assumes “the vision-language pre-training learns powerful encoders for recognizing visual concepts.”, whereas the motivation behind the paper is understanding if pre-trained visual language models learn discoverable visual concepts. I think the assumptions about VLMs make sense, but this should be reflected appropriately in the motivation of the paper. \n- In section 3.3, It was difficult to understand the reason for “realigning” the image-text interface, when the assumption is that the VLMs learn powerful encoders for recognizing visual concepts. Furthermore, I struggled to follow the methodology in this section. For example, why does one construct a concept-category association matrix $W_{llm}$ rather than learning the weight matrix $W$? Second, how is this related to the linear projection layers of the VLM (i.e., the architecture was not clear). \n- In section 3.5, intervention accuracy is used to indicate high interpretability of visual recognition. However, it is unclear what the definition of intervention accuracy is and how it is calculated. \n- In section 3.5, the paper describes precision as being measured with respect to each correctly classified image, where the concepts are ranked according to the dot product between concept activation vectors and the weight matrix. However, in section 3.1, the paper describes understanding the importance of each concept in terms of only the weight matrix $W$. Why choose the former (an instance-based explanation) for the evaluation rather than the later (a global explanation)? \n\nExperimental setup \n- For which concept bottleneck models was the concept association matrix learned as $W$ versus constructed as $W_{llm}$? \n- Section 3.4 discusses choosing visual concepts for downstream benchmarks, but the benchmarks at this point had not been introduced. Thus, it was confusing to follow the experimental setup without knowing what the benchmarks actually were.\n- Section 3.4 also briefly mentions generalizability, which is then redefined in section 3.5. This made it difficult to understand the analysis in 3.4, so it would be helpful to have the definitions come before. \n\nExperiments\n\n- Beginning in Section 4.2, I found it challenging to follow the various experiments and the specific research questions they address. To improve the flow of the paper, it would be helpful to clearly state the research question each experiment is designed to answer. As it stands, the current order makes it difficult to track the purpose of each experiment.\n- The baselines in Table 1 lack sufficient explanation, making it unclear what CDL is being compared to—specifically, the CLIP and VDES baselines need further clarification. Additionally, Table 2 presents only a single value per column, although the main text suggests that this experiment was conducted across all datasets. Are the results in Table 2 averaged?\n- It seems that Figure 4 may display incorrect results for this paper. Based on the experiment’s goal of comparing LaBo to CDL, the figure legend mentions “LaBo (Ours),” which is confusing. It would be helpful to clarify the comparison being made here and provide the correct experimental results. \n- Figure 4 depicts a few-shot learning setting, but the experiment setup is unclear. In typical few-shot learning, a small number of training examples are used to fine-tune a classifier. However, Figure 4 mentions that \"concepts are selected,\" which is ambiguous. It would be helpful to clarify what is meant by this. \n- In Figure 5, it is unclear how many image examples and human annotations were used for these results. How were these examples selected, and how many annotations were made per example? Furthermore, how were such results calculated, and what is the Y-axis for these plots? \n- In section 4.5, the paper performs an ablation study testing the utility of CC-3M versus concept discovery on downstream datasets. However, Table 5 only shows the evaluation in comparison to CUB on in-domain and ImageNet on cross-domain. Why were not all downstream datasets used in this ablation, and what was the reasoning behind choosing one dataset for each in- and cross-domain comparison? \n\nMinor Comments\n\n- Figure 1: VDES is not defined in the first column. \n- Citation  missing for GPT-3-text-davinci-002. \n- The text in the figures is very small and could be made larger for better readability\n\n[1] 6Bring, J.: How to standardize regression coefficients. The American Statistician 48(3), 209–213 (1994)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work explores if pretrained vision-language models automatically learn visual concepts during their training, proposes a framework for extracting such visual concepts, and shows that such visual concepts can be used to train downstream concept bottleneck models. Prior works have focused on understanding if pretrained vision-language models automatically learn visual concepts, but fall short in two ways: first, their prompting strategies that probe for concepts are inconsistent, leading to differing conclusions about whether or not VLMs automatically learn visual concepts. Second, many prompting strategies might include non-visual concepts or include object categories in the prompts which can act as ‘shortcuts’, leading the VLM to rely on the object category rather than visual concepts during retrieval. To address this, this paper proposes a concept discovery and learning (CDL) framework. This framework first discovers general concepts using the CC-3M dataset, choosing those that have the most mutual information between a VLM and LLM. Then, concept bottleneck models can be trained on top of the discovered concepts.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Overall, this work has potential. However, in its current form, many of its contributions are not clearly communicated and could be better highlighted with improvements in writing. Specifically, the organization of the paper feels more like a long list of ideas rather than presenting the main points of each section (e.g., contributions) followed by the specific details (e.g., how those contributions are implemented). This is particularly evident in two areas: (1) in the introduction, where many details of the method are included but lack the necessary background and context for clarity, and (2) in the experiments section, where the experiments are presented consecutively but each one lacks sufficient detail for understanding, and the specific research question being addressed is often unclear. Furthermore, I am unable to fully evaluate the utility of the proposed method, as it appears that some of the experiments (e.g., Figure 4) may present incorrect results.", "broader_impact_concerns": "I have no concerns about the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bJgD8pcdtm", "reviewer": "Reviewer_cs4X", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The authors identified limitations of prior work (i.e., shortcuts and limited visual distinctiveness) and propose an approach that addresses both;\n- The concepts learned with the CDL framework are more general and discriminative, and also align better with human judgement;\n- The work includes a thorough evaluation on diverse benchmark datasets, and contains human evaluation to demonstrate the quality of the CDL concepts;\n- By removing the two limitations identified, the authors developed a framework that discovers more interpretable concepts that should generalise better to multiple domains.\n\nWeaknesses:\n- The framework depends on multiple components, i.e., a VLM (CLIP), an LLM (GPT3), and a data pool (CC3M) and their replacement with other backbones or pools was not considered. Specifically, I believe that the rationale behind this work (i.e., more general concepts, more interpretable concepts) would gain from larger data pools, e.g., CC12M. It would be interesting to see how the framework performs with larger and smaller data pools;\n- The final concept selection process relies on a generalisability score and a fixed budget for the number of concepts. I could see some benefits in having a data-driven method to address this point.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The work study whether Vision-Language Models (VLMs) inherently learn and encode visual concepts. It proposes a novel Concept Discovery and Learning (CDL) framework to extract generic and visually discriminative concepts, addressing limitations in prior work, i.e., category name bias and lack of visual distinctiveness. The authors argue that previous approaches, while utilising concept prompts, fail to prove whether VLMs actually learn visual concepts or rely on shortcuts, e.g., prompts containing both category names and concepts might achieve high accuracy simply because the VLM recognises the object from its name, not the descriptive concept. The proposed framework tackles these challenges by (i) identifying a diverse set of generic visual concepts from a web-scale image-caption dataset, and (ii) employing a multi-modal approach to select concepts that are relevant to describe an object and reliable to recognise it within an images. Additionally, the authors develop a suite of human evaluation metrics to assess the quality of the discovered concepts in terms of generalisability, precision, interpretability, and thoroughness. Through extensive evaluation on nine benchmark datasets, the authors demonstrate that their approach achieves comparable performance to prior works while yielding concepts that are more aligned with human understanding.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- The authors could expand their work by testing other data pools from where to extract the concepts. Currently, it is hard to understand what is the advantage of having a larger image-text dataset as we lack statistics for the number and types of concepts extracted. Moreover, CC3M is quite polished and small compared to other web-scale datasets, and having an analysis in this direction would shed some light also on the type of data pool we should consider to increase generalisability and (potentially) visual discriminability, i.e., I would expect different image-text datasets to have differences in their captioning style, and some could be more beneficial than others for the task. The authors could refer to [1] for a potential list of web-scale datasets to consider, or work with subsets of different dimensions of CC3M if the first approach is unfeasible.\n- The authors should expand on the pool of concepts discovered by the model, i.e., describing their quantity w.r.t. other works, and providing some other statistics on the matter, e.g., average concept length, POS tags.\n\n[1] Singh, Amanpreet, et al. \"Flava: A foundational language and vision alignment model.\" CVPR. 2022.", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: After the revisions, all reviewers are in favor of accepting the paper. The AC is inclined to agree.
Decision: Accept (poster)