Language Model as Visual Explainer

Authors: xingyi  yang
xinchao  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 06:38:35

Abstract: In this paper, we present Language Model as Visual Explainer (\texttt{LVX}), a systematic approach for interpreting the internal workings of vision models using a tree-structured linguistic explanation, without the need for model training. Central to our strategy is the collaboration between vision models and LLM to craft explanations. On one hand,  the LLM is harnessed to delineate hierarchical visual attributes, while concurrently,  a text-to-image API retrieves  images that are most aligned with these textual concepts. By mapping the collected texts and images  to the vision model's embedding space,  we construct a hierarchy-structured visual embedding tree. This tree is dynamically pruned and grown by querying the LLM using language templates, tailoring the explanation to the model.  Such a scheme allows us 
to seamlessly incorporate new attributes while eliminating undesired concepts based on the model's representations. When applied to testing samples, our method provides human-understandable explanations in the form of attribute-laden trees. Beyond explanation, we retrained the vision model by calibrating it on the generated concept hierarchy, allowing the model to incorporate the refined knowledge of visual attributes. To access the effectiveness of our approach, we introduce new benchmarks and conduct rigorous evaluations, demonstrating its plausibility, faithfulness, and stability.

Main paper: 
 introduction: 
 Unlocking the secrets of deep neural networks is akin to navigating through an intricate, ever-shifting maze, as the intricate decision flow within the networks is, in many cases, extremely difficult for humans to fully interpret. In this quest, extracting clear, understandable explanations from these perplexing mazes has become an imperative task.While efforts have been made to explain computer vision models, these approaches often fall short of providing direct and human-understandable explanations. Standard techniques, such as attribution methods [46, 55, 90, 68, 1, 62, 65, 64] , mechanical interpretability [22] and prototype analysis [10, 49] , only highlight certain pixels or features that are deemed important by the model. As such, these methods often require the involvement of experts to verify or interpret the outputs for nontechnical users. Natural language explanations [27, 7, 41, 35] , on the other hand, present an attractive alternative, since the produced texts are better aligned with human understanding. Nevertheless, these approaches typically rely on labor-intensive and biased manual annotation of textual rationales for model training.In this study, we attempt to explain AI decision in a human-understandable manner, for example, tree-structured language. We call this task visual explanatory tree parsing. To implement this, we present a systematic approach, Language Model as Visual Explainer (LVX), for interpreting vision models structured natural language, without model training. I see mostly muscular legs, not slim legs. A key challenge is that vision models, trained solely on pixel data, inherently lack comprehension of textual concepts within an image. For example, if a model labels an image as a "dog", it is unclear whether it truly recognizes the features like the wet nose or floppy ear, or if it is merely making ungrounded guesses. To address this challenge, we link the vision model with a powerful external knowledge provider, to establish connections between textual attributes and image patterns. Specifically, we leverage large language models (LLM) such as ChatGPT and GPT4 as our knowledge providers, combining them with the visual recognition system. Figure 1 (Left) describes a toy case, where the LLM is interacts with the vision model to explore its capability boundaries. By doing so, we gain insights into the what visual attributes can be recognized by the model.Vision Model Parse Tree ×𝒕 itersThe pipeline of our approach is illustrated in Figure 1 , which comprises two main stages, the construction phase and the test phase.In the construction phase, our goal is to create an attribute tree for each category, partitioning the feature space of a visual model via LLM-defined hierarchy. We begin by extracting commonsense knowledge about each category and its visual attributes from LLMs using in-context prompting [45] . This information is naturally organized as a tree for better organization and clarity. Utilizing a textto-image API, we gather corresponding images for each tree node. These images are subsequently inputted into the vision model to extract prototype embeddings, which are then mapped to the tree.Once created, the tree is dynamically adjusted, based on the properties of the training set. Specifically, each embedding of the training sample is extracted by the vision model.Such embedding then navigates the parse tree based on their proximity to prototype embeddings. Infrequently visited nodes, representing attributes less recognizable, are pruned. Conversely, nodes often visited by the model indicate successful concept recognition. Those nodes are growned, as the LLM introduces more detailed concepts. Consequently, LVX yields human-understandable attribute trees that mirror the model's understanding of each concept.In the test phase, we input a test sample into the model to extract its feature. The feature is then routed in the parse tree, by finding its nearest neighbors. The root-to-leaf path serves as a sample-specific rationale for the model, offering an explanation of how the model arrived at its decision.To assess our method, we compiled new annotations and developed novel metrics. Subsequently, we test LVX on these self-collected real-world datasets to access its effectiveness.Beyond interpretation, our study proposes to calibrate the vision model by utilizing the generated explanation results. The utilization of tree-structured explanations plays a key role in enhancing the model's performance, thereby facilitating more reliable and informed decision-making processes.Experimental results confirm the effectiveness of our method over existing interpretability techniques, highlighting its potential for advancing explainable AI.To summarize, our main contributions are:• The paper introduces a novel task, visual explanatory tree parsing, that interprets vision models using tree-structured language explanations.• We introduce the Language Model as Visual Explainer (LVX) to carry out this task, without model training. The proposed LVX is the first dedicated approach to leverage LLM to explain the visual recognition system.• Our study proposes utilizing the generated explanations to calibrate the vision model, leading to enhanced performance and improved reliability for decision-making.• We introduce new benchmarks and metrics for a concise evaluation of the LVX method. These tools assess its plausibility, faithfulness, and stability in real-world datasets. 
problem definition: 
 We first define our specialized task, called visual explanatory tree parsing, which seeks to unravel the decision-making process of a vision model through a tree. Let us consider the trained vision model f , defined as a function f : X → Y, where X represents the input image space and Y denotes the output label space. In this study, our focus lies on the classification task, where f = g • h is decomposed into a feature extractor g and a linear classification head h. The output space is Y ∈ R n , where n signifies the number of classes. The model is trained on a labeled training set D tr = {x j , y j } M j=1 , and would be evaluated a test set D ts = {x j } L j=1 . The ultimate objective of our problem is to generate an explanation T for each model-input pair (f, x) on the test set, illuminating the reasoning behind the model's prediction ŷ = f (x). This unique explanation manifests as a tree of attributes, denoted as T = (V, E), comprising a set of N v nodes V = {v i } Nv i=1 and N e edges E = {e i } Ne i=1 . The root of the tree is the predicted category, ŷ, while each node v i encapsulates a specific attribute description of the object. These attributes are meticulously organized, progressing from the holistic to the granular, and from the general to the specific. Figure 2 provides an example of the parse tree.Unlike existing approaches [53, 3] that explaining visual-language models [48, 47, 51, 86, 93] , we address the more challenging scenario, on explaining vision models trained solely on pixel data. While some models can dissect and explain hierarchical clustering of feature embeddings [67, 77] , they lack the ability to associate each node with a textual attribute. It is important to note that our explanations primarily focus on examining the properties of the established network, going beyond training vision model or visual-language model [2, 44] for reasoning hierarchy [20] and attributes [31] from the image. In other words, visual-language model, that tells the content in the image, can not explain the inner working inside another model. Notably, our approach achieves this objective without supervision and in open-vocabulary manner, without predefined explanations for model training. 
language model as visual explainer: 
 This section dives deeper into the details of LVX. At the heart of our approach is the interaction between the LLM and the vision model to construct the parsing tree. Subsequently, we establish a rule to route through these trees, enabling the creation of coherent text explanations.Before constructing our trees, let's take a moment to reflect how humans do this task. Typically, we already hold a hierarchy of concepts in our minds. When presented with visual stimuli, we instinctively compare the data to our existing knowledge tree, confirming the presence of distinct traits. We recognize familiar traits and, for unfamiliar ones, we expand our knowledge base. For example, when we think of a dog, we typically know that it has a furry tail. Upon observing a dog, we naturally check for the visibility of its tail. If we encounter a hairless tail, previously unknownQ: this is a deer because A: { "Concepts": …, "Substance": ["four legs", "two ears", "two eyes"], "Attributes": { "Head": { "Ears": ["pointy", "sensitive"], "Eyes": ["large", "brown"] }, "Body": { "Tail": ["short", "furry"], "Legs": ["long", "slender", "hooves"] } }, "Environment": … } Q: this is a seal because A:{ "Concepts": …, "Substance": ["flippers", "fur", "teeth"],"Attributes": { "Head": { "shape": "round", "Nose": ["small", "black"], "Eyes": ["large", "dark"] }, "Body": { "Tail": ["short", "rounded"] } }, "Environment": ["coastal areas"] } Instruction: Generate the all sentences that describe a concept according to the each attributeAttribute Tree Prompt Tree { "Concepts": …, "Substance": ["A seal has flippers.", "A seal has fur.", "A seal has teeth."],"Attributes": { "Head": { "shape": "A seal's head is round in shape.", "Nose": ["A seal's nose is small.", "A seal's nose is black"], "Eyes": ["A seal's eyes are large.", "A seal's eyes are dark."] }, "Body": { "Tail": ["A seal's tail is short.", "A seal's tail is rounded."] } }, "Environment": ["A seal lives in coastal areas."] } A seal's eyes are dark. A seal's tail is short.A seal lives in coastal areas.A seal's nose is small. to us, we incorporate it into our knowledge base, ready to apply it to other dogs. This process is typically termed Predictive Coding Theory [15] in cognitive science.Our LVX mirrors this methodology. We employ LLM as a "knowledge provider" to construct the initial conceptual tree. Subsequently, we navigate through the visual model's feature space to assess the prevalence of each node. If a specific attribute is rarely observed, we remove the corresponding nodes from the tree. Conversely, if the model consistently recognizes an attribute, we enrich the tree by integrating more nuanced, next-level concepts. This iterative process ensures the refinement and adaptation of the conceptual tree within our pipeline, which gives rise to our LVX.Generating Textual Descriptions for Visual Concepts. We leverage a large language model (LLM) as our "commonsense knowledge provider" [42, 95] to generate textual descriptions of visual attributes corresponding to each category. The LLM acts as an external database, providing a rich source of diverse visual concept descriptions. The process is illustrated in Figure 3 .Formally, assume we have a set of category names, denoted as C = {c i } n i=1 , where i represents the class index. For each of these classes, we prompt an LLM L to produce visual attribute tree. We represent these attributes as d i = L(c i , P), where d i is a nested JSON text containing textual descriptions associated with class c i . To help generate d i , we use example input-output pairs, P, as in-context prompts. The process unfolds in two stages:• Initial Attribute Generation: We initially generate keywords that embody the attributes of each class. This prompt follows a predefined template that instructs the LLM to elaborate on the attributes of a visual object. The template is phrased as "This is a <CLSNAME> because" . The output JSON contains four primary nodes: Concepts, Substances, Attributes, and Environments. As such, the LLM is prompted to return the attributes of that concept. Note that the initial attributes tree may not accurately represent the model; refinements will be made in the refinement stage.• Description Composition: Next, we guide the LLM to create descriptions based on these attributes. Again we showcase an in-context example and instruct the model to output "Generate sentences that describe a concept according to each attribute." .Once the LLM generates the structured attributes d i , we parse them into an initial tree, represented asEQUATIONi ), using the key-value pairs of the JSON text. Those generated JSON tree is then utilized to query images corresponding to each factor.Visual Embeddings Tree from Retrieved Images. In order to enable the vision model to understand attributes generated by the LLM, we employ a two-step approach. The primary step involves the conversion of textual descriptions, outputted by the LLM, into images. Then, these images are deployed to investigate the feature region that symbolizes specific attributes within the model.The transition from linguistic elements to images is facilitated by the use of arbitrary text-to-image API. This instrumental API enables the generation of novel images or retrieval of existing images that bear strong relevance to the corresponding textual descriptions. An initial parse tree node, denoted by v, containing a textual attribute, is inputted into the API to yield a corresponding set of K support images, represented as{ x i } K i=1 = T2I(v).The value of K is confined to a moderately small range, typically between 5 to 30. The full information of the collected dataset will be introduced in Section 4.Our research incorporates the use of search engines such as Bing, or text-to-image diffusion models like Stable-Diffusion [56] , to derive images that correspond accurately to the provided attributes.Following this, the images are presented to the visual model to extract their respective embeddings, represented as p i = g( x i ). As such, each tree node contains a set of support visual features P = {p k } K k=1 . This procedure allows for the construction of an embedding tree, consisting of paired text and visual features. These pairs are arranged in a tree structure prescribed by the LLM. It is important to note that the collected images are not employed in training the model. Instead, they serve as a support set to assist the model in understanding and representing the disentangled attributes effectively. As such, the visual model uses these embeddings as a map to navigate through the vast feature space, carving out territories of attributes, and laying down the groundwork for further exploration and explanation of a particular input.Tree Refinement Via Refine Prompt. Upon construction, the parse tree structure is refined to better align with the model's feature spaces. This stage, termed Tree Refinement, is achieved through passing training data as a query to traverse the tree. Nodes that are seldom visited indicate that the model infrequently recognizes their associated attributes. Therefore, we propose a pruning mechanism that selectively eliminates these attributes, streamlining the tree structure. For nodes that frequently appear during the traversal, we further grow the tree by introducing additional or more detailed attributes, enriching the overall context and depth of the tree. The procedure is demonstrated in Figure 4 .Initially, we treat the original training samples, denoted as (x j , y j ) ∈ D tr , as our query set. Each sample is passed to the visual model to extract a feature, represented as q j = g(x j ).Next, the extracted feature traverses the y j -corresponding tree. Its aim is to locate the closest semantic neighbors among the tree nodes. We define a distance metric between q j to support set P as the point-to-set distance D(q j , P ). This metric represents the greatest lower bound of the set of distances from q j to prototypes in P . It is resilient to outliers and effectively suppresses non-maximum nodes.EQUATIONIn our paper, similar to [10, 58] , we set d(q, p) = -log(1 +1 ||q-p|| 2 ) 2 .It emphasizes close points while moderating the impact of larger distances. Following this, we employ a Depth-First Search (DFS) algorithm to locate the tree node closest to the query point q j . After finding this node, each training point (x j , y j ) is assigned to a specific node of the tree. Subsequently, we count the number of samples assigned to a particular node v * , using the following formula:EQUATIONIn this formula, 1 is the indicator function and P v denotes the support feature for node v. Following this, we rank each node based on the sample counter, which results in two operations to update the tree architectureT (t+1) i = Grow(Prune(T (t) i )), where t stands as the iteration number• Tree Pruning. Nodes with the least visits are pruned from the tree, along with their child nodes.• Tree Growing. For the top-ranked node, we construct a new inquiry to prompt the LLM to generate attributes with finer granularity. The inquiry is constructed with an instruction template "Add visual attributes for the <NodeName> of a <ClassName>, to the json" .• Common Node Discrimination. In cases where different categories share common nodes (e.g. "human" and "dog" both have "ear"), we execute a targeted growth step aimed at distinguishing between these shared elements. To achieve this differentiation, we utilize a contrasting question posed to the LLM "The <NodeName> of <ClassName1> is different from <Classname2> because" .The revised concept tree generated by the LLM provides a comprehensive and detailed representation of the visual attribute. To refine the attribute further, we employ an iterative procedure that involves image retrieval and the extraction of visual embeddings, as illustrated in Figure 1 . This iterative process enhances the parse tree by incorporating new elements. As each new element is introduced, the attribute areas within the feature space become increasingly refined, leading to improved interpretability. In our experiment, we performed five rounds of tree refinement.Once the tree is established, the model predicts the class of a new test sample x ′ and provides an explanation for this decision by finding the top-k nearest neighbor nodes.Specifically, the model predicts the category ŷ for the test instance x ′ as ŷ = f (x ′ ). The extracted image feature q ′ corresponding to x ′ is routed through the tree. Starting from the root, the tree is traversed to select the top-k nearest neighbor nodes {v i } k i=1 based on the smallest D(q ′ , P vi ) values, representing the highest semantic similarity between q ′ and the visual features in the tree's nodes. The paths from the root to the selected nodes are merged to construct the explanatory tree T for the model's prediction. This parse tree structure reveals the sequence of visual attributes that influenced the model's classification of x ′ as ŷ. It facilitates the creation of precise, tree-structured justifications for these predictions. Importantly, the routing process involves only a few feature similarity computations per node and does not require queries to the large language model, resulting in exceptionally fast computation.The created parse tree offers a two-fold advantage. Not only does it illustrates the logic of a specific prediction, but it also serves as a by-product to refine the model's predictions by introducing hierarchical regularization for learned representation. Our goal is to use the parse tree as pseudo-labels, embedding this hierarchical knowledge into the model.To operationalize this, we employ a hierarchical multi-label contrastive loss (HiMulCon) [92] , denoted as L HM C , to fine-tune the pre-trained neural network. This approach enhances the model by infusing structured explanations into the learning process, thus enriching the representation. Specifically, we apply the LVX on all training samples. The explanatory path Tj provides a hierarchical annotation for each training sample x j . The model is trained with both the cross-entropy loss L CE and L HM C as follows:EQUATIONHere, λ is a weighting coefficient. The explanation Tj is updated every 10 training epochs to ensure its alignment with the network's evolving parameters and learning progress. Notably, the support set isn't used in model training, maintaining a fair comparison with the baselines. 
experiment: 
 This section offers an in-depth exploration of our evaluation process for the proposed LVX framework and explains how it can be utilized to gain insights into the behavior of a trained visual recognition model, potentially leading to performance and transparency improvements. Figure 5 : Plausibility comparison on three visual tree parsing benchmarks. We plot the mean±std across all networks architectures. For both scores, higher values indicate better performance.Data Annotation and Collection. To assess explanation plausibility, data must include human annotations. Currently, no large-scale vision dataset with hierarchical annotations is available to facilitate reasoning for visual predictions. To address this, we developed annotations for three recognized benchmarks: CIFAR10, CIFAR100 [39] , and ImageNet [57] , termed as H-CIFAR10, H-CIFAR100, and H-ImageNet. These annotations, detailed in Table 8 , serve as ground truth for model evaluation, highlighting our dataset's unique support for hierarchical attributes and diverse visual concepts. Note that, we evaluate on hierarchical datasets only, as our method is specifically designed for structured explanations.As an additional outcome of our framework, we have gathered three support sets to facilitate model explanation. In these datasets, each attribute generated by the LLM corresponds to a collection of images that showcase the specified visual concepts. These images are either retrieved from Bing search engine3 using attributes as queries or are generated using Stable-diffusion. We subsequently filter the mismatched pairs with the CLIP model, with the threshold of 0.5. Due to the page limit, extensive details on data collection, false positive removal, limitations, and additional evaluation of user study and on medical data, such as X-ray diagnoses, are available in the supplementary material.Evaluation Metrics. In this paper, we evaluate the quality of our explanation from three perspectives: Plausibility, Faithfulness and Stability.• Plausibility measures how reasonable the machine explanation is compared to the human explanation. We measure this by the graph distance between the predicted and ground-truth trees, using two metrics: Maximum Common Subgraph (MCS) [54, 33] , and Tree Kernels (TK) [71] . We calculate their normalized scores respectively. Specifically, given a predicted tree T pred and the ground-truth T gt , the MCS score is computed as|M CS|×100 √ |T pred ||Tgt|, and the TK score is computed asT K(T pred ,Tgt)×100 √ T K(T pred ,T pred )T K(Tgt,Tgt). Here, | • | represents the number of nodes in a tree, and T K(•, •)denotes the unnormalized TK score. We report the average score across all validation samples.• Faithfulness states that the explanations should reflect the inner working of the model. We introduce Model-induced Sample-Concept Distance (MSCD) to evaluate this, calculated as the average of point-to-set distances 1Nv v∈V D(q j , P v ) between all test samples and tree nodes, reflecting the alignment between generated explanation and model's internal logic. The concept is simple: if the explanation tree aligns with the model's internal representation, the MSCD is minimized, indicating high faithfulness.• Stability evaluates the resilience of the explanation graph to minor input variation, expecting minimal variations in explanations. The MCS/TK metrics are used to assess stability by comparing explanations derived from clean and slightly modified inputs. We include 3 perturbations, including Gaussian additive noise with σ ∈ {0.05, 0.1} and Cutout [18] augmentation.Baselines. We construct three baselines for comparisons: Constant, using the full category template tree; Random, which selects a subtree randomly from the template; and Subtree, choosing the most common subtree in the test set for explanations. Additionally, we consider TrDec Baseline [79] , a strategy utilizing a tree-topology RNN decoder on top of image encoder. Given the absence of hierarchical annotations, the CLIP model verifies nodes in the template trees, serving as pseudo-labels for training. We only update the decoder parameters for interpretation purposes. These models provide a basic comparison for the performance of LVX. More details are in the appendix.For classification performance, we compare LVX-calibrated model with neural-tree based solutions, including a Decision Tree (DT) trained on the neural network's final layer, DNDF [38] , and NBDT [77] .Models to be Explained. Our experiments cover a wide range of neural networks, including various convolutional neural networks (CNN) and transformers. These models consist of VGG [66] ,ResNet [26] , DenseNet [29] , GoogLeNet [72] , Inceptionv3 [73] , MobileNet-v2 [59] , and Vision Transformer (ViT) [19] . In total, we utilize 12 networks for CIFAR-10, 11 networks for CIFAR-100, and 8 networks for ImageNet. For each model, we perform the tree refinement for 5 iterations.Calibration Model Training. As described in Section 3.3, we finetune the pre-trained neural networks with the hierarchical contrastive loss based on the explanatory results. The model is optimized with SGD for 50 epochs on the training sample, with an initial learning rate in {0.001, 0.01, 0.03} and a momentum term of 0.9. The weighting factor is set to 0.1. We compare the calibrated models with the original ones in terms of accuracy and explanation faithfulness.Plausibility Results. We evaluated LVX against human annotations across three datasets, using different architectures, and calculating MCS and TK scores. The results, shown in Figure 5 , reveal LVX outperforms baselines, providing superior explanations. Notably, TrDec, even when trained on CLIP induced labels, fails to generate valid attributes in deeper tree layers-a prevalent issue in long sequence and structure generation tasks. Meanwhile, SubTree lacks adaptability in its explanations, leading to lower scores. More insights are mentioned in the appendix.Faithfulness Results. We present the MSCD scores for ResNet-18(RN-18), ResNet-50(RN-50), and ViT-S, contrasting them with SubTree and TrDec in Table 3 . Thanks to the incorporation of tree refinement that explicitly minimizes MSCD, our LVX method consistently surpasses benchmarks, demonstrating lowest MSCD values, indicating its enhanced alignment with model reasoning.Stability Results. The stability of our model against minor input perturbations on the CIFAR-10 dataset is showcased in Table 2 , where MCS/TK are computed. The "Clean" serves as the oracle baseline. Our method, demonstrating robustness to input variations, retains consistent explanation results (MCS>60, TK>85). In contrast, TrDec, dependent on an RNN-parameterized decoder, exhibits higher sensitivity to feature variations. Model and Data Diagnosis with Explanation. We visualize the sample explanatory parse tree on ImageNet validation set induced by ViT-B in Figure 6 . The explanations fall into three categories: (1) correct predictions with explanations, (2) incorrect predictions with explanations, and (3) noisy label predictions with explanations. We've also displayed the 5 nearest neighbor node for each case.What's remarkable about LVX is that, even when the model's prediction is wrong, it can identify correct attributes. For instance, in a case where a "white shark" was misidentified as a "killer whale" (b-Row 2), LVX correctly identified "fins", a shared attribute of both species. Moreover, the misrecognition of the attribute "wide tail flukes" indicates a potential error in the model, that could be later addressed to enhance its performance.Surprisingly, LVX is able to identify certain noisy labels in the data, as shown in c-Row 2. In such cases, even experienced human observers might struggle to decide whether a "pig bank with band" should be classified "piggy bank" or "band aid". It again underscores the superior capabilities of our LVX system in diagnosing the errors beyond model, but also within the data itself.Calibration Enhances Interpretability and Performance. Our approach involves fine-tuning a pre-trained model with the loss function outlined in Section 3.3, using parsed explanatory trees to improve model performance. Table 4 compares the classification performance of our model with that of other neural tree methods. Our model clearly outperforms the rest.Neural tree models often face challenges in balancing interpretability with performance. In contrast, LVX achieves strong performance without relying on a strict decision tree. Instead, decisions are handled by the neural network, with concepts guided by the LLM through Equation 3. This approach enhances the model's ability to disentangle visual concepts while preserving high performance.In addition, we compared the quality of the generated parsed tree with or without calibration, in Figure 5 . The calibration process not only improved model performance, but also led to more precise tree predictions, indicating enhanced interpretability. We also test the calibrated model on OOD evaluations in Appendix, where we observe notable improvements. 
ablation study and analysis: 
 In this section, we present an ablation study on the refinement stage of LVX. We also apply the method to different neural networks to observe variations in model's behavior. Ablation 1: No Refinement. To study the impact of refinement stage, we present a baseline called w/o Refine. In this setup, the initial tree generated by LLMs is kept fixed. We evaluate the method using the MSCD for faithfulness and MCS for plausibility on the CIFAR-10 and CIFAR-100 datasets.The results show in Table 6 that incorporating image model feedback indeed improves tree alignment with the classifier's internal representation, as reflected in higher MCS scores. The refined trees also better match human-annotations.Ablation 2: Refinement Criteria. In our original method, tree refinement is based on feature similarity to the training set. To explore an alternative, we use average activation magnitude on generated data as the criterion for concept familiarity. Concepts with activation magnitudes ≤ η are pruned. This method, referred to as ActMag, is evaluated on CIFAR-10. We report the MCS, MSCD for performance, and average tree depth as an indicator of tree complexity.Table 7 shows that feature similarity achieves better results than ActMag. Specifically, setting a threshold is challenging for ActMag, leading shallow trees (η = 0.3) or too deep ones (η = 0.01). Analysis: CNN vs. Transformer. We use our LVX to compare CNN and Transformer models and identify which concepts they miss. We compared ConvNeXt-T (CNN) and DeiT-B (Transformer) on 26,928 concepts we collected on ImageNet, from sub-categories of Concepts, Substances, Attributes, and Environments. We measured accuracy across 4 sub-categories and tree depths.Results show that ConvNeXt-T is better at local patterns (Attributes, Substances), while DeiT-B perform better on Environments which needs global semantics. Additionally, DeiT-B is more accurate at shallow depths, whereas ConvNeXt-T performs better at deeper levels. These findings aligns with earlier research showing that CNN are biased towards textures over shape [23, 84] .Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes]Justification: Code has been uploaded as supplementary material. Data will be available.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion: 
 In this study, we introduced LVX, an approach for interpreting vision models using tree-structured language explanations without hierarchical annotations. LVX leverages large language models to connect visual attributes with image features, generating comprehensive explanations. We refined attribute parse trees based on the model's recognition capabilities, creating human-understandable descriptions. Test samples were routed through the parse tree to generate sample-specific rationales.LVX demonstrated effectiveness in interpreting vision models, offering potential for model calibration.Our contributions include proposing LVX as the first approach to leverage language models for explaining the visual recognition system. We hope this study potentially advances interpretable AI and deepens our understanding of neural networks.[ Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: All details mentioned.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: Errorbar presented.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: All discussed in the main paper.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: Yes, all properly credited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes] Justification: Code and data are well documented.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [No] Justification: Not applicable Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: This document presents supplementary experiments and information regarding our proposed LVX framework. In Section C, we provide an overview of the algorithm pipeline for LVX. In Section D, we detail the process of data collection and its subsequent analysis. In additionally, Section E provides the user study results; Section F shows the calibrated training further improve the OOD performance. Section G showcases the additional experimental results obtained from a specialized application on X-Ray diagnosis. Furthermore, in Section H, we demonstrate the explanation results achieved using self-supervised models. We also provide the raw experimental values in Section I. Finally, we outline the experimental setup, metric definitions, and dataset collection protocols.Neural Tree. Neural Trees (NTs) intend to harmonize the performance of Neural Networks (NNs) and interpretability of Decision Trees (DTs) [17, 21, 60, 96, 14] within a unified model. They evolved from mimicking NNs with DTs [17, 21, 60, 96, 14] to becoming inherently interpretable tree-structured networks, adapting their structure via gradient descent [70, 94, 32, 85, 75, 38] . Neural-Backed Decision Trees (NBDTs) [77] use a trained NN as a feature extractor, replacing its final layer with a decision tree. Our model builds on these advances to create a hierarchical tree from a pre-trained NN and provides post-hoc explanations without additional training, which increases interpretability and potentially enhances performance.Prototype-based Explainable Model. Prototype models use representative training data points to symbolize classes or outcomes [16, 30, 37] . Revived in deep learning and few-shot learning [69, 82] , they justify decisions by comparing new instances to key examples [10, 88, 40] . Recent work has developed this approach through hierarchical and local prototypes [49, 34, 74] . However, the prototypes serve as an indirect explanation for model's prediction, necessitating further human justification. Our LVX addresses this by assigning semantic roles to prototypes through LLM, turning them from simple similarity points to data points with clear definitions, thereby enhancing explainability.Composing Foundation Models. Model composition involves merging machine learning models to address tasks, often using modular models for sub-tasks [5, 28, 4, 83] , restructured by a symbolic executor [89] . Recently, Language Learning Models (LLMs) have been used as central controllers, guiding the logic of existing models [63, 25, 87, 43] or APIs [61] , with language as a universal interface [91] . However, composing language model with non-language ones lacks a unified interface for bilateral communication. In this study, we propose the use of a text-to-image API as a medium to enable the language model to share its knowledge with the visual task. This allows the vision model to benefit from the linguistic context and knowledge hierarchy, thereby enhancing its transparency.In this section, we present the pseudocode for the LVX framework, encompassing both the construction stage and the test stage. The algorithmic pipelines are outlined in Algorithm 1 and Algorithm 2.The category-level tree construction pipeline, as demonstrated in Algorithm 1, involves an iterative process that utilizes a large language model (LLM), like ChatGPT. This process allows us to construct an attribute tree for each category. It begins by generating prompts based on the category name and using them to gather attribute information. This forms the initial tree structure. Support images are collected using a text-to-image API, and their visual features are associated with the corresponding tree nodes. The process iterates until the maximum run, continuously refining the attribute tree for corresponding category.During the test stage, as outlined in Algorithm 2, the test samples undergo a traversal process within the constructed category-level trees. The goal is to locate the subtree that best aligns with the correct for j = 1 to M do 11:Extract feature for training data: qj = g( xj). 12:Assign training data to a tree node: v * = argmin v∈V (t) y j D(qj, Pv). 1: Prediction on xts: q = g(xts) and ŷ = h(q). 2: Find the top-matched sub-tree in T ŷ T * = argminT * ⊆T ŷ k i=1 D(q, Pv i ) s.t. T * = {V * , E * }, vi ∈ V * , |V * | = k3: return T * as the prediction explanation.prediction rationales. This sample-wise attribute tree process enables the identification of pertinent attributes and explanations linked to each test sample, providing insights into the decision-making process of the model.In summary, the LVX employs an iterative approach to construct category-level trees, leveraging the knowledge of LLMs. These trees are then utilized during the test stage to extract relevant explanations. This methodology enables us to gain understanding of the model's decision-making process by revealing the underlying visual attributes and rationales supporting its predictions.The creation of test annotations for three datasets involved a semi-automated approach implemented in two distinct steps. This process was a collaborative effort between human annotators, a language model (ChatGPT), and CLIP [53] , ensuring both efficiency and reliability.1. Concept Tree Creation. In this first step, we utilized ChatGPT4 to generate initial attribute trees for each class with the category name, detailed Section 3.1. 2. Attribute Verification. To determine whether an attribute was present or absent in an image, we employed an ensemble of predictions from multiple CLIP [53] models 5 . We filtered the top-5 attributes predicted by CLIP and sought human judgments to verify their correctness.To streamline this process, we developed an annotation tool with a user interface, which is illustrated in Figure 7 . 3. Manual Verification. In this phase, annotators examined the accuracy of existing attributes and introduced new relevant ones to enrich the concept trees. Subsequently, human annotators conducted a thorough review, refinement, and systematic organization of the attribute trees for each class. To optimize the image collection process, we merge the retrieved images from all models, leading to time and effort savings. This approach allows us to reuse an image already in the dataset if it matches an attribute generated by the LLM. As a result, after the initial models have finished collecting data, subsequent models can simply pull relevant images from this existing pool instead of collecting new ones, saving both time and effort. Data Statistics. We present the statistics of the support datasets collected for CIFAR10, CIFAR100, and ImageNet, highlighting the diversity and comprehensiveness of our dataset. Table 8 and Figure 8 provide an overview of these statistics. Specifically, we include the number of attributes, the number of samples for each category, the total number of samples in the dataset, as well as the distribution of tree depths. This rich collection of data showcases the diverse range of attributes and categories covered in our dataset, making it a valuable resource for training and evaluation purposes.While collecting a newly curated dataset can be advantageous for our tasks, it is important to acknowledge certain limitations when using such datasets for explanation purposes. Three key limitations arise: false positive simages, the presence of out-of-distribution samples and potential bias in the dataset.• False Positive Images. We observed that both Bing and the Stable Diffusion model occasionally generate imperfect images from textual descriptions, manifesting incorrect or entangled patterns. For instance, the word "crane" could represent either a construction machine or a bird, leading to ambiguity. Furthermore, an image described as "a dog with a long tail" could potentially include not only the tail but also the head and legs, reflecting a broader scope than intended.• Out-of-Distribution Samples. Newly collected datasets may include samples that are out-of-distribution, i.e., they do not align with the source data distribution of interest. These out-of-distribution samples can introduce challenges in generating accurate and reliable explanations. As a result, explanations based solely on a newly collected dataset may not generalize well to unseen instances outside the support dataset distribution.• Data Bias. Biases in the collection process or the underlying data sources can inadvertently influence the dataset, leading to biased explanations. Biases emerge due to various factors, such as data collection source, or imbalances in attribute distributions. Consequently, relying solely on a newly collected dataset for explanations may introduce unintended biases into the interpretation process.Solutions. To deal with mistakes in the gathered images, we used two approaches. First, we used the CLIP model to sift through the images because it's good at understanding how close an image is to text concepts, helping us remove most mixed-up and incorrect images. Second, we manually sorted out words that have more than one meaning. For instance, we made it clear whether "crane" refers to the bird or the machine by labeling it as "crane (bird)" or "crane (machine)".To mitigate the challenges posed by OOD samples and data bias, we adopt a cautious approach. Specifically, we do not directly train our models on the newly collected support dataset. Instead, we utilize this dataset solely for the purpose of providing disentangled attributes for explanations. By decoupling the training data from the support data, we aim to reduce the impact of OOD samples and potential data biases, thus promoting a more robust and unbiased analysis.Despite these limitations, our emphasis on obtaining a dataset with distinct attributes sharpens our analysis and interpretation of model behavior. This approach allows us to extract meaningful insights in a controlled and clear way. In this section, we evaluate the performance of our LVX method in security-critical domains, specifically medical image analysis. We train neural networks for chest X-ray diagnosis and utilize LVX to interpret and calibrate the predictions.We adopted the DenseNet-121 architecture for disease diagnosis in our study. The model was trained on the Chestx-ray14 dataset [78] , which consists of chest X-ray images encompassing 14 diseases, along with an additional "No Finding" class. The DenseNet-121 architecture is specifically designed to generate 14 output logits corresponding to the different diseases. During training, we employed a weighted binary cross-entropy loss [78] for each disease category to optimize the model. For optimization, we utilized the Adam optimizer [36] with an initial learning rate of 1e-4, a weight decay of 1e-5, and a batch size of 32. The model underwent training for a total of 18 epochs.Model Explanation. To enhance interpretability, we incorporated our LVX framework into the model. Instead of acquiring images from online sources, we gathered the support set directly from the training data. To accomplish this, we utilized a parse tree generated by the ChatGPT language model. Leveraging this parse tree, we applied a MedCLIP [80] model to retrieve the most relevant images for each attribute from the training set. These retrieved images served as our support sets for the LVX framework.Compared to applying the LVX framework on single-label classification, the Chestx-ray14 dataset poses a multi-label classification challenge. In this dataset, each sample can belong to multiple disease categories simultaneously. Therefore, we modified the LVX framework to accommodate and handle the multi-label nature of the classification task.Specifically, for each input image x, we predict its label ŷ = f (x) ∈ {0, 1} 14 . To create the visual parse tree, we begin by establishing the root node. If all elements of ŷ are 0, the root node is set to "No Findings". Conversely, if any element of ŷ is non-zero, the root node is labeled as "has Findings". For each positive finding, we construct a separate parse tree, with these sub-trees becoming the children nodes of the root. By combining these sub-trees, we obtain a comprehensive and coherent explanation for the image. This modification enables us to effectively handle the multi-label nature of the classification task, providing meaningful and interpretable explanations for images with multiple positive findings.To establish the ground-truth explanation label, we adopt a MedCLIP [80] model to filter the top-5 attributes for each positive finding of the image. These attributes are then organized into a tree structure. This approach serves as an automatic explanation ground-truth, thereby eliminating the requirement for manual annotations from domain experts.In addition to providing explanations, we aim to calibrate the model predictions with the parse tree.To achieve this, we apply a modified hierarchical contrastive loss individually on each finding. We then calculate the average of these losses, which serves as our overall loss term. We thus fine-tune the model for 3 epochs using the hieracical term and weighted cross-entropy.Explanation Results. We compare the explanation performance of our proposed LVX against the Random and Constant baselines. The numerical results, depicted in Figure 9 , highlight the superiority of our LVX approach.Additionally, we showcase the parsed visual tree in Figure 10 , to provide a clearer illistration of our results. Notably, our approach effectively interprets the decision-making process of black-box neural networks. For instance, in the case on the right, our method accurately identifies the presence of visible fluid in the lung space and establishes its relevance to the model's prediction. Consequently, LVX enables clinical professionals to make well-informed justifications for their patients, enhancing the overall decision-making process.Calibration Results. Table 11 presents the comparison between the baseline models and the model with calibration, measured in terms of the Area Under the Curve (AUC) score for each disease type. The AUC score provides a measure of the model's ability to discriminate between positive and negative cases.The calibrated model shows notable improvements in several disease types compared to the baseline. Notably, Hernia demonstrates the most significant improvement, with an AUC score of 0.936 compared to 0.914 for the baseline. This indicates that the calibration process has enhanced the model's ability to accurately detect Hernia cases.In summary, the LVX method markedly improves model accuracy, demonstrated by enhanced calibration performance across different disease types. The integration of visual attributes boosts both accuracy and reliability of predictions, leading to better diagnostic results. These findings underscore the LVX method's potential to elevate model performance, particularly in medical diagnostics. In this section, we focus on self-supervised models to assess their interpretability. Differing from supervised models, self-supervised models develop representations without labeled data. Our aim is to understand the interpretability of these representations and uncover the underlying structures derived from the input data alone.Model to be Explained. Our objective is to offer a comprehensive explanation for self-supervised models trained on ImageNet-1k. These models include ResNet50 trained using SimCLR [11] , BYOL [24] , SwAV [8] , MoCov3 [12] , DINO [9] , and ViT-S trained with MoCov3 [13] and DINO [9] . The networks are subsequently fine-tuned through linear probing while keeping the backbone fixed.We then utilize our LVX to provide explanations for their predictions. Additionally, we compare these self-supervised models with their supervised counterparts to highlight the differences in representation between the two approaches.Numerical Results. Table 12 presents the results of self-supervised models. Our analysis reveals a strong correlation between the explanatory performance and the overall model accuracy.However, we also noticed that self-supervised models based on transformer architecture exhibit greater attribute disentanglement compared to supervised models, despite potentially having slightly lower performance. This phenomenon is evident when comparing the DINO ViT-S/16 model and the supervised ViT-S/16 model within the context of tree parsing explanation. Although the DINO ViT-S/16 model shows slightly lower overall performance, it outperforms the supervised model in terms of providing accurate attribute explanation.These results underscore the potential benefits of self-supervised learning in uncovering meaningful visual attributes without explicit supervision. While self-supervised models may exhibit marginally lower performance on certain tasks, their ability to capture rich visual representations and attribute disentanglement highlights their value in understanding complex visual data. This section presents the raw numerical results for Figure 5 , as depicted in the main paper. Specifically, Table 13 provides the results for CIFAR-10, Table 14 for CIFAR-100, and Table 15 for ImageNet. We also observed that larger networks within the same model family deliver better results. As the models improve, so does the accuracy of the explanations, suggesting that larger networks facilitate more effective explanations. This is demonstrated by the increase in MCS and TK scores as ResNet deepens on CIFAR-100 and ImageNet, aligning with the general belief that larger neural networks offer enhanced generalization and structural representation capabilities.In this section, we provide detailed information about our experimental setup to ensure the reproducibility of our method. To evaluate the effectiveness of our proposed tree parsing task, we have developed three metrics that leverage conventional tree similarity and distance measurement techniques.• Tree Kernels (TK): Tree Kernels (TK) evaluate tree similarity by leveraging shared substructures, assigning higher scores to trees with common subtrees or substructures. To enhance the match, we set the decaying factor for two adjacent tree layers to 0.5, where larger values lead to better matches. Let's define the subtree kernel mathematically: Tree Kernel: Given two trees, T 1 and T 2 , represented as rooted, labeled, and ordered trees, we define the subtree kernel as follows:T K(T 1 , T 2 ) = T T ′ θ(T, T ′ ) × θ(T, T ′ ) × λ max(depth(r),depth(r ′ ))Let T K(T 1 , T 2 ) denote the similarity between subtrees T 1 and T 2 using the subtree kernel. Additionally, let θ(T, T ′ ) represent the count of shared common subtrees between trees T and T ′ . Furthermore, let r and r ′ be the roots of T and T ′ respectively. λ < 1.0 is the decaying factor that make sure the tree closer to the root hold greater significance. The θ(T, T ′ ) is computed recursively as follows:1. If both T and T ′ are leaf nodes, then θ(T, T ′ ) = 1 if the labels of T and T ′ are the same, and 0 otherwise. 2. If either T or T ′ is a leaf node, then θ(T, T ′ ) = 0. 3. Otherwise, let {T 1 , T 2 , . . . , T n } be the child subtrees of T , and {T ′ 1 , T ′ 2 , . . . , T ′ n ′ } be the child subtrees of T ′ . -If the labels of r and r ′ are the same, then θ(T, T ′ ) is the sum of the products of θ(T i , T ′ j ) for all combinations of i and j, where i ranges from 1 to n and j ranges from 1 to n ′ .-If the labels of r and r ′ are different, then θ(T, T ′ ) is 0.-Additionally, if T and T ′ are isomorphic (have the same structure), then θ(T, T ′ ) is incremented by 1.In the paper, the Tree Kernel (TK) score is normalized to accommodate trees of different sizes. The normalized TK score is computed as:T K(T pred ,Tgt)×100T K(T pred ,T pred )T K(Tgt,Tgt). The kernel value serves as a measure of similarity, where higher values indicate greater similarity.• Maximum Common Subgraph (MCS) [54, 33] : The Maximum Common Subgraph (MCS) identifies the largest shared subgraph between two trees, measuring the similarity and overlap of their hierarchical structures. Here's the mathematical definition of the Maximum Common Subgraph:Maximum Common Subgraph: Given two trees,T 1 = (V 1 , E 1 ) and T 2 = (V 2 , E 2 ), where V 1 and V 2 are the sets of vertices and E 1 and E 2 are the sets of edges for each graph, respectively, we define the Maximum Common Subgraph as:MCS(T 1 , T 2 ) = (V MCS , E MCS ) maximize |V MCS | subject to V MCS ⊆ V 1 and V MCS ⊆ V 2 E MCS ⊆ E1 and E MCS ⊆ E2For any pair of vertices u, v in V MCS , if (u, v) is an edge in E MCS , then (u, v) is an edge in T 1 and T 2 , and vice versa.In our paper, we report the normalized MCS score as our measurement of tree similarity|MCS(T pred ,Tgt)|×100 √ |T pred ||Tgt|, where a higher score indicates greater similarity between the graphs.Here, | • | represents the number of nodes in a tree. We employ this normalization to address the scenario where one tree is significantly larger and encompasses all other trees as subtrees.By dividing the MCS score by the square root of the product of the numbers of nodes in the predicted tree (T pred ) and the ground truth tree (T gt ), we ensure a fair comparison across trees of varying sizes.• Tree Edit Distance (TED) [6] : The Tree Edit Distance (TED) quantifies the minimum number of editing operations required to transform one hierarchical tree into another. It measures the structural dissimilarity between trees by considering node and edge modifications, insertions, and deletions. With smaller TED, the two graphs are more similar. Let's define the Tree Edit Distance formally:Tree Edit Distance: Given two trees T 1 and T 2 , represented as rooted, labeled, and ordered trees, we define the Tree Edit Distance between them as T ED(T 1 , T 2 ).The T ED(T 1 , T 2 ) is computed recursively as follows:-If both T 1 and T 2 are empty trees, i.e., they do not have any nodes, then T ED(T 1 , T 2 ) = 0. -If either T 1 or T 2 is an empty tree, i.e., it does not have any nodes, then T ED(T 1 , T 2 ) is the number of nodes in the non-empty tree. -Otherwise, let r 1 and r 2 be the roots of T 1 and T 2 , respectively. Let T The T ED(T 1 , T 2 ) is the final result obtained after applying the above recursive computation.For our experiments, we utilize publicly available pre-trained models. Specifically, we employ CI-FAR10 models from https://github.com/huyvnphan/PyTorch_CIFAR10, CIFAR100 models from https://github.com/chenyaofo/pytorch-cifar-models, and ImageNet models from torchvision package and timm package. The self-supervised models are downloaded from their respective official repositories.To explain visual models using tree-structured language without annotations, we devised four basic baselines, Random, Constant, Subtree and TrDec, for comparison with our LVX method.Random Baseline. The Random baseline generates random explanations by predicting an image's category and randomly sampling 5 nodes from its category-specific tree. The connected nodes form a tree-structured explanation, providing a performance baseline for random guessing.Constant Baseline. The Constant baseline Produces a fixed tree-structured clue for images of the same class, using an initial explanatory tree T (0) i as a template. This baseline assesses LVX against a non-adaptive, static approach. Subtree Baseline. This method involves selecting the most common subtree from the test set for explanations, testing the efficacy of using frequent dataset patterns for generic explanations.TreDec Baseline. Based on [79] , the TrDec strategy implements a tree-topology RNN decoder over an image encoder. In the absence of hierarchical annotations, this baseline uses the CLIP model to verify nodes in the template trees, which act as pseudo-labels for training. This method focuses on the effectiveness of a structured decoding process in explanation generation. This comparison demonstrates the effectiveness of LVX in creating explanations tailored to individual image content, clearly outperforming methods based on random guessing, static templates, and basic learning-based approaches.Our system, LVX, depends on an external Large Language Model (LLM) to provide textual explanations. While this integration adds significant functionality, it also introduces the risk of inaccuracies. The LLM may not always deliver correct information, leading to potential misinformation or erroneous explanations.Additionally, our approach involves generating explanations based on the last embedding layer of the neural network. This method overlooks the comprehensive, multi-level hierarchical structure of deep features, potentially simplifying or omitting important contextual data that could enhance the understanding of the network's decisions.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.In practice, we implement d(q, p) = -log( ||q-p|| 2 +1 ||q-p|| 2 +ϵ ), incorporating ϵ > 0 to ensure numerical stability.https://www.bing.com/images/https://chat.openai.com/https://github.com/mlfoundations/open_clip 


Decision: Originally the paper received mixed reviews, concerns raised included:

1. the loose coupling between how the explanation tree is constructed and the inner workings of the classification model
2. the reliance on only the attributes for the explanation
3. the validity of the metrics
4. general motivation
5. practical value
6. additional baselines needed
7. clarity of presentation
8. insufficient qualitative insights

Following the rebuttal, it seems to the AC that most concerns were addressed as acknowledged by the reviewers, with some concerns remaining on the need for a major revision to improve clarity (rev. UiCA) who raised their score to 4 (still below acceptance). However, as the majority of reviewers recommend acceptance, with UiCA raising their score, the AC believes this paper can still be a valuable contribution to the proceedings. The AC strongly urges the authors to incorporate all the discussions into their revised manuscript, paying special attention to the constructive suggestions of UiCA (as well as all other reviewers ofc).

Reviews: review: paper_summary: paper_summary: This work proposes a method to understand the prediction of an image classification  model using a tree of attributes. The tree of attributes is originally constructed in a text-only by querying gpt for identifying attributes of given concepts. They then use an image-to-test model (or retrieval model) to obtain a set of images corresponding to each attribute. At inference time, they match the input image to each attribute at the current tree level. The path from the root node to the leaf node is meant to interpret the models decision making process.
strength_weakness: strength_weakness: The primary weakness of this work is that the way the tree is constructed is not affected by the image classification model at all but the explanation for the inner working of this model must be a path that exists in the tree. This then presupposes that the tree contains the same reasoning path used by the image classification model which seems to be a large assumption. For example, the model may identify a certain kind of bird by some spurious correlation in the training data. However, as the tree is created by a language model which does not know about this data, this is very unlikely to exist in the tree. However, this method will output some sort of path which is meant to be the explanation even though it is impossible for the tree to produce the correct explanation in this case.

The evaluation metric for measuring the accuracy of the explanation seems poorly defined (see questions sections).

Human eval doesn’t make sense in this case as humans cant evaluate how well a given explanation corresponds to the true decision making process of the model. (they can only measure how well it corresponds to a process they may have used)

Some motivation for this method seems unfounded. For example, the authors state that “Upon observing a dog, we naturally check for the visibility of its tail.” I think this is not correct.
questions: questions: Faithfulness Score: Can you explain more directly how this score measures the inner workings of the model? 

Figure 3 uses the wrong form of “dear”/“deer”
limitations: limitations: yes
paper_summary: paper_summary: The paper suggests a new approach to building a hierarchical tree of visual attributes (represented with language) that matches the decision-making mechanisms of classifiers. The approach is based on using an LLM to suggest a tree of attributes that are related to a specific class, and then refine the tree according to its matches to the training date (this is done by comparing image embeddings of per attribute synthetic generated images and the train images). The hierarchical explanations are then evaluated using three measures (defined by the authors): plausibility, faithfulness, and stability. The tree explanations are shown to be beneficial when fine-tuning the classifier to match better the found structure, which slightly improves performance. The authors also exemplified the usage of the explanation for characterizing miss classification.
strength_weakness: strength_weakness: * My main concern is that unintentionally *the suggested tree representation mainly explains the training set, and not the network mechanism* (in a very complicated way):

(1) Solely using the last layer of the net for the image embedding: Classifiers are known to have a hierarchical structure of attribute representations that are combined to make the classification decision. By looking only at the last layer embedding the author completely disregards that, and the tree representation constructed does not really explain the inner mechanism of the network but only the mechanism of one particular layer.

(2) The refinement stage is made on the training samples; This means that relevant class attribution can be directly extracted by characterizing the leading attribution of the training data, with no connection to the network mechanism itself or any of it embeddings, without any need to synthesize synthetic data. 

This stage, combined with the fact that the image representations are extracted only from the last layer, makes the tree representation mainly visualize the hierarchy of the training set not the hierarchy of the operation of the network. 


* My second main concern is *the usability of this type of visual explanation*. The author provided some experiments for potential applications but these seem limited; Finetuning classifiers to better obey the representation only marginally improves performance compared to baselines (no baseline performance of the model before fine-tuning was provided). Using the tree for misclassification examination seems only anecdotal with no large-scale explanation or evaluation. Can the trees replace the classifiers and be used to classify images? Is it possible to compose the tree to get a model lever tree?



* Does the explanation tree go beyond the structure of the hierarchy provided in context? I can imagine other important attributes, like the presence of singular objects vs many objects, or different types of conditions like the presence of a dog but only if it is leashed. According to the text, it seems like the “subjects” of each node are fixed according to the in-context example, which significantly limits the flexibility of the representation. Furthermore- how was the format in-context example decided? it seems to have a lot of influence on the results but that was not discussed in the paper.

* In the tree refinement stage, there is no definition for what does it to have “nodes that are seldom visited”, this seems to be an important variable. what is the criterion for that?

* Evaluation- 
The faithfulness score seems to directly connect with the way that the tree was defined.
Baseline comparisons are rather ablation study.


* The clarity of text and figures can be improved. Intro is very general and some definitions can be interpreted in many ways, for example, it will be much clearer if the authors state that by in vision model they mean a classifier. Another example is defining the image embedding way before using it which makes it hard to follow details. Also unclear how L_{HMC} is calculated.
The scheme in Fig. 1 was unclear to me. fiuger 5- how were accuracy and MCS calculated here?
questions: questions: see weaknesses
limitations: limitations: limitations are discussed in the appendix
paper_summary: paper_summary: The goal of the paper is to bridge the gap between human comprehension and AI decision. For this purpose, the authors propose a Language Model as Visual Explainer (LVX), an approach to interpret the internal workings of vision models through tree-structured linguistic explanations, without the need for additional model training.  The propose method leverages the collaboration between vision models and large language models (LLM) to generate these explanations. The LLM is used to outline hierarchical visual attributes, while a text-to-image API retrieves images that best match these textual descriptions. By mapping these texts and images to the vision model’s embedding space, they create a hierarchical visual embedding tree. This tree is dynamically adjusted by querying the LLM with language templates, allowing for the addition new attributes and removal of irrelevant concepts based on the model’s representations. Additionally, the authors propose a new benchmark and new metrics to demonstrate the plausibility, faithfulness, and stability of the newly introduced method.
strength_weakness: strength_weakness: Weaknesses:
- some concepts presented in the paper require more details
questions: questions: Here are my concerns:
- Eq. 3: How is the loss function $L_{HMC}$ defined
- Section 4.1 -> Evaluation Metrics -> Plausability: How is the unnormalized TK score defined?
- How does your approach may cope with fine-grained datasets, such as Flowers or CUB-200? I assume that the robustness of the approach relies on the capability of the LLM to cope with fine-grained datasets. Do you think that your approach could handle such case or some modifications (fine-tuning) should be performed on the LLM in order to deal with this particular case. Please provide your insight with respect to this problem.
limitations: limitations: Limitations are related with the curation of the new dataset and imply the following aspects: false positive simages, the presence of out-of-distribution samples and potential bias in the dataset. This research work does not have any negative societal impact.
paper_summary: paper_summary: This paper builds an attribute tree using LLMs to explain image classifiers. To this end, the paper designs a framework that expands and prunes nodes for the tree. Using this framework, the paper provides a richly annotated version of CIFAR and ImageNet, which is 5 times more expressive than prior works. With this attribute tree, the paper outperforms previous decision tree approaches such as NBDT.
strength_weakness: strength_weakness: 1. Explanation is more than an attribute tree

The paper focuses on building attribute trees using LLMs. However, there are more diverse ways to interpret the behaviors of vision models, particularly using language. For example, [1,2] use vision-language models to understand the failures of vision models through language explanations. [3-5] use language to derive concepts for concept bottleneck models (CBMs), though this paper only cites [5]. [6] generates language to explain the neurons in the network. Among these approaches, the paper should discuss the advantages and drawbacks of the proposed method of creating an attribute tree.

For this reason, I believe the current paper title is overstated. I recommend clarifying its scope in the title to focus on the construction of hierarchical attributes and decision trees.

[1] Kim et al. Discovering and Mitigating Visual Biases through Keyword Explanation. CVPR 2024.\
[2] Wiles et al. Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning. arXiv 2023.\
[3] Yuksekgonul et al. Post-hoc Concept Bottleneck Models. ICLR 2023.\
[4] Oikarinen et al. Label-Free Concept Bottleneck Models. ICLR 2023.\
[5] Yang et al. Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification. CVPR 2023.\
[6] Hernandez et al. Natural Language Descriptions of Deep Visual Features. ICLR 2022.


---
2. Insufficient qualitative insights

The paper mostly focuses on quantitative metrics such as plausibility, faithfulness, and stability, which support the validity of the proposed explanation. However, an explanation is only meaningful when it provides new insights to humans. In this sense, the paper should carefully examine individual data points and their corresponding explanations and discuss the lessons learned from them.

I know that Figure 6 provides some examples of decision trees. However, the paper could delve deeper, for example, by addressing: 1) What are the common issues with current classifiers? 2) What are the differences between different models, such as ResNet vs. ViT? As the paper has built a tool, there are numerous directions to explore using it.

---
3. Necessity of enormous attributes

The paper creates datasets with rich attributes (H-XX), which are 5 times more expressive than the prior datasets (DR-XX). However, do we really need all these attributes? More analysis and justification of the attributes should be provided, clearly showing the limitations of prior works.
questions: questions: Can the method be improved using multimodal models like GPT-4V instead of language-only models like GPT-4?
limitations: limitations: Discussed in the Appendix.

Comments: title: Thank Reviewer U6GY for the question
Thank the Feedback from Reviewer U6GY
Acknowledgement of Rebuttal
Response to Reviewer UiCA
Great Appreciation to Reviewer
Response to the Rebuttal
Thank the Reviewer for the Constructive Comments
Thank the Reviewer for the Constructive Comments
comment: Big thanks to Reviewer U6GY for raising the score and acknowledge our efforts! We appreciate your concern and fully understand that our faithfulness score isn't perfect; it involves certain simplifications, particularly in linear measurements over prototype similarity. It matches the classifier’s decision process, but up to the last layer.

Luckily, despite these simplifications, this score can handle the co-occurrent attributes, in the case you raised.

`>>> Q3`**Faithfulness score and Co-occurrence of Attributes**

`>>> A3`That's a great question. In fact, our faithfulness score can handle the co-occurring attributes. It calculates the sum of feature distances, which can be explained as computing the *negative log-likelihood (NLL) of attributes A and B jointly existing* in the image.

- **Probabilistic Rational:** Consider each attribute as an isotropic Gaussian $P(x|i) \sim \mathcal{N}(\mathbf{p}_i, \mathbf{I})$, with its mean at a prototype embedding $\mathbf{p}_i$. When assuming conditional independence among these Gaussians, the joint probability $P(x|1,\dots,N) = \prod_i^N P(x|i)$.

  By calculating the distance between an embedding $\mathbf{q}$ and these prototypes $\sum_i ||\mathbf{p}_i-\mathbf{q}||^2$, we are actually computing the negative log-likelihood that $\mathbf{q}$ belongs to the joint distribution. This can be expressed as:

    $$\sum_{i}^N ||\mathbf{p}_i-\mathbf{q}||^2  \propto  \sum_i^N -\log(P(\mathbf{q}|i)) = -\log(P(\mathbf{q}|1,\dots,N))$$

    Thus, the feature distance reflects the NLL, showing *how likely the sample contains all the attributes simultaneously*. While these assumptions may somewhat be overly strict, in the probabilistic view, our faithfulness score indeed can measure the co-occurrence of attributes.

- **Joint Attribute Data:** Besides the probabilistic rationale, we also deliberately *collect data containing all attributes within a path from root to leaf*. For example, if the support image contains `a furry tail` for a dog, it will, of course, include `a tail`, because it is an ancestor node in the tree. So, given a test image, if multiple attributes on the same path are identified, they must co-exist in the image.

In both sense, our faithfulness metric becomes a good indicator for co-occurring attributes.

--------------
Should you have any further questions or concerns, please let us know and we will try our best to clarify.

Again, thank your for all the suggestions and questions this week—they've really helped us to improve our paper!
We truly thank Reviewer UiCA for raising the score and all the supportive feedbacks. We plan to include all analytic and ablation experiments in the paper and revise the writing to further strengthen our arguments. Here’s a quick plan for our revisions:

`>>> Q5`**Revision Plan**

`>>> R5`
1. For the *experimental part*,

- We'll discuss why using training set attributes doesn't improve explanations (`R1-4`);
- We'll show how removing the training set images worsens results (`A3`) ;
- We’ll add more analysis and insights using LVX to explain the models (`A4`);

We hope to include these updates in the main paper, as the NeurIPS allow one more page for camera-ready submission.

2. For the *writing and figure part*, we realize that the current content is overly dense—lots of information and long descriptions have made some figures too small.

     As suggested, we'll streamline these sections, simplify the logic, and increase the font size for figures for better clarity.

-----------

Should you have any further questions or concerns, please let us know and we will try our best to clarify.

Again, big thanks for all the suggestions—they really help us and this paper a lot!
I still have some concerns over this faithfulness evaluation metric as it does not seem to me to truly match the process of a classifier. For example, it cannot handle a case where a class either has both attribute A and B or neither (but never only one of these attributes). Essentially, I don't think a mathematical argument for a linear classifier on top of a set of attribute applies to this case (of a non linear classifier on top of a complex uninterpretable set of features). 

That being said, the authors make a compelling case over the use of this interpretability method in previous works. For this reason I raise my score to a 5.
Thank you for the additional explanation. I strongly suggest that all these additional explanations will appear in the paper. 
The paper and figures still need some major revisions to improve readability. 
I decided to increase my score to 4.
Thank the reviewer again for your kind and encouraging comments!

Best!
Dear Reviewer U6GY,

We are sincerely thank you for raising the score and your time to review our work. Besides, we still want to clarify some of the points the reviewer raised.

`>>> Q1`**LLM must Generate at some point the True Reasoning**

`>>> R1`Yes, you are right, we somehow assume LLM can eventually generate true reasoning, even if not in the first round.

While this assumption may have limitations, it offers benefits in terms of **human interpretability** and **efficiency**, as shown in `Q3`. We have also attempted to address potential issues through *tree refinement*, which optimizes the reasoning paths that should be explored.

In addition, this assumption—that explanations can be generated by a model—is common. For example, it is used in counterfactual explanations with diffusion models [A, B] and in training LLMs to generate explanations [C].

We acknowledge that this approach has its limitations and will be explored further in our ongoing research.

**References:**
- [A] Jeanneret, G., Simon, L., & Jurie, F. Diffusion models for counterfactual explanations. ACCV 2022.
- [B] Prabhu, Viraj, et al. Lance: Stress-testing visual models by generating language-guided counterfactual images. NeurIPS 2023.
- [C] Hernandez, Evan, et al. Natural language descriptions of deep visual features. ICLR 2021.

`>>> Q2`**Is the Faithfulness Score Really Faithful?**

`>>> R2`Thanks for the nice question. We believe that measuring feature distance inherently provides a faithful explanation of the model's behavior. This is rooted in how classifiers are trained: Deep classifiers are *trained to perform prototype matching*.

- **Mathematical Rationale:**  Consider a feature extractor $g$ with a linear classifier $W$. Deep networks are trained to minimize the classification error $\text{min}_{g, W} L(W \circ g(x), y)$. 

  If we view each column of $W$ as a prototype $W_i$, this objective translates into a prototype matching loss, aligning the sample's embedding $g(x)$ with the prototype $W_y$ of its class $y$: $\text{min}_{g, W} L(W_y, g(x))$
  
   In this way, classifier make prediction based on its feature distance to its prototypes. In return, measuring this distance provide a faithful way to do explanation.

- **Common Practice:** Despite above reason, using feature distance as a measure in prototype-based explanation is a standard practice in literatures [9,10] .

We understand there are other ways to measure the causality or relevance between model decision and attributes. Using feature distance with prototypes is one of them and, of course, not a very bad one. 

-------

Once again, thank the reviewer for the thoughtful feedback and for recognizing the potential of our work.
Thank you to the authors for pointing out that the tree is able to be influenced by the Image Models during the refinement process. I also appreciate the new experiments which demonstrate that this refinement process does lead to a better score.

I also thank the authors for adding citations to support their biological inspiration. 

I do however still hold two of my concerns from the previous round of reviews:

1. Even with the influence of the Vision model during the refinement process, the tree can only be updated by the LLM. This means that the LLM must generate at some point the true reasoning behind the visions models decisions. I think it is not always reasonable to assume that the LLM will do this as the LLM is likely trained on human understanding of concepts which may not always align with the internal working of the model.
2. I still do not believe the faithfulness score truly measured the inner workings of the model. Just because the models internal representation of an image is close to the prototype of a given attribute, does not mean that the model is classifying an image a certain way because of that attribute.

I will raise my score to a 4, as the authors addressed a portion of my concerns. However, I believe this work needs more investigation into if the evaluation metric is actually measuring the models reasoning.
I want to thank the authors for addressing all my concerns.
Dear Reviewer UiCA,

Thank you for your question. We are truly grateful for your time and the opportunity to clarify our approach. 

------

If our understanding is correct, your main concern seems to be why our LVX needs LLM to refine trees, instead of using pre-existing attributes from the training set directly.

To address this, we provide a brief response in `Q1` and an expanded explanation in `Q2`. Additionally, we illustrate the difference with an example in `Q3` and offer a quantitative comparison in `Q4`.

`>>> Q1`**Quick Answer**

`>>> R1`LVX **does not** merely replicate attributes seen in the training set. Instead, it dynamically adjusts attributes—sometimes *incorporating unseen attributes or excluding existing ones in the training set*—based on the classifier's perception.

`>>> Q2`**Detailed Answer**

`>>> R2`We can best clarify this by comparing two methods that focus solely on *examining training attributes* and explaining how they differ from LVX.
 
**Method 1: Human-Annotated Attributes.** 
In this method, each class in the dataset comes with detailed, human-annotated attributes. There is no need for further refinement by an LLM; these attributes are organized into trees directly. We retrieve images for each attributes. For a test image, we match it to these generated image like a nearest-neighbor classifier to determine attributes.

**Method 2: LLM-Generated Fixed Attributes.** 
Here, attributes are initially generated by an LLM based solely on the class name. This method represents the LVX in its initial stage, without refinements.

Compared to LVX, those methods differ mainly in three ways:

1.  **Attributes $\neq$ Explanations:** Unlike `Method 1` and `Method 2`, where attributes are predefined and not influenced by the classifiers, LVX refines these attributes to better explain how the model sees images. This makes LVX not just about predicting attributes but providing model-specific explanations.
2.  **Static vs Dynamic:** `Method 1` and `Method 2` involve a finite set of static attributes. LVX, however, can add or remove attributes based on real-time classifier feedback. It enables open-vocabulary capabilities, as detailed in our paper `Line 108`.
3.  **Annotation cost:** LVX eliminates the need for annotation required by `Method 1`, reducing costs.

`>>> Q3`**Example: LVX Adds Attributes Not Found in Training Data**

`>>> R3`We provide an example where LVX introduces attributes not originally present in the data. In `Fig 6` the top-right example, the `Hook` is mis-classified as `Nematode`. Actually, all images of `Nematode` in ImageNet are  *black and white 2D microscope images* and do not have 3D attributes like **"Cylindrical"**. However, LVX introduces such non-existent attributes to the attribute tree, which helps explain why such misclassifications occur. This is not achievable for `Method 1`.
  
`>>> Q4`**Quantitative Comparison**

`>>> R4`We do an experiment to compare LVX with `Method 1` and `Method 2` in terms of faithfulness and plausibility for the generated trees.

-   **Setup**: Since ImageNet and CIFAR do not have human-annotated attributes, we use the CUB-200 dataset for our experiments. This dataset has 312 binary attributes organized into tree structures, which is ideal for `Method 1`. We compare LVX with `Method 1` (human-annotated attributes) and `Method 2` (LLM-generated attributes without refinement) using the ViT-S classifier. We measure faithfulness using the MSCD score and plausibility using the MCS score.

-   **Results**: The table below shows the performance differences. `Method 1` produces trees that align more with human recognition, indicated by a higher MCS score. However, it does not capture the classifier's internal workings as effectively as LVX, which *achieves a significantly lower faithfulness MSCD score*. `Method 2` performs the worst on both metrics.

| Network | CUB-200 (MCS&#8593;/MSCD&#8595;) | | |
|--|--|--|--|
| |LVX | `Method 1`| `Method 2`|
| ViT-S|28.64/**-1.592** | **29.13**/-0.532 | 26.32/-0.478|

------

Once again, thank the reviewer for the thoughtful feedback and for recognizing the potential in our work. We will incorporate the discussion in our revision. 

Please let me know if you need any further clarification.
Thank you for your detailed reply. I still wonder how the method is different than constructing the tree based on *only* examining the attributes of images of a given class C in the training set (without feed-forwarding them to the network). Can you please clarify that?
Dear Reviewer FVKA,

Truly thank you for the nice words and the recognition of our efforts. We will incorporate all the suggestions to make our work more scientific.

One more thing to double-check is that the author mentioned "stick to my original rating of weak acceptance". But the current rating is baseline acceptance rather than WA(6). Not sure if there is any misunderstanding from our side, any error in the system or R-FVKA just meant it.

Anyway, your suggestions are valuable for us.

Best!
Thank you for the detailed response and experiments. I believe the construction of hierarchical concepts is valuable for XAI, despite concerns about its usefulness raised by other reviewers. Therefore, I stick to my original rating of borderline weak acceptance.

I appreciate the decision to revise the paper title to better clarify the scope; the new title will be more scientific than the previous overly-hyped one.

I also value the new experiments comparing models using their XAI method, which show that current models are biased towards color over shape and size, and that CNNs and ViTs excel in local and global pattern recognition, respectively. While these insights aren't entirely new, it's reassuring to see them align with prior work.
Dear Reviewer UiCA,

We would like to thank you again for your constructive comments and kind effort in reviewing our submission. Please do let us know if our response has addressed your concerns, and we are more than happy to address any further comments.

Thanks!
Dear Reviewer U6GY,

We would like to thank you again for your constructive comments and kind effort in reviewing our submission. Please do let us know if our response has addressed your concerns, and we are more than happy to address any further comments.

Thanks!