Towards Multi-dimensional Explanation Alignment for Medical Classification

Authors: lijie  hu
songning  lai
wenshuo  chen
hongru  xiao
hongbin  lin
lu  yu
jingfeng  zhang
di  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-06 16:15:57

Abstract: The lack of interpretability in the field of medical image analysis has significant ethical and legal implications. Existing interpretable methods in this domain encounter several challenges, including dependency on specific models, difficulties in understanding and visualization, and issues related to efficiency. To address these limitations, we propose a novel framework called Med-MICN (Medical Multi-dimensional Interpretable Concept Network). Med-MICN provides interpretability alignment for various angles, including neural symbolic reasoning, concept semantics, and saliency maps, which are superior to current interpretable methods. Its advantages include high prediction accuracy, interpretability across multiple dimensions, and automation through an end-to-end concept labeling process that reduces the need for extensive human training effort when working with new datasets. To demonstrate the effectiveness and interpretability of Med-MICN, we apply it to four benchmark datasets and compare it with baselines. The results clearly demonstrate the superior performance and interpretability of our Med-MICN.

Main paper: 
 introduction: 
 The field of medical image analysis has witnessed remarkable advancements, especially for the deep learning models. Deep learning models have exhibited exceptional performance in various tasks, such as image recognition and disease diagnosis [31, 46, 1] , with an opaque decision process and intricate network. However, this lack of transparency is particularly problematic in the medical domain, making it challenging for physicians and clinical professionals to trust the predictions made by these deep models. Thus, there is an urgent need for the interpretability of model decisions in the medical domain [43, 13, 57] .The medical field has strict trust requirements. It not only demands high-performing models but also emphasizes comprehensibility and earning the trust of practitioners [20] . Thus, Explainable Artificial Intelligence (XAI) has emerged as a prominent research area in this field. It aims to enhance the transparency and comprehensibility of decision-making processes in deep learning models and large language models by incorporating interpretability [65, 21, 22, 64, 63, 25, 8] . Various methods have been proposed to achieve interpretability, including attention mechanisms [56, 61, 27, 26, 24] , saliency maps [70, 16] , DeepLIFT and Shapley values [38, 4] , influence functions [34, 55] . These methods strive to provide users with visual explanations that shed light on the decision-making process of the model. However, while these post-hoc explanatory methods offer valuable information, there is still a gap between their explanations and the model decisions [42] . Moreover, these post-hoc Figure 1 : Med-MICN demonstrates multidimensional interpretability, encompassing concept score prediction, concept reasoning rules, and saliency maps, achieving alignment within the interpretative framework. The 'Peripheral ground-glass opacities' is c 0 , and along the y-axis, it sequentially becomes c 1 , . . . , c 7 .explanations are generated after the model training and cannot actively contribute to the model fine-tuning process, hindering them from being a faithful explanation tool.Thus, there is increasing interest among researchers in developing self-explanatory methods. Among these, concept-based methods have garnered significant attention [50, 2, 35] . Concept Bottleneck Model (CBM) [35] initially predicts a set of pre-determined intermediate concepts and subsequently utilizes these concepts to make predictions for the final output, which are easily understandable to humans. Concept-based explanations provided by inherently interpretable methods are generally more comprehensible than post-hoc approaches. However, most existing methods treat concept features alone as the determination of the predictions. This approach overlooks the intrinsic feature embeddings present within medical images, thus degrading accuracy [44] . Moreover, while these concepts are human-understandable, they lack semantic meanings, thus questioning the faithfulness of their interpretability [39] . To improve further human trust, several recent works [3] aim to leverage syntactic rule structures to concept embeddings. However, there are still several potential issues. First, unlike CBMs, current concept models with logical rules mainly focus on the supervised concept case, which is quite strict for biomedical images as concept annotation is expensive. Second, while current concept models (with logical rules) provide interpretations via concepts, we found that the importance of these concepts is misaligned with other explanations, especially the explanation given by saliency maps [70, 16] . This will lead to a possible reduction in human trust when using these models.To address these challenges, we introduce a new and innovative end-to-end concept-based framework called the Med-MICN (Medical Multi-dimensional Interpretable Concept Network), as illustrated in Figure 3 . As shown in Figure 2 , Med-MICN is an end-to-end framework that leverages Large Multimodals (LMMs) to generate concept sets and perform auto-annotation for medical images, thereby aligning concept labels with images to overcome the high cost associated with medical concepts annotation. In contrast to typical concept-based models, our interpretation is notably more diverse and precise (shown in Figure 1 ). Specifically, we map the image features extracted by the backbone through a concept encoder to obtain concept prediction and concept embeddings, which are then input into the neural symbolic layers for interpretation. This also establishes alignment between image information and concept embeddings by utilizing a concept encoder, leading to the derivation of predictive concept scores. Furthermore, we align concept semantics with concept embeddings by incorporating neural symbolic layers. Thus, we effectively align image information with concept semantics and concept saliency maps, achieving comprehensive multidimensional alignment. Additionally, unlike most concept-based methods, we use concept embeddings to complement the original image features, which enhances classification accuracy without any post-process. Our main contributions can be summarized as follows:• We proposed an end-to-end framework called Med-MICN, which leverages the strength of different XAI methods such as concept-based models, neural symbolic methods, saliency maps, and concept semantics. Moreover, Med-MICN generates rules and utilizes concept embeddings to complement the intrinsic medical image features, which improves accuracy.• Med-MICN offers an alignment strategy that includes text and image information, saliency maps, and concept semantics. It is model-agnostic and can easily transfer to other models. Our outputs are interpreted in multiple dimensions, including concept prediction, saliency maps, and concept reasoning rules, making it easier for experts to identify and correct errors.• Through extensive experiments on four benchmark datasets, Med-MICN demonstrates superior performance and interpretability compared with other concept-based models and the black-box model baselines. 
related work: 
 Concept Bottleneck Model. The Concept Bottleneck Model (CBM) [35] has emerged as an innovative deep-learning approach for image classification and visual reasoning by incorporating a concept bottleneck layer into deep neural networks. However, CBM faces two significant challenges. Firstly, its performance often falls short of the original models without the concept bottleneck layer, attributed to incomplete information extraction from the original data to bottleneck features. Secondly, CBM extensively depends on meticulous dataset annotation. To solve these problems, researchers have delved into potential solutions. For example, [7] have extended CBM into interactive prediction settings by introducing an interaction policy to determine which concepts to label, ultimately improving the final predictions. Additionally, [41] has addressed the limitations of CBM by proposing a novel framework called Label-free CBM, which offers promising alternatives. Post-hoc Concept Bottleneck models [66] can be applied to various neural networks without compromising model performance, preserving interpretability advantages. Despite much research in the image field [18, 33, 32, 45, 47, 36, 28, 23, 37] , concept-based method for the medical field remains less explored, which requires more precise results and faithful interpretation. [9] used a conceptual alignment deep autoencoder to analyze tongue images representing different body constituent types based on traditional Chinese medicine principles. [35] introduced CBM for osteoarthritis grading and used ten clinical concepts such as joint space narrowing, bone spurs, calcification, etc.However, previous research heavily relies on expert annotation datasets or often focuses solely on concept features to make predictions while overlooking the intrinsic feature embeddings within images. Furthermore, while the concept neural-symbolic model has been explored in the graph domain [3] , its application to images, particularly in the medical domain, has been largely absent. Additionally, our work addresses these gaps by proposing an end-to-end framework with an alignment strategy that leverages various explainable methods, including concept-based models, neural-symbolic methods, saliency maps, and concept semantics, to provide comprehensive solutions to these challenges.Explanation in Medical Image. Research on the interpretability of deep learning in medical image processing provides an effective and interactive approach to enhancing medical knowledge and assisting in disease diagnosis. User studies involving physicians have revealed that doctors often seek explanations to understand AI results, especially when the outcomes are related to their own hypotheses or differential diagnoses [60] . They also turn to explanations to resolve conflicts when their judgments differ from those of AI [6] , thereby enhancing the intelligence of medical models. Previous studies have visualized lesion areas through methods such as heatmaps [59] and attention visualization [12] , aiding in the identification of lesion regions and providing visual evidence. Additionally, utilizing language model-based methods like LLM or LMM to generate medical reports complements the interpretation of model results (ChatCAD [58] , XrayGPT [52] , Med-PaLM [49] ). Saliency maps have emerged as the most common and clinically user-friendly explanation for medical imaging tasks [53, 68, 69] . Recent research underscores the importance of understanding the pivotal features influencing AI predictions, particularly when clinicians must compare AI decisions with their own clinical assessments in cases of decision incongruity [54] . In addition to the image's intrinsic feature recognition, assisted discrimination methods based on concept injection are widely employed in assisted medical diagnosis [35, 7] . Compared to relying solely on self-supervised training, conceptual feature-based supplementation integrates expert knowledge, offering more accurate assistance for interpreting detection results.However, previous research on medical images relies on single-dimensional explanations, potentially lacking sufficient decision information for physicians. Furthermore, erroneous single-dimensional explanations could significantly impact physicians' judgments. Thus, there is a pressing need for a multi-dimensional explanatory framework where explanations across various dimensions complement each other. In instances of incorrect explanations, physicians can turn to explanations in alternative dimensions to aid their judgment. We measure this with cosine similarity in a text embedding space. We removed concepts too similar to classes or each other. The former conflicts with our interpretability goals, while the latter leads to redundancy in concepts. We set the thresholds for these two filters at 0.85 and 0.9, respectively, ensuring that their similarities are below our threshold.3. Remove concepts we cannot project accurately: Remove neurons that are not interpretable from the BioViL [5] . This step is actually described in section 4.2.We give the details with examples for neural-symbolic layer [3] .Concept Polarity. For each prediction class j, there exists a neural network Φ j (•). This network takes each concept embedding as input and produces a soft indicator, a scalar value in the [0, 1] range. This soft indicator represents the role of the concept within the formula. As an illustration, consider a specific concept like "Crazy-paving pattern". If its value after passing through Φ j (•) is 0.8, it indicates that the "Crazy-paving pattern" has a positive role with a score of 0.8 for class j. We use the notation I o,i,j to represent the soft indicator for concept c i .Concept Relevance. For each prediction class j, a neural network Ψ j (•) is utilized. This network takes each concept embedding as input and produces a soft indicator, a scalar value within the range of [0, 1], representing the relevance score within the formula. To illustrate, let us consider a specific concept such as "Multilobar distribution". If its value after passing through Ψ j (•) is 0.2, it implies that the relevance score of "Multilobar distribution" in the inference of class j is 0.2. We denote the soft indicator for concept c i as I r,i,j .Prediction via Concept Rules. Finally, for each class j, we combine the previous concept polarity vector I o,j and the concept relevance vector I r,j to obtain the logical inference output. This is achieved by the following expression:ŷj = ∧ N i=1 (¬I o,i,j ∨ I r,i,j ) = min i∈[N ] {max{1 -I o,i,j , I r,i,j }},Intuitively, the aforementioned concept reasoning rules allow us to determine the relevance of each concept and whether a concept has a positive or negative role in predicting the label as class j. Intuitively ¬I o,i,j ∨ I r,i,j means if it is irrelevant, then we set it to 1, and if it is relevant, then we set it to be the relevance score. It is strange at first glance. However, since finally we will take the conjunction (or minimum) for all concepts to get ŷj , so we will filter the i-th concept if ¬I o,i,j ∨ I r,i,j = 1, i.e., if it is irrelevant. Thus, we take the neg here. Since we need to consider each concept for the final prediction, we finally use ∧ for all concepts.Our model exhibits remarkable efficiency in its training process. We utilized only a single GeForce RTX 4090 GPU, and the training duration did not exceed half an hour. We configured the model to run for 100 epochs with a learning rate set at 5e-5. Additionally, all images were resized to a uniform dimension of (256, 256).COVID-CT. The COVID-CT dataset was obtained from [29] and comprises 746 CT images, consisting of two classes (349 COVID and 397 NonCOVID). We divided this dataset into a training set and a test set with an 8:2 ratio, and the data were split accordingly. DDI. DDI [10] is a dataset comprising diverse dermatology images designed to assess the model's ability to correctly identify skin diseases. It consists of a total of 656 images, including 485 benign lesion images and 171 malignant lesion images. These images are divided into training and test sets, with an 80% and 20% split, respectively.Chest X-Ray. The Chest X-Ray [14] dataset comprises 2D chest X-ray images of both healthy and infected populations. It aims to support researchers in developing artificial intelligence models capable of distinguishing between chest X-ray images of healthy individuals and those with infections. The dataset includes 5933 images, divided into 5309 training images and 624 testing images.Fitzpatrick17k. Fitzpatrick17k [15] dataset is a dermatological dataset that includes a wide range of skin colors. In order to better compare the model performance, we filtered the malignant and nonmalignant classes in 3230 images that have been relabeled by SkinCon [11] . And we divided the training and test set according to an 8:2 ratio.SSSD-COVID. SSSD-COVID incorporates a Masked Autoencoder (MAE) for direct pre-training and fine-tuning on a small-scale target dataset. It leverages self-supervised learning and selfdistillation techniques for COVID-19 medical image classification, achieving performance levels surpassing many baseline models. Our method is trained exclusively on the COVID-CT dataset without considering the effects of introducing knowledge from other datasets. Notably, our approach outperforms SSSD-COVID in terms of performance. Furthermore, SSSD-COVID falls under the category of black-box models, indicating that our method, to some extent, overcomes the accuracy degradation issue introduced by the incorporation of concepts.Label-free CBM. Label-free CBM is a fully automated and scalable method for generating concept bottleneck models. It has demonstrated outstanding performance on datasets like ImageNet. In our comparisons, our model outperforms this model significantly in terms of accuracy. Regarding interpretability, our model not only possesses the same level of interpretability as this model but also provides explanations in multiple dimensions, such as concept reasoning and saliency maps. This helps doctors gain a more diverse and precise understanding of the model's decision-making process when using our model. DCR. Deep Concept Reasoner (DCR), a concept-based model combining neural-symbolic methods, has demonstrated promising interpretability performance on simple datasets such as XOR, Trigonometry, and MNIST-Addition. However, its performance tends to degrade on more complex or challenging datasets.Accuracy is the ratio of the number of correctly categorized samples to the total number of samples:Acc = TP + TN TP + TN + FP + FN .Where TP denotes true positive, TN denotes true negative, FP represents false negative, and FN represents false negative.The precision is the proportion of all samples classified as positive categories that are actually positive categories:Precision = TP TP + FP .Recall is the proportion of samples that are correctly categorized as positive out of all samples that are actually positive categories:Recall = TP TP + FN .The F1 score is the reconciled mean of precision and recall:F1 = 2 × Precision × Recall Precision + Recall .AUC denotes the area under the ROC curve, which is a curve with True Positive Rate (Recall Rate) as the vertical axis and False Positive Rate (False Positive Rate) as the horizontal axis.We provide our detailed utility evaluation for four datasets in 4 : Results for COVID-CT. We conducted ten repeated experiments and calculated the average results for each metric. Red and blue indicate the best and the second-best result. Our approach outperforms the baseline backbone models while also providing interpretability. When compared to other state-of-the-art interpretable models, our model exhibits significantly higher accuracy.More samples of instance-level predictions for COVID-CT and DDI datasets are visualized in Figure 5 , 6, 7, 8, 9, 10, 11, and 12. We also append the corresponding rules to it, which can reflect the multi-dimensional interpretation better. Taking the COVID-CT sample as an example, it can be found that the model predicts the concept score prediction accurately, and for the COVID samples, the correlation of the first three concepts is greater, which leads to a higher prediction score of their concepts, thus generating real concept rules, and then assists the model judgment. This can also be reflected in the saliency map of the model in the inference stage, where the phenomena described by the relevant concepts can get higher attention in the saliency map, thus further reflecting the multidimensional interpretation of Med-MICN.In this section, we will discuss how each step in our proposed concept filtering affects the results of our method. In general, our utilization of filters has two main goals: First, improving the interpretability of our models. Second, improving computational efficiency and complexity by reducing the number of concepts.Although our initial goal was not to improve model accuracy (a model with more concepts is generally larger and more powerful [41] ), the more precise concepts after filtering make the concatenated features more effective for classification and slightly improve the accuracy. To evaluate the impact of each filter, we trained our models separately on COVID-CT and DDI while removing one filter at a time and one without using any filter at all. The results are shown in the Table 8 . From Table 8 , it is noticeable that our model's accuracy does not exhibit high sensitivity to the choice of filters.On the COVID-CT dataset, the accuracy of our model remains largely unaffected by the choice of filters. Furthermore, employing filters on the DDI dataset results in improved model accuracy. This phenomenon arises due to the relatively sparse nature of concepts within the DDI dataset images, where increasing the number of concepts did not yield superior solutions. 14 , the results show that our model consists of stronger robustness against perturbation. This is evidenced by the marginal decrease in test accuracy as the attack radius increases. The model's detection performance fluctuates slightly at ρ a of 4/255, with this variability diminishing as the perturbation level rises, underscoring the robustness of our model against significant disturbances.We conducted a computational cost analysis for Med-MICN and the baseline models. By inputting a random tensor of size (1, 3, 244, 224) into the model and computing the FLOPs and parameters, the results are presented in Table 9 . Experimental evidence demonstrates that Med-MICN incurs only negligible computational cost compared to the baseline models while it achieves improvements in accuracy and interpretability. In the case of the "neural symbolic" method, additional acceleration techniques may be required when dealing with large sample sizes. However, it is worth noting that medical datasets tend to be relatively small.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The main claims presented in the abstract and introduction accurately reflect the paper's contributions and scope. We proposed an end-to-end framework called Med-MICN, which leverages the strengths of various XAI methods such as concept-based models, neural symbolic methods, saliency maps, and concept semantics. In Section 1 and Abstract. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: in Section E. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries: 
 Concept Bottleneck Models. To introduce the original Concept Bottleneck Models, we adopt the notations used by [35] . We consider a classification task with a concept set denoted as C = C 1 , C 2 , . . . , C N and a training dataset represented as {(x i , y i , c i )} M i=1 . Here, for i ∈ [M ], x i ∈ R d represents the feature vector, y i ∈ R dz denotes the label (with d z corresponding to the number of classes), and c i ∈ R dc represents the concept vector. In this context, the j-th entry of c i represents the weight of the concept p j . In CBMs, our goal is to learn two representations: one that transforms the input space to the concept space, denoted as g : R d → R dc , and another that maps the concept space to the prediction space, denoted as f : R dc → R dz . For any input x, we aim to ensure that its predicted concept vector ĉ = g(x) and prediction ŷ = f (g(x)) are close to their underlying counterparts, thus capturing the essence of the original CBMs.Fuzzy Logic Rules. As described by [17, 3] , continuous fuzzy logic extends upon traditional Boolean logic by introducing a more nuanced approach to truth values. Rather than being confined to the discrete values of either 0 or 1, truth values are represented as degrees within the continuous range of {0, 1}. Conventional Boolean connectives including t-norm ∧ :[0, 1] × [0, 1] → [0, 1], t-conorm ∨ : [0, 1]×[0, 1] → [0, 1], negation ¬x = 1-x.The logical connectives, including ¬, ∨, ∧, ⇒, ⇐, ⇔, are utilized to convey the logical relationships between concepts and their representations. For example, consider the problem of deciding whether an X-ray lung image has COVID, given the vocabulary of concepts "ground-glass opacities (GO)," "Localized or diffuse presentation (LDP)," and "lobar consolidation (LC)." A simple decision rule can be y ⇔ c GO ∧ ¬c LC . From this rule, we can deduce that (1) Having both "no LC" and "GO" is relevant to having COVID. (2) Having LDP is irrelevant to deciding whether COVID exists. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: Code [Yes] DATA [No] Justification:We will soon open-source the code, and the datasets used are publicly available for download by anyone. In Section C and Section 5.1.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
medical multi-dimensional interpretable concept network: 
 Here, we present Med-MICN (Figure 3 ), a novel framework that constructs a model in an automated, interpretable, and efficient manner. (i) In traditional CBMs, the concept set is typically generated through annotations by human experts. When there is no concept set and concept labels, we first introduce the automated concept labeling alignment process (Figure 2 ). (ii) Then, the concept set (output by LLMs such as GPT4-V) is fed into the text encoder to obtain word embedding vectors. Our method utilizes Vision-Language Models (VLMs) to encode the images and calculate cosine distances to generate heatmaps. We apply average pooling to these heatmaps to obtain a similarity score aligned with the concept set through a threshold to obtain concept labels. (iii) Next, we extract image features using a feature extraction network and then map them through a concept encoder to obtain concept embeddings. (iv) We finally use these concept embeddings as input into the neural symbolic layers to generate concept reasoning rules and incorporate them as complementary features to the intrinsic spatial features for predictions, proving multi-dimensional interpretation alignment. We provide details for each component of Med-MICN as follows.Given M c classes of target diseases or pathology, the first step of our paradigm is to acquire a set of useful concepts related to the classes. A typical workflow in the medical domain is to seek help from experts. Inspired by the recent work, which suggests that instruction-following large language models present a new alternative to automatically generate concepts throughout the entire process [41, 40] . We propose to generate a concept set using LMMs, such as GPT-4V, which has extensive domain knowledge in both visual and language, to identify the crucial concepts for medical classification. Figure 2 (a) illustrates the framework for concept set generation. Details are in Appendix A.Generating Concept Heatmaps. Suppose we have a set of N useful concepts C = {C 1 , C 2 , . . . , C N } obtained from GPT-4V. Then, the next step is to generate pseudo-(concept) labels for each image in the dataset corresponding to its concept set. Inspired by the satisfactory performance in concept recognition within the medical field demonstrated by VLMs [62] , we use BioViL [5] to generate the pseudo-labels for the concepts of each image. Figure 2 (b) illustrates the detailed automatic annotation process of concepts.Given an image x and a concept set, its feature map V ∈ R H×W ×D and the text embedding t i ∈ R D for each concept are extracted as follows:V = Θ V (x), t i = Θ T (c i ), i = 1, . . . , Nwhere Θ V and Θ T are the visual and text encoders, t i is the embedding of the i-th concept in the concept pool, H and W are the height and width of the feature map.Given V and t i , we can obtain a heatmap P i , i.e., a similarity matrix that measures the similarity between the concept and the image can be obtained by computing their cosine distance:P h,w,i = t T i V h,w ||t i || • ||V h,w || , h = 1, . . . , H, w = 1, . . . , Wwhere h, w are the h-th and w-th positions in the heatmaps, and P h,w,i represents a local similarity score between V and t i . Then, we derived heatmaps for all concepts, denoted as {P 1 , P 2 , . . . , P N }.Calculating Concept Scores. As average pooling performs better in downstream medical classification tasks [62] , we apply average pooling to the heatmaps to deduce the connection between the image and concepts:s i = 1 H•W H h=1 W w=1 H h,w,i .Intuitively, s i is the refined similarity score between the image and concept c i . Thus, a concept vector e can be obtained, representing the similarity between an image input x and a set of concepts: e = (s 1 , . . . , s N ) T .Alignment of Image and Concept Labels. To align images with concept labels, we determine the presence of a concept attribute in an image based on a threshold value derived from an experiment. If the value s i exceeds this threshold, we consider the image to possess that specific concept attribute and set the concept label to be True. We can obtain concept labels for all images c = {c 1 , . . . , c M }, where C i ∈ {0, 1} N is the concept label for the i-th sample. Finally, to ensure the truthfulness of concepts, we discard all concepts for which the similarity across all images is below 0.45. To achieve higher annotation accuracy, we only annotated 20% of the data for fine-tuning the model to adapt to different datasets. We sampled 10% of the pseudo-labels generated and compared them with expert annotations [11] . It is notable that in the case where the concept set and concept labels are given, we can directly skip Section 4.1 and 4.2.In Section 4.2, we get the concept labels for all input images. However, as we mentioned in the introduction, such representation might significantly degrade task accuracy [39, 67] . To overcome this issue, recently [67] propose using concept embeddings, which increase the task accuracy of concept-based models while weakening their interpretability. Motivated by this, we use these concept embeddings to increase the accuracy and leverage our concept label to enhance interpretability. In the following, we provide details.X = {(x m , y m )} M m=1 , we use a backbone network (e.g., ResNet50) to extract features F = {f (x m )} M m=1 .Then, for each feature, it passes through a concept encoder [67] to obtain feature embeddings f c (x m ) and concept embeddings ĉm . The specific process can be represented by the following expression:f (x m ) = Θ b (x m ), f c (x m ), Ĉm = Θ c (f (x m )) for m ∈ [M ],where Θ b and Θ c represent the backbone and concept encoder, respectively.To enhance the interpretability of concept embeddings, we utilize binary cross-entropy to optimize the accuracy of concept extraction by computing L c based on ĉ = {ĉ m } M m=1 and concept labels c in Section 4.2:EQUATIONNeural-Symbolic Layer. Our next goal is to use concept embedding to learn concept rules for prediction, which is motivated by [3] . We aim to generate rules involving the utilization of two sets of feed-forward neural networks: Φ(•) and Ψ(•). The output of Φ(•) signifies the assigned role of each concept, determining whether it is positive or negative (such as "no LC" and "GO"). On the other hand, the output of Ψ(•) represents the relevance of each concept, indicating whether it is useful within the context of the sample feature (such as "LDP"). The overall process can be divided into three distinct parts: (i) Learning the role (positive or negative) of each concept called Concept Polarity. For each prediction class j, there exists a neural network Φ j (•). This network takes each concept embedding as input and produces a soft indicator, a scalar value in the [0, 1] range. This soft indicator represents the role of the concept within the formula; (ii) Learning the relevance of each concept called Concept Relevance. Similar to concept polarity, for each prediction class j, a neural network Ψ j (•) is utilized; (iii) Output the logical reasoning rules of the concept and the contribution scores of concepts. For each class j, we combine the previous concept polarity vector I o,j and the concept relevance vector I r,j to obtain the logical inference output. This is achieved by the following expression (Details are in Appendix B):EQUATIONIn this section, we will discuss how we derive the class of medical images and the process of network optimization. First, we have the loss L c in (1) for enhancing the interpretability of concept embeddings. Also, as the concept embeddings are input into the neural-symbolic layer to output logical reasoning rules of the concept and prediction ŷneural,m in (2) for x m , we also have a loss between the predictions given by concept rule and ground truth, which corresponds to the interpretability of our neural-symbolic layers. In the context of binary classification tasks, we employ binary cross-entropy (BCE) as our loss function. For multi-class classification tasks, we use crossentropy as the measure. Using binary classification as an example, we calculate the loss L neural by comparing the output ŷ from the neural-symbolic layer to the label y as follows:EQUATIONClassification Loss. Note that as L neural in ( 3) is purely dependent on the concept rules rather than feature embeddings, we still need a loss for final prediction performance. In a typical classification network, the process involves obtaining the feature f (x m ) and passing it through a classification head to generate the classification results. What sets our approach apart is that we fuse the previously extracted f c (x m ) with the f (x m ) using a fusion module as input to the classification ahead. This can be expressed using the following formula:ỹm = W F • Concat(f (x m ), f c (x m )),Note that W F represents a fully connected neural network. For training our classification model, we use categorical cross-entropy loss, which is defined as follows:L task = CE(ỹ, y),Formally, the overall loss function of our approach can be formulated as:L = L task + λ 1 • L c + λ 2 • L neural ,where λ 1 , λ 2 are hyperparameters for the trade-off between interpretability and accuracy. 
experiment: 
 In this section, we introduce the experimental settings, present our superior performance, and showcase the interpretability of our network. Due to the space limit, additional experimental details and results are in the appendix C.Datasets. We consider four benchmark medical datasets: COVID-CT [29] for CT images, DDI [10] for dermatology images, Chest X-Ray [14] , and Fitzpatrick17k [15] for a dermatological dataset with skin colors.Baselines. We compared our model with other state-of-the-art interpretable models, such as Label-free CBM [41] and DCR [3] , to highlight the robustness of our interpretability capabilities. Furthermore, we conducted comparisons with black-box models, such as SSSD-COVID [51] .Evaluation Metrics. In this study, we concentrate on medical image classification. To comprehensively evaluate our classification model, we employ a range of key metrics, including binary accuracy, precision, recall, F1-Score, and AUC value. Specifically, we calculate each metric for positive and negative samples in each dataset, alongside utilizing AUC for further insight into classification performance.Experimental Setup. We employed three different models as our backbones: ResNet-50 [19] , VGG19 [48] , and DenseNet169 [30] . To demonstrate the superiority of our method, we use the same backbones without any additional processing. For the concept encoder, we utilized the same structure as described in [35] . Before training, we obtained pseudo-labels for the concepts in each image using the automatic labeling process. Although previous work has shown that operations like super-resolution reconstruction can improve model accuracy on our dataset, to demonstrate the superiority of our method, we used the original images as our input. During training, we adopted the Adam optimizer with a learning rate of 5e-5 throughout the training stages. For hyperparameter selection, we set λ 1 = λ 2 = 0.1. 1 : Utility results. We conducted ten repeated experiments and calculated the average results for each metric. Red and blue indicate the best and the second-best result. Our approach outperforms the baseline backbone models while also providing interpretability. Our model exhibits significantly higher accuracy compared to other state-of-the-art interpretable models.Med-MICN delivers superior performance. In Table 1 , our method achieved different improvements on various backbones for the COVID-CT dataset. Taking Acc as an example, it increased by 3.39% with ResNet50 and by 3.45% with VGG compared to backbones. Compared to SSSD-COVID, our method outperforms in terms of Acc and F1, with improvements of 4.68% and 7.15%, demonstrating the superiority of our method in enhancing model accuracy. Additionally, our method possesses interpretability, which is not achievable by SSSD-COVID. On the other hand, our method achieved significant improvements on different backbones for the DDI dataset. For instance, Acc increased by 6.01% with VGG. Despite the significant differences between the two datasets in terms of modality, our method demonstrated significant effects on both datasets, indicating its good generalizability. Details are shown in Table 4 , 5, 6, and 7 in Appendix.Meanwhile, when compared to existing well-performing interpretable models, our approach demonstrates significant advantages in terms of accuracy and other metrics across different datasets. This indicates that our joint prediction of image categories using both concept and image feature spaces outperforms predictions based solely on a limited concept space.Explanation across multiple dimensions. In our approach, we discover and generate concept reasoning rules based on the neural-symbolic layer. Analyzing these rules enhances the interpretability of our network. In addition, our approach derives concept prediction scores through the concept encoder, and during evaluation, it also produces saliency maps. Through multi-dimensional explanations, we can observe the basis for the model decision from different perspectives. In concept score prediction, we can observe how the model maps data to concept dimensions, allowing doctors to observe and correct concept results, thereby rectifying prediction errors caused by incorrect concept predictions.In concept reasoning rules, we can deduce the model classification criteria, further explaining the model decisions. Additionally, during the evaluation process, we can generate saliency maps for the As shown in Figure 4 , it is evident that relying solely on single-dimensional interpretable strategies, such as saliency maps or concept embedding enhancements, does not furnish adequate interpretability to effectively address the problem. However, by integrating multi-dimensional strategies, the model can align the information of each dimension, thus obtaining more comprehensive interpretable information and ultimately yielding more correct prediction results. Specifically, when feature extraction is solely reliant on saliency maps, obtaining accurate attention in the feature region often proves challenging, and conceptual information tends to be unstable when supplemented solely by concepts. In contrast, our proposed multi-dimensional interpretable strategy transcends dependence on a single interpretable strategy, opting instead for a more generalized multi-dimensional augmentation approach. This approach enables the model to complement the single-dimensional methods and achieve heightened accuracy.The ablation experiments presented in A.1 Concept Generation with GPT-4VIn this section, we will explain our process for generating concept sets using GPT-4V. Specifically, we ask GPT-4V the following:[User]:• List the useful visual attributes to recognize {class} in a {type} in this {image}:• List the things most commonly seen around a {class} in this {image}:• List in bullet points with no extra context.Note that the image here refers to the images we random sample from the dataset, and type represents the modality of medical image (e.g. X-rays, CT). Full prompts and example outputs are illustrated in the appendix D.2. Combining the concepts received from different classes and images gives us a large, somewhat noisy set of initial concepts, which we further improve by filtering. We found using GPT-4V to generate initial concepts to perform better than using GPT-3, which was used in Label-free CBM [41] . We initiate the procedure with the following conversation:[User]: List the useful visual attributes to recognize COVID in a CT in this image.[GPT-4V]: When assessing CT images for signs of COVID-19, radiologists often look for certain visual attributes that can suggest the presence of the disease. Here are some of the key visual attributes they may look for: 1. Ground-glass opacities (GGOs): These are hazy areas that do not obscure the underlying lung structures. They are called "ground-glass" because they resemble the appearance of frosted glass. 2. Consolidation: ...[User]: List the things most commonly seen around COVID in this image.[GPT-4V]: Based on the image you have provided and considering the common radiological findings associated with COVID-19, here are the things that might be commonly seen:1. Possible consolidation areas. 2. Absence of significant pleural effusions. 3. ...... Then, we can use the following inquiry to structure the output from GPT-4V:[User]: List in bullet points with no extra context.[GPT-4V]:• Ground-Glass Opacities (GGOs) -Cyst Table 3 : We have recorded the concept set generated through GPT-4V and subsequently cleaned it through filtering. This concept set will be used in our training process.After concept set generation, a concept set with some noise can be obtained. The following filters are set to enhance the quality of the concept :1. The length of concept: To keep concepts simple and avoid unnecessary complication, We remove concepts longer than 30 characters in length. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: All detailed specifications are presented in the experimental section. In Section C and Section 5.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We conducted ten repeated experiments and calculated the average results for each metric. In Section C and Section 5.1.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: We pointed out the positive role our model will play in assisting doctors in making decisions regarding diseases in the future. In Section 1 and Section E. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: Yes, we have cited the original papers that produced the code packages or datasets, provided the version information and URLs when possible, and included the name of the license for each asset. In Section C and Section 5.1.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: After discussion, all reviewers agree to accept this paper. 
The authors are encouraged to further improve this work by addressing feedback from the reviewers, in particular, the following:

1. Add evaluation of the generated heatmaps
2. Compare the interpretability between the proposed method and the baseline methods
3. Improve the notations

Reviews: review: paper_summary: paper_summary: The authors proposed a novel interpretable model called Med-MICN in this work. This method manages medical image classification tasks with multi-dimensional aspects with neural symbolic and concept semantics. With the help of LLMs, the work performed superior on four medical benchmark datasets. In the ablation study, the authors presented that the performance of the proposed method performed better with the complementary multi-dimensional loss functions in evaluating the classification result.
strength_weakness: strength_weakness: - The authors show that the proposed method performed superior to the baseline methods in image classification (Tab. 1). However, there is a missing part on the evaluation of the interpretability. How correct are those heatmaps generated from the proposed model, do they align with the labels or the clinicians?

- Also the comparison of the interpretability between the proposed method and the baseline methods is missing. In Tab. 1, the authors mentioned those methods are interpretable, but they did not evaluate the interpretability. This could be crucial for medical applications.

- For section 4.3, the description of Neural-Symbolic Layer is not easy to follow. What is the difference between the functions of "Concept Polarity" and "Concept Relevance"? It seems like their outputs are the same.
questions: questions: - The authors used LLM (GPT-4V) in their proposed method. Would it be better or make a difference, if the LLM is a more medical-specific model?

- How do authors deal with different numbers of concept sets from the LLM? If I understood correctly, the LLM could output several different concepts and does not necessarily fit into the input size of the text encoder. (Figure 2.a)

- Did authors try with different $\lambda_1$ and $\lambda_2$?
limitations: limitations: - The proposed method provides an opportunity to interpret the medical image classification results and in the meanwhile also increase the classification performance. However, heatmaps look somehow blurry and may not be trustable (Fig 1).
paper_summary: paper_summary: This work introduces Med-MICN, an explainable framework for medical image classification. This framework leverages a concept bottleneck framework combined with a neural symbolic reasoning framework to generate simple explanations for its predictions. In general, strong performance is demonstrated.
strength_weakness: strength_weakness: In general, the notation used in the paper is inconsistent and somewhat ill defined. For example, in section 3, N is used to denote the number of sample in a dataset, and k the number of concepts. In section 4, N becomes the number of concepts, and M the number of samples. Lower case k is then used to index along the height dimension of a feature map. Several objects ($I_{o, j}$ and $I_{r,j}$, for example) are not clearly described upon their introduction.

There are a few seemingly key hyper parameters ($\lambda_1$, $\lambda_2$, the threshold used for binarizing concept vectors), but the experimental details do not indicate that there was a validation/development set used to select these values.

If these two points can be addressed, I believe this paper will warrant acceptance.
questions: questions: There appear to be some notational issues in section 4.2 — initially H and W are used as the height and width of the feature map, but in the definition of $H_{p,k,i}$, it seems that P and K are used to fill the same roll.

“to ensure the truthfulness of 196 concepts, we discard all concepts for which the similarity across all images is below 0.45. “ — does this refer to all training images?

The equation following line 214 is quite confusing — what is the dimensionality of each of the three values returned by $\theta_c$? How does it return $f(x_m)$?

During training, which components are trained jointly? It seems like $L_{neural}$ only applies to the neural symbolic component, while $L_c$ and $L_{task}$ apply only to the concept extraction component and some additional classification fully connected layer.

Figure 4 is somewhat unclear — which cells in the figure should be compared to which?
limitations: limitations: Discussion of limitations is appropriate.
paper_summary: paper_summary: This paper focuses on the explainable classification of medical images with multi-dimensional explanation. The proposed Med-MICN framework contains four modules including feature extraction, auto-annotation, concept embedding, and neural symbolic layer. The incorporation of fuzzy logic rules is novel in the interpretation study of deep neural networks. The experimental results show that Med-MICN surpasses the baseline networks such as vgg and resent, and also outperforms concept bottleneck models (CBM). The generated attention maps and predicted concepts match better than traditional CBM.
strength_weakness: strength_weakness: 1. The whole presentation of the paper should be further improved including mathematical symbols, figures 2 and 3, and equations. The symbols should be better matched with Figures 2 and 3 to improve readability. It is difficult to understand these figures without reading the contents. Some symbols like Cosine should be given in the figures. Detailed notations are needed in the figures or in the captions.
2. It is better to give pseudo codes of training and inferences to help understand the whole workflow.
3. At the bottom of Page 3, "In instances ... judgment", why do the authors claim alternative explanations can aid judgment? Any references? More explanations may confuse the physicians.
4. In line 131, does $y_{i}$ the one-hot labels?
5. In the Concept labeling alignment module, it is not clear why the authors use "average pooling to heatmaps". I think this will lose some spatial localizations of lesions, for example, one heatmap with multiple lesions and another with one lesion, both of them may have similar average pooling results.
6. Under the line 183, the $V$ is three-dimensional, and why here it has two subscripts.
7. In line 194, does the representation be $c \in \{C_{1}, ..., C_{M}\}$ ?
8. In line 142, the logic symbols should be defined may be in supplementary.
9. Lack of training details. In line 277, how the learning rate decay? Any data augmentations were used?
questions: questions: See weakness.
limitations: limitations: The whole framework of Med-MICN is complex, and the so-called multi-dimensional explanations are attention maps and predicted concepts which have been proposed in traditional CBM models.
paper_summary: paper_summary: This work introduces a novel end-to-end concept-based framework called Med-MICN, which is quite inspiring and important for the next XAI era due to its multi-dimensional, powerful interpretability, and efficient performance. Furthermore, they propose an automated process for efficiently and accurately obtaining concept labels, which are costly to acquire in reality.
strength_weakness: strength_weakness: 1. The caption of the picture is not detailed enough, as shown in figure 3.

2. The author considers interpretability on the basis of concept embedding, neural-sybolic reasoning and saliency map. The author lacks the reason why it is necessary to analyze the interpretation from these aspects and whether there are other more dimensions.
questions: questions: See weeknesses.
limitations: limitations: NA

Comments: title: Response to Rebuttal
Discussion
Thanks for your responses
comment: We sincerely appreciate your positive feedback on our rebuttal, as well as the increase in your rating. Your valuable suggestions have greatly improved the experimental completeness and overall readability of our work.
We sincerely appreciate your prompt and positive feedback on our rebuttal, as well as the increase in your rating. Your valuable suggestions have greatly improved the clarity and overall readability of our work.
Thanks for your detailed rebuttal. All of my concerns have been addressed, and I will increase my score.
Thanks for your detailed responses and valuable new experiments. I think it's a solid work and raised my score to 8.
I appreciate the authors' effort in answering my concerns. My concerns are majorly well addressed. I have increased my score accordingly.
1.  What is the concept prediction accuracy in this model？

The questions you raised have been very insightful. We used our previous work to perform concept labeling on the DDI dataset. We compared our predicted concepts with the ground truth to determine the accuracy of different backbones, as shown in the table below.

| Backbone | RN50  | VGG   | DenseNet |
|----------|-------|-------|----------|
| ACC      | 87.12 | 92.13 | 92.83    |

As can be seen, the concept accuracy predicted by Med-MICN is relatively high, indicating better reliability for the concepts it explains.

2. What is the model's robustness against perturbation? This point is also crucial for practical applications in the medical field.

We conducted stability experiments by applying Gaussian noise with δ=0.1 to the images. We compared the saliency maps generated by our method to those produced by the baseline backbone. Our results show that our method maintains greater stability in focusing on the regions of interest compared to the baseline. Due to the rebuttal policy, we cannot present these results, bu We will present images in the later version.
Dear Reviewer #WJ93,

Many thanks for taking the time to review our paper. We have provided our responses to your comments. Can we know whether our responses have addressed your concerns? We are more than happy to hear from you :)


Best wishes,
Authors of the paper #2244
We sincerely appreciate your prompt and positive response to our rebuttal, along with the improved rating. We are currently conducting experiments to address your additional questions. Thank you for your time!
We are profoundly grateful for your timely and positive responses to our rebuttal and the uplift in your rating. Your valuable suggestions have improved the readability of our work. We will make a clearer visual separation between areas of the figure corresponding to different methods.
I appreciate the author's effort to answer my questions. My concerns are well addressed and I will update my score. Btw, I personally have two minor additional questions:

1. What is the concept prediction accuracy in this model？

2. What is the model's robustness against perturbation? This point is also crucial for practical applications in the medical field.
Thank you for your thorough response. You have addressed the majority of my concerns, and particularly those around notation. My primary concern with the selection of $\lambda_1$ and $\lambda_2$ was that, since they seem to have been selected using test set performance, the models may have been overfit to each test set. However, since they were held constant across datasets after converging on one reasonable set of values, I am somewhat less concerned about this.

I think I understand Figure 4 a bit better now, but think the layout could still be improved for clarity. As is, it's a bit confusing how the "saliency map" heading bleeds into the section of the figure from CBM. I think clearer visual separation between areas of the figure corresponding to different methods would improve readability.

That said, this is a fairly minor point, and my primary concerns have been addressed. I've increased my score accordingly, and appreciate the authors' response.