A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis

Authors: yue  yang
mona  gandhi
yufei  wang
yifan  wu
michael s yao
chris  callison-burch
james c gee
mark  yatskar
  io/knobo
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-04 04:57:25

Abstract: While deep networks have achieved broad success in analyzing natural images, when applied to medical scans, they often fail in unexcepted situations. We investigate this challenge and focus on model sensitivity to domain shifts, such as data sampled from different hospitals or data confounded by demographic variables such as sex, race, etc, in the context of chest X-rays and skin lesion images. A key finding we show empirically is that existing visual backbones lack an appropriate prior from the architecture for reliable generalization in these settings. Taking inspiration from medical training, we propose giving deep networks a prior grounded in explicit medical knowledge communicated in natural language. To this end, we introduce Knowledge-enhanced Bottlenecks (KnoBo), a class of concept bottleneck models that incorporates knowledge priors that constrain it to reason with clinically relevant factors found in medical textbooks or PubMed. KnoBo uses retrieval-augmented language models to design an appropriate concept space paired with an automatic training procedure for recognizing the concept. We evaluate different resources of knowledge and recognition architectures on a broad range of domain shifts across 20 datasets. In our comprehensive evaluation with two imaging modalities, KnoBo outperforms fine-tuned models on confounded datasets by 32.4% on average. Finally, evaluations reveal that PubMed is a promising resource for making medical models less sensitive to domain shift, outperforming other resources on both diversity of information and final prediction performance.

Main paper: 
 introduction: 
 Robustness to domain shifts is a key property for models operating on medical images because transfer scenarios arise widely. Deep networks have achieved broad success in analyzing natural images (everyday human contexts), but when applied to medical scans, they often substantially degrade under distribution shift [86, 15] . Medical datasets are small, and unidentified confounds in them combined with model misspecification can dramatically degrade performance [46, 21, 24] . Such failure erodes confidence as models do not learn the right information from training data, hampering adoption by medical professionals. We study such problems by investigating the performance of systems in the presence of confounded data and address a main shortcoming we discover.Model sensitivity to domain shift can be measured by introducing synthetic confounds into data and evaluating on samples where the confound misleads the model. For example, in Figure 1 , we introduce confounded datasets for chest X-ray and skin lesion images where, during training, positive data is sampled from one group and negative from another. This association is reversed at testing time, creating an adversarial out-of-distribution (OOD) evaluation. In 5 such constructed confounds per modality, covering scenarios of race, sex, age, scan position, and hospital, we find models unable to generalize well, dropping over 63% on average over an in-distribution (ID) evaluation. are more robust to domain shifts (e.g., race, hospital, etc) than fine-tuned vision transformers [17] .Priors are an important signal allowing models to adopt appropriate hypotheses in low or misleading data regimes. We hypothesize that existing visual backbones lack an appropriate prior for robust generalization in medicine. Like previous work identifying that vision backbones have a deep image prior even when entirely untrained [72, 78] , we compare the quality of image representations produced by untrained networks on natural versus medical images. Given the output from a frozen untrained visual backbone, we train a linear classifier for predicting a diversity of labels (see Figure 2 ). Across architecture, these untrained models are higher quality featurizers of natural images than directly using pixels as features. In contrast, across multiple medical modalities, the deep image prior in current major visual backbones is no more effective than using pixels (and often worse).To address the lack of an effective deep image prior for medical images, we propose using an inherently interpretable model design. We draw inspiration from medical education, where students first learn from textbooks and later in a more practical setting during the residency with an attending doctor. Our models mimic this pattern: first, documents are used to identify important knowledge, and then they learn by example from data. We employ concept bottleneck models (CBMs) [41] and enrich them with information derived from resources broadly accessible to medical students. CBMs are a class of inherently interpretable models that factor model decisions into human-readable concepts that are combined linearly. Our methods build on recent approaches for language model (LM) guided bottleneck construction where LMs are prompted for discriminative attributes [90] .We introduce Knowledge-enhanced Bottlenecks (KnoBo) to incorporate knowledge priors that encourage reasoning with factors found in medical documents. KnoBo extends CBMs to medical imaging and employs retrieval-augmented generation into concept design. For example, we extract concepts from medical textbooks as natural language questions like Is there ground-glass opacity? to help the model classify whether an X-ray is positive for a respiratory infection. As illustrated in Figure 3 , KnoBo factors learning into three parts: (1) an interpretable bottleneck predictor, (2) a prior over the structure of the bottleneck, and (3) a prior over predictor parameters. This factorization allows us to guide the model with a prior rooted in medical documents. The approach relies on an iterative retrieval process where an LM summarizes documents to propose concepts, forming our medical image prior (Sec 4.2). Given the concepts, a pretraining corpus of reports and images is used to construct a classifier for a concept (Sec 4.3). Finally, a CBM is learned using predictions from classifiers on data while regularized by a prior formed from LM generations. (Sec 4.4).We evaluate KnoBo on our benchmark of confounded tasks. Averaged over confounds, KnoBo increases OOD performance by 41.8% and 22.9% on X-ray and skin lesion datasets, respectively. KnoBo's success in OOD performance comes at little sacrifice in ID settings, providing a better model overall when averaging the two data settings. We also explore 5 different sources of knowledge and reveal that PubMed outperforms other resources in terms of both diversity of information and final prediction performance. Overall, our work demonstrates that a key missing ingredient for robustness to distribution shift in medical imaging models is a prior rooted in knowledge.Random Pixel CNN ViT 0 Figure 2 : Classification performance on natural and medical images through linear probing using features extracted from untrained and frozen models versus pixels features (See Sec 3 for details). 
related work: 
 The rapid advancement in medical foundation models offers the opportunity to develop healthcare AI [55, 49] . However, their lack of transparency presents risks in real-world applications [5] . Interpretability is crucial for using models in high-stakes domains [30, 79, 87] . Previous work has primarily focused on post-hoc interpretability [76, 95, 29, 75] , which may not provide faithful explanations [69] . As an alternative, inherently interpretable methods produce explanations that align with the model's reasoning processes [8, 3] . In this work, we build upon Concept Bottleneck Models (CBMs) [41] , which predict by linearly combining human-designed concepts. Recent work [91, 60, 90] scales the applications of CBMs by aligning concepts and images with CLIP [64] and prompting language models to generate concept bottlenecks automatically. In our work, we treat CBMs as the architecture to incorporate knowledge priors for mitigating medical domain shift problems. Our bottlenecks, built from the medical corpus, are attributable and more trustworthy. Domain Generalization and Robustness are critical in medical domains where the distribution of imaging protocols, devices, and patient populations can significantly vary [25] . A line of work studies various domain-shift problems [42, 2] , proposing algorithms to learn invariant representations [57, 22, 80, 63] and employing domain/group information for reweighting [71, 93, 44, 96] . However, many studies show those methods do not improve over standard Empirical Risk Minimization (ERM) [67, 26, 33, 27] . Fine-tuning the last layer [68, 40] or selectively fine-tuning a few layers [47] is sufficient for robustness against spurious correlations in those datasets. We address domain shifts in medical imaging from a novel perspective by employing interpretable models to integrate knowledge priors. Our approach encourages models to adhere to diagnostic rules similar to those doctors use rather than relying on spurious correlations. Concurrent work shows bottleneck models can perform well on out-of-domain X-ray data but severely reduced in-domain performance as a consequence [89] . In contrast, we demonstrate a significantly better compromise between OOD and ID performance, using a broader set of modalities and constructing our bottlenecks from medical documents.Knowledge Rich Multimodal Reasoning. Knowledge plays an important role in clinical diagnosis [6] . Some multimodal tasks [82, 53, 74] require models to use explicit outside knowledge to make correct predictions. Previous methods [52, 50, 31 ] retrieve documents for each example from the external knowledge base as context for models to generate the answer. Our work focuses on leveraging knowledge in medical image classification. Retrieval-Augmented Generation [48, 23] has been shown to be beneficial for knowledge-intensive tasks [38] , including biomedicine [20, 84] . The retrieved medical documents are either used as context during inference [88] or data for pretraining [94] . In contrast, we treat documents as background knowledge for large language models to build concept bottlenecks. Instead of retrieving documents for every input, we build a global knowledge prior from a medical document corpus, which is shared across all examples. 
deep image priors for medical images: 
 This section revisits the concept of deep image priors [72, 78] , i.e., some data-agnostic assumptions from model structure, in the context of image classification across various domains. By comparing linear probing using features extracted by untrained deep networks against pixel-based features, we observe that existing vision backbones lack suitable priors for medical domains. This observation motivates our knowledge-enhanced bottlenecks (Sec 4) to integrate more robust priors into models.Setup. Consider a dataset of image-label pairs, D = (I, y) , where I is an image and y ∈ Y denotes the label from one of N classes. The model learns to predict P y|I, θ , where θ is the model parameters. We employ a frozen, untrained vision backbone V to extract features from I, producing a feature vector x = V(I), where x ∈ R d . A linear mapping function f θ : R d → Y is then trained to classify these features into label spaces. In this case, the model parameters θ will inherit the implicit architectural priors of V. As a baseline, we extract a subset of d pixels directly from the image as the feature without any model-based priors, represented as x p ∈ R d . We compare the classification performance using x versus x p to probe the efficacy of the vision backbone's priors.Experiments. We evaluate two state-of-the-art vision backbones, ViT-L/14 [17] and ConvNext-L [51] , on three categories of images: natural photos (e.g., ImageNet [70] ), X-rays (e.g., NIH-CXR [83] ), and skin lesion images (e.g., HAM10000 [77] ). Each image category has 5 datasets, and we report their average performance in Figure 2 (see Table 11 in the Appendix C.1 for full results). Figure 2 (left) shows vision backbones have effective priors for natural images, with ViT notably outperforming pixel by 14.7%. However, pixel features surpass those extracted by vision backbones for specialized domains such as X-ray and skin lesion images. This underscores these deep networks' lack of image priors appropriate for these domains, which can hamper model learning and hurt generalizability. Without guidance from appropriate priors, models can overly rely on data, risking catastrophic failures. We aim to overcome this by injecting additional priors into models.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]Answer: [Yes] Justification: The code is provided in the submission as a zip file, and we will release the code and data on Git Hub after review. Guidelines: • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
knowledge-enhanced bottlenecks: 
 In this section, we present Knowledge-enhanced Bottlenecks (KnoBo), a class of CBMs that incorporate knowledge priors that address the failures we identified in Section 3. Figure 3 presents an overview of our method. From left to right, we optimize three terms: ( where we assume the priors over structure C and parameters θ are independent. The structure prior P (C) (Sec 4.2) is formulated as the construction of a bottleneck with N C concepts, C = c 1 , c 2 , ..., c N C , derived from a background corpus B. Each concept is a factor that humans will use when solving the same task. The bottleneck predictor P y|I, C, θ (Sec 4.3) is a concept bottleneck model that predicts the label conditioned on the input image, bottleneck, and learned parameters. The bottleneck predictor is inherently interpretable, and each parameter in θ has semantics, denoting the association between concepts and labels. The parameter prior P (θ) (Sec 4.4) regularizes the learning of model parameters θ with information derived from human knowledge. Jointly optimizing over structure and model parameters is intractable, so we first select a high-quality concept space and then optimize the parameters of the bottleneck jointly with the parameter prior. Q ′ ← [] , set of new queries for q in Q do B ′ = Retrieve (B, q) C ′ = LLM (B ′ ) C ← C + C ′ , Q ′ ← Q ′ + C ′ end for Q ← Q ′ /* update queries */ end whileGiven a background corpus B that spans various documents, we aim to identify a bottleneck structure containing concepts beneficial for classifying labels y ∈ Y. As outlined in Algorithm 1, we use the class names Y as initial queries to retrieve relevant documents B ′ ⊂ B. Large Language Models (LLMs) are then prompted to generate concepts using these retrieved documents as context: C ′ = LLM(B ′ ). These newly generated concepts are added to the bottleneck and used as new queries to retrieve additional documents. This iterative process continues to expand the bottleneck until a predetermined number of concepts N C is reached. Such concept structures have high likelihood under the language model, conditioned on the background corpus, and the language model probability serves as an implicit prior.With the structure prior from the background corpus, we optimize (1) the grounding function G : R d → R N C , which maps the input image to the concept space, and (2) a linear layer f : R N C → Y that projects concept predictions onto labels. In practice, we implement G as a set of grounding functions: G = {g c } c∈C where each g c predicts the probability of an image I having the concept c, P (c|I) = g c (x), and x ∈ R d is the image feature. Specifically, we derive training examples for grounding functions from a pretraining dataset of image-text pairs. We use the language model to estimate the presence of a concept in the image based on the information in the accompanying text.Concept Grounding. Suppose we have a pretraining dataset D pre of image-text pairs (I, t) , where t is a textual description (such as a clinical report) of the image. Based on t, we can infer if a concept c is present in the image. This can be automated by prompting a large language model to generate a response indicating whether the text implies the concept. This way, we label our pretraining data as positive and negative examples for each concept c, which can be used to train its grounding function. With those annotated training examples, we implement each grounding function as a binary logistic regression classifier:g c (x) = σ x • W ⊤ c, where W c ∈ R d is the weights of grounding function and σ is the sigmoid activation. Finally, we form a collection of grounding functions G = {g c } c∈C to map an image feature x into N C probabilities over all bottleneck concepts, with G(x) ∈ R N C . Linear Layer. Using concept probabilities G(x) as input, we train a simple linear function f to make the final label prediction: ŷ = f G(x) = G(x) • W ⊤ , where W ∈ R N ×N C is the linear weight matrix, with N the number of classes and N C the number of concepts.The bottleneck predictor is inherently interpretable because the parameters of the linear layer encode the affinity between labels and concepts. Therefore, we can guide the parameters based on prior knowledge, i.e., if the label y is positively related to concept c based on background knowledge, the weight w y,c ∈ W should be high. We hope the learned parameters do not deviate too much from this assumption, otherwise, the model may capture spurious correlations in the data.To enforce this, we let language models define a weight matrix of priors W prior ∈ R N ×N C , with each element w y,c ∈ {-1, +1} indicating the sign of a preferred correlation between the label y and concept c. The prior loss is calculated as the L1 distance between between W and W prior :EQUATIONin which we apply tanh activation on W to scale the linear weights to (-1, 1), matching the scale of the weights in the prior matrix. This adjustment aligns the model's parameters with the expected sign of the correlations based on prior knowledge. The final loss function to train the linear layer is the sum of the cross-entropy loss and the prior loss:L = L CE + L prior .In summary, we search for a structure C that is consistent with prior knowledge from a background corpus B to severe as the bottleneck for the predictor P y|I, C, θ . The parameters θ are aligned with the predefined correlations between labels and concepts identified by language models. 
experimental setup: 
 This section introduces (1) the confounded and unconfounded medical datasets to evaluate the robustness of our knowledge-enhanced bottlenecks (Sec 5.1), ( 2) the black-box and interpretable baselines for comparison (Sec 5.2), and (3) the implementation details of our method (Sec 5.3).We evaluate two groups of datasets for each modality: (1) the confounded datasets, which aim to assess the robustness of models by creating splits with spurious correlations; (2) the unconfounded datasets are randomly split to measure the models' performance in natural settings.Confounded Datasets. As illustrated on the left of Figure 1 , we formulate the confounded datasets as binary classification tasks, where each class is confounded with one factor. The confounding combinations are reversed for in-domain (train and validation), and out-of-domain (test) splits.The confounded datasets of chest X-ray are constructed from NIH-CXR [83] and CheXpert [35] with their provided attributes: (1) NIH-sex uses sex (male, female) as the confounding factor; Unconfounded datasets. We evaluate 10 datasets with random splits, 5 for each modality. X-ray: Pneumonia [39] , COVID-QU [9] , NIH-CXR [83] , Open-i [16] , and VinDr-CXR [58] . Skin Lesion: HAM10000 [77] , BCN20000 [14] , PAD-UFES-20 [62] , Melanoma [36] , and UWaterloo [45] .All datasets are split into train/validation/test and ensure the validation and test set are balanced across classes. Detailed statistics and additional information on each dataset are provided in Appendix A.Pretraining Datasets. The training of vision backbones and concept grounding functions utilizes datasets with image-text pairs. For X-rays, we choose MIMIC-CXR [37] , which contains 377,110 X-ray images with accompanying clinical reports. Since there is no existing text-annotated dataset for skin lesion images, we employ GPT-4V [61] to generate captions (see examples in Figure 9 ) for a subset of 56,590 images from ISIC, without overlap of the confounded and unconfounded datasets.We compare KnoBo against both black-box models and interpretable concept bottleneck models.Black-box Models. We include two end-to-end fine-tuning baselines: (1) ViT-L/14 [17] and (2) DenseNet121 [32] , both pretrained on the pretraining datasets mentioned earlier. Additionally, (3) Linear Probe extracts visual features with the frozen ViT-L/14 encoder and learns a linear layer for classification. (4) Language-shaped Learning (LSL) [56] aims to disentangle the impact of knowledge and interpretable structure. Inspired by LSL via captioning, we finetune a ViT-L/14 with the same data used for concept grounding functions and apply a linear layer (see Appendix B.2).Post-hoc CBM (PCBM-h) [91] ensembles concept bottleneck models with black-box residual predictors. We let PCBM-h use the same bottlenecks as our KnoBo method;(2) LaBo [90] applies language models to generate concepts, followed by the submodular selection to identify a subset that enhances performance. Following their original settings, PCBM-h and LaBo use CLIP (fine-tuned on medical pretraining datasets) to align concepts with images. All baselines use backbones trained on the same pretraining data as our method to ensure a fair comparison. Appendix B.2 provides additional details about the baselines.Evaluation Metrics. We use accuracy as the metric since all evaluated datasets are single-label classification tasks with balanced validation and test sets. For confounded datasets, we report indomain (ID, validation), out-of-domain (OOD, test), and domain-average (mean of ID and OOD) accuracies, along with domain gaps (∆ = |ID -OOD|), where a lower ∆ indicates better robustness.For unconfounded datasets, we report test accuracy. A robust and performant model must achieve a good compromise between confounded and unconfounded datasets. For all the baselines and our KnoBo method, the checkpoints with the highest validation accuracy are evaluated on the test set.Pretraining of Medical CLIP. We fine-tune OpenCLIP [34] (ViT-L/14 pretrained on LAION-2B [73] ) on the pretraining medical data for each modality. Unlike previous work [18, 85, 92] that directly pairs medical images with sentences from clinical reports, we preprocess the reports by employing GPT-4 [1] to extract short phrases. Our CLIP models perform the best for both X-ray and skin lesion datasets in zero-shot and linear probing, as shown in Table 9 in the Appendix.Medical Corpus. We download 5.5 million articles from PubMed and segment them into 156.9 million snippets to serve as documents for retrieval. Alternatively, we take the medical corpus organized by MEDRAG [88] , including documents from Wikipedia, StatPearls, and medical textbooks. We employ BM25 [66] as the ranking function for document retrieval.KnoBo Details. We select GPT-4 (gpt-4-0613) as the underlying LLM for retrieval-augmented concept generation (Sec 4.2). For training concept grounding functions (Sec 4.3), we opt for Flan-T5-XXL [10] to annotate clinical reports for each concept, considering cost-efficiency. Unless otherwise specified, KnoBo uses bottlenecks constructed from PubMed, each with 150 concepts. Figure 5 shows our prompt, and during concept generation, we apply several heuristic filters (Appendix B.3). 
results: 
 In this section, we discuss KnoBo's performance on confounded and unconfounded medical image datasets (Sec 6.1) and analyze different knowledge resources and our model design (Sec 6.2). PROMPT is our baseline without retrieving documents for concept generation. We report the accuracy of confounded (Confd, average over ID and OOD), unconfounded (Unconfd) datasets, and the overall performance of all datasets. Diversity measures the difference between the concepts in a bottleneck.KnoBo is more robust to domain shifts. Table 1 shows the results on 10 confounded datasets of X-ray and skin lesions. Black-box models excel at in-domain (ID) data but drop significantly on outof-domain (OOD) data, especially in datasets confounded by hospitals/resources (NIH-CheXpert and ISIC-hospital), which can be common when collecting medical datasets [12, 81] . KnoBo outperforms baselines in OOD and domain-average accuracy by large margins, ranking top-1 in eight datasets and second-best in the other two. End-to-end models (ViT-L/14, DenseNet) exhibit larger domain gaps than linear probes, as they have more parameters to optimize performance on in-domain data and capture spurious correlations. Shaping the visual representations with knowledge (LSL) improves robustness but underperforms KnoBo, with lower ID, OOD, and average performance across most datasets. PCBM-h combines interpretable and black-box predictions but exhibits behaviors similar to black-box models with severe drops across domains. Unlike KnoBo, which uses medical documents to create one global bottleneck for each modality, LaBo builds a bottleneck for each dataset using the in-domain data, which can be biased and affected by confounding factors and performs more poorly. In summary, KnoBo mitigates the catastrophic failures in domain shifts encountered by black-box and is more robust against various confounding factors across modalities.KnoBo performs the best across confounded and unconfounded data. In this section, we compare the bottlenecks constructed from different knowledge sources. We evaluate the impact of each KnoBo component on the final performance, including bottleneck size, concept grounding function, and parameter prior. Additional analyses are available in Appendix C.Knowledge Sources. Besides the empirical results on confounded and unconfounded datasets, we measure the diversity of bottleneck C asDiversity(C) = 1 |C| 2 -|C| ci∈C i̸ =j cj ∈C 1 -sim(c i , c j ), where the sim(•) is the cosine similarity of concept features encoded by sentence transformer [65] . The Diversity computes the distance between each concept and every other concept in the bottleneck. Table 3 compares different knowledge sources. The retrieval-augmented bottlenecks perform better than those generated by prompting, especially for skin lesions, where more specific knowledge is required because prompting lacks diversity. Across both modalities, PubMed is the best overall, performing better for the X-ray modality than other knowledge sources and among the best for skin lesion modalities. Moreover, shown in Table 6 , our retrieval-augmented concepts are attributable, which allows doctors to verify the source of knowledge. Human Evaluation on Bottlenecks. In evaluations by two medical students, information from all knowledge sources is rated as highly relevant and groundable. Two medical students evaluated the quality of bottlenecks using two metrics: (1) Relevance measures the concept's relevance to diagnosing diseases on a scale from 1 (not at all relevant) to 4 (mostly relevant), and (2) Groundability assesses the verifiability of the concept from the image on a scale from 1 to 4. We evaluated 30 randomly sampled concepts from each bottleneck. Table 4 presents these metrics for bottlenecks constructed from five different knowledge sources. While all bottlenecks show good relevance, groundability scores are lower, reflecting the challenge of deriving visual concepts from text-only data.Bottleneck Size. Figure 4 compares KnoBo and linear probes while varying the number of concepts/features. KnoBo consistently outperforms linear probes across all metrics when given the same quota of features, and KnoBo can obtain better performance with fewer features. This indicates that interpretable concept scores have more effective priors than black-box visual features.Ablations. Table 5 summarizes experiments ablating major components of our approach. Row 2 shows the performance of using dot-products from prompted CLIP models as concepts, which Table 6 : We show concepts from various bottlenecks by image modality (corpus), with the queries for retrieval and the corresponding reference documents to generate the concept. Every concept is attributable, allowing medical professionals to verify its origin in the supporting documentation.markedly reduces performance. This shows the importance of knowledge grounding in ensuring KnoBo's effectiveness. However, this step can be simplified as more advanced medical foundation models are available. Row 3 shows performance omitting the parameter prior. It is an important mechanism for constraining the final learning phase, resulting in consistent OOD improvements. This is also reflected in Figure 4 , where with the parameter prior, KnoBo performs better on OOD splits while decreasing on in-domain as the parameter prior constrains the model to rely on data. 
conclusion and limitation: 
 In this paper, we analyze domain-shift problems in medical image analysis and identify a missing medical deep image prior as a main contributor to poor performance. To address this, we introduce knowledge-enhanced bottlenecks (KnoBo) to integrate knowledge priors from medical documents. Across two medical image modalities under various domain shifts, KnoBo significantly improves robustness over black-box baselines.Limitation. KnoBo assumes the availability of medical multimodal datasets, limiting applications to rare conditions. While our work improves robustness, medical experts do not fail in these ways, and they should be used in conjunction with models. Our preliminary work with PubMed suggests it is a valuable resource for developing medical models, and future research can explore how to utilize such fruitful knowledge resources more effectively. Table 7 and 8 show the detailed statistics for all 20 datasets evaluated in this paper, where we list the number of classes with class names, the number of images for training, validation, and testing. For confounded datasets, here we explain some details about how we define the confounding factors:Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Figure 4 and 6 have error bars. Broadly, the paper covers many datasets and reports each individually. This provides enough evidence about the variance of the approach.We do not rerun methods on each dataset as this would be computationally wasteful. Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: This information is provided in the appendix B.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: This information is provided in the last section of the paper in the main body.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for the responsible release of data or models with a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: We will release the data and models with an appropriate warning that these systems should not be used for any direct diagnosis without randomized trials supporting their efficacy.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We cite all dependencies of our work and verify that we are using their data, models, and methods in a way consistent with their terms.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: Dataset• NIH-sex and ISIC-sex are built based on the patient sex from NIH-CXR [83] and ISIC.• NIH-age defines young as patient's age ≤ 30 and old as patient's age ≥ 60.• NIH-pos refers to the patient's position during an X-ray procedure. The standard position, Posterior Anterior (PA), involves the patient standing, while the Anterior-Posterior (AP) view is used when the patient cannot stand and must lie down. • CheXpert-race selects racial subgroups (White, Black or African American) from CheXpert [35] .• NIH-CheXpert confounds classes by sourcing X-rays from either the NIH-CXR or CheXpert datasets. Specifically, data for one disease is obtained from one dataset, while data for a different disease is sourced from the other dataset. • ISIC-site focuses on lesions located either on the head or an extremity.• ISIC-color organizes images based on the Fitzpatrick scale of skin tones [19] . Fitzpatrick I is classified as light skin, and III, IV, and V as dark skin, according to the annotations from [4] . • ISIC-hospital introduces confounds in classes by using lesion images exclusively from either the Hospital Clínic de Barcelona or the Medical University of Vienna.We experimented with existing CLIP models in the medical domain and found their performance to be unreliable. Therefore, we decide to train our own CLIP models for X-ray and skin lesion images.Pretraining Dataset. For X-rays, we utilize the MIMIC-CXR dataset [37] , specifically selecting only the PA and AP X-rays, which results in 243,334 images, each accompanied by a clinical report written by doctors. For Skin Lesion images, we employ the ISIC dataset and use GPT-4V [61] to generate clinical reports for 56,590 images, examples are shown in Figure 9 . We preprocess these reports by extracting medically relevant findings, each described in a short and concise term. The example below demonstrates a report alongside the findings captured by GPT-4. In total, we assemble 953K image-text pairs for X-rays and 438K for skin lesion images. Training Details. We utilize the training script from OpenCLIP [34] and select ViT-L/14 as the backbone. Training is performed on 4 RTX A6000 GPUs for 10 epochs with a batch size of 128 and a learning rate of 1e -5 . We choose checkpoints based on the lowest contrastive loss on validation sets.CLIP Baselines. We compare various CLIP models across unconfounded datasets for two modalities, including OpenAI-CLIP [64] , OpenCLIP [34] , PubMedCLIP [18] , BioMedCLIP [92] , PMC-CLIP 1and MedCLIP [85] . We evaluate these models in both zero-shot and linear probe scenarios. In zero-shot, GPT-4 generates prompts for each class, and we use the ensemble of cosine similarities between the image and prompts as the score for each class. In linear probing, we use the CLIP models as image encoders to extract features for logistic regression. Additionally, we include DenseNet-121 [32] (fine-tuned on the pretraining datasets with cross-entropy loss) as a baseline for linear probing.Results. Figure 9 shows that our CLIP models perform best in both zero-shot and linear probing scenarios for both modalities. We find that preprocessing the text data with an LLM significantly enhances zero-shot performance. Existing medical CLIP models outperform general CLIP models on X-ray datasets but not on skin lesion images, possibly because X-ray data are more prevalent and accessible in the medical domain. While our CLIP models excel with careful data curation, training converges quickly, suggesting the current contrastive objective might not fully exploit the information from the data, potentially taking shortcuts, such as comparing images from different patients instead of focusing on diseases. Future research should explore more suitable objectives and larger-scale data collections to develop more robust medical foundation models.This section outlines the implementation details of all baselines compared with our knowledge bottlenecks for medical image classification. All baselines are run on a single RTX A6000 GPU.• ViT-L/14: We utilize the visual encoders from the CLIP models we pretrained in Sec B.1 and add a classification head for downstream classification datasets. We unfreeze the ViT-L/14 backbone and train all parameters with a learning rate of 1e -6 and a batch size of 64 for 20 epochs. • DenseNet-121: Similarly to ViT-L/14, we add a classification head to the pretrained DenseNet and train the entire network end-to-end. For X-rays, we use the DenseNet pretrained on MIMIC-CXR by TorchXRayVision [13] . For skin lesion images, we pre-train the DenseNet from scratch on ISIC using a cross-entropy loss. When fine-tuned for downstream classification datasets, we train the DenseNet with a learning rate of 1e -5 , a batch size of 64, and also for 20 epochs. • Linear Probe: We employ visual encoders (ViT-L/14) from pretrained CLIP models to extract features for images in downstream classification datasets. We train a linear layer to map these features into labels for 200 epochs with a learning rate of 1e -3 and a batch size of 64. • LSL [56] : We fine-tune the pretrained CLIP with contrastive loss on annotated concept data (PubMed bottleneck), using the same data as for concept grounding functions (4.3) . Training instances are triplets (I, c, y), where I is the image, c is a textual concept, and y ∈ {0, 1} is a binary label indicating whether the image contains this concept. Given the visual encoder V and textual encoder T of the CLIP, the cosine similarity between an image and a concept is s (I, c) = cos V(I), T (c) . The contrastive loss function is defined asL contrast = y • max 0, m -s (I, c) + (1 -y) • s (I, c), where m = 0.6 is the margin. We fine-tune the CLIP with concept annotations for 20 epochs with a learning rate of 1e -6 and a batch size of 64. After obtaining the fine-tuned CLIP, we extract features and train a linear probe in the same manner as the linear probe baseline. • PCBM-h [91] and LaBo [90] : We use their codebases to implement these baselines. The concept alignment in both models is achieved using CLIP to compute the dot product between image and concept features. PCBM-h uses the same bottleneck as KnoBo, which is generated from PubMed.For LaBo, we employ GPT-4 to generate candidate concepts for submodular selection.This section provides additional details about the implementation of KnoBo.Medical Corpus. We utilize a comprehensive medical corpus for retrieval-augmented generation, detailed as follows: ( I want you to filter and summarize the information in these documents and generate knowledge in the form of *binary questions*, e.g., "Is there lung opacity?" ["Is the lesion asymmetric?"]. Please follow instructions below strictly: 1. Those knowledge will be used to guide the diagnosis on chest X-rays [skin lesion images], so they must be *visually identifiable* from chest X-ray [skin lesion images] only.2. The binary questions should be concise and not too specific which can be reused for different cases.3. The binary questions must not contain the class(disease) name, e.g., you *must not* generate "Is there cardiomegaly?" ["Is the lesion malignant?"] as the knowledge for "Cardiomegaly" ["Malignant Lesion"]. 4. If there is not much information in the some documents, you can ignore those documents. If none of the documents contain useful information, you can skip this task by typing 'skip'. Please answer without additional information and do not add numbers or bullet points in the answer.Figure 5 : Prompt template for retrieval-augmented concept bottleneck generation. The text in the square brackets is words that need to be changed when using this prompt for skin lesion images. 4) Wikipedia (6.5M docs, 29.9M snippets). The StatPearls, Textbooks, and Wikipedia sources are obtained from MEDRAG [88] . Unlike the abstract-only approach of PubMed in MEDRAG, we utilize full articles from PubMed, including all paragraphs. We employ the retrieval codebase of MEDRAG and select BM25 as the ranking function.Retrieval-augmented Concept Bottleneck Generation. Figure 5 illustrates the prompt template we use to generate concepts from documents. We retrieve the top 10 documents for each query as context for the large language model (GPT-4) to generate concepts. After generating concepts, we validate each concept based on three criteria before inclusion in the bottleneck: (1) the concept must be distinct from existing concepts; (2) it must be visually identifiable from the image; and (3) there must be sufficient positive and negative instances in the pretraining corpus to support training its grounding function. A concept is added to the bottleneck only if it meets all three criteria, as judged by another language model (GPT-4). We initially target 200 concepts per bottleneck but ultimately select the top 150 with the highest grounding accuracy for inclusion. This selection is due to some concepts lacking sufficient reports to effectively train their grounding functions, making 150 the minimum size for all the bottlenecks we construct. Table 6 shows examples of generated concepts.Concept Grounding. We use a language model to annotate 2,000 clinical reports for each concept from the pretraining corpus. To efficiently label reports and achieve a balance of positive and negative examples, we retrieve the top 1,000 reports showing high textual similarity (measured by Sentence Transformer [65] ) to the concept as its potential positive examples and randomly sample another 1,000 for potential negatives. We use Flan-T5-XXL [10] as the underlying large language model. Specifically, the annotation task is treated as a next token prediction, similar to the approach proposed by McInerney et al. [54] , where we compare the probabilities of the next token being Yes or No to determine if the report contains the concept. Table 10 shows no big difference in final classification performance when using Flan-T5 versus GPT-4 for annotating concepts on reports.This section presents additional analysis and ablation studies on our method.We provide further details about the deep image prior experiments in Sec 3. Datasets. We evaluate the deep image prior on three categories of images. The X-ray and skin lesion images are the same as those described in the unconfounded datasets section (Sec A). For natural images, we select five datasets: (1) CIFAR-10 [43], (2) STL-10 [11] , (3) ImageNet-10 [70]2 , (4) Food-101 [7] , and (5) Flower-102 [59] .Setup. We employ the vision backbones ViT-L/14 [17] and ConvNext-L [51] , both implemented by OpenCLIP [34] , and initialize them using Kaiming initialization [28] following PyTorch's default settings 3 . Both backbones extract feature vectors of size 768. For the pixel value baseline, we convert the image to grayscale, resize it to 28 × 28, and then flatten it into a vector of 784 dimensions. We use the first 768 values of this pixel vector to match the size of the deep features. All features are passed through a linear layer to predict the labels with a learning rate of 1e -3 , a batch size of 64 for 200 epochs. Additionally, we include a random baseline for comparison.Results. Table 11 displays the full results of all methods across the three image categories. ViT-L/14 excels on natural datasets with significant gains over pixel baselines. ConvNext-L performs worse than ViT-L/14 but is still notably more effective than pixel-based methods. For X-ray images, the pixel baseline clearly surpasses the two networks on almost all datasets, indicating that deep models lack priors or even have harmful priors for X-ray imaging. For skin lesion images, which are closer to natural images, the pixel value baseline performs comparably to ViT-L/14 and better than ConvNext-L. Overall, the results suggest that deep networks lack sufficient priors for medical domains, potentially affecting their generalizability.Table 12 shows the comprehensive results of all baselines across the 10 unconfounded medical datasets. KnoBo performs competitively in the X-ray category, securing top-1 positions for two datasets and achieving an average ranking of third among all methods. For skin lesion datasets, KnoBo's performance is limited by the smaller scale and lower quality of the pretraining corpus, which is annotated by GPT-4V rather than by human experts. This affects the effectiveness of the grounding functions for lesion concepts. With access to larger-scale and higher-quality pretraining data, KnoBo could potentially close the performance gap with black-box baselines.Top-3Is there a visible enlargement of the heart?Top-3Top-3Top-3 14 displays examples of failed concepts identified by medical students during human evaluation. Each concept can be traced back to its source document to verify its accuracy. Figure 9 presents examples of GPT-4V-annotated clinical reports for skin lesion images. Notably, the second example demonstrates GPT-4V's capability to assess lesion size using the scale provided in the image.NeurIPS Paper Checklist Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Justification: Having reviewed the policy, we follow all bullets across the three areas.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).https://huggingface.co/ryanyip7777/pmc_vit_l_14We use the 10 classes selected by Imagenette: https://github.com/fastai/imagenette.https://pytorch.org/docs/stable/nn.init.html 


Decision: All reviewers are in agreement and there is a strong consensus regarding the novelties presented in the paper, the thoroughness of the evaluations, and the clarity of the writing. The paper demonstrates significant contributions to the field and provides comprehensive experiments that validate the proposed approach, making it a well-received submission. However, some ethical concerns were raised during the review process, which the ethics reviewers believe are addressable by the authors. These concerns mainly revolve around specific aspects of data usage and consent, but with appropriate revisions and clarifications, the paper is expected to meet ethical standards. The authors are advised to address all raised issues in the final paper.

Reviews: review: paper_summary: paper_summary: This paper proposed Knowledge-enhanced Bottlenecks (KnoBo) to leverage the concept bottleneck model (CBM) to improve model robustness to various domain shifts and confounders. KnoBo explores using retrieval augmented generation to generate concept space with a large number of concepts using a large language model (i.e., GPT-4). KnoBo brings significant overall improvement when large-scale image-text pairs are available for concept grounding. A comprehensive evaluation across various scenarios and various baselines confirms the effectiveness of KnoBo in reducing the gap between in-distribution and out-of-distribution performance.
strength_weakness: strength_weakness: 1. The concept grounding seems pretty data-hungry, requiring large-scale multi-modal datasets for pretraining (e.g., MIMIC-CXR with over 300k image-text pairs). For smaller datasets (ISIC with about 60k images with generated captions from LLM), the proposed KnoBo only brings marginal overall improvement. This requirement significantly limits the broad application of the proposed framework in the data-scarcity domain, such as cancer/tumor classification. 
2. Lack of a detailed discussion of limitations (one one-line limitation in Section 7) and presentation of failure cases. No failure cases (wrong classification / wrong concept) were included in either the main text or supplemental material. 
3. Some results are confusing to the reviewer (See No. 2 in the Questions section).
questions: questions: 1. In Table 1, the best performance of ISIC-age should be ViT-L/14 and LSL, not the proposed KnoBo (which is bolded). 
2. The KnoBo with 150 concepts has an average OOD Acc of 58.8 across Chest X-ray Datasets, as stated in Tables 2 and 4. Why does Figure 4 indicate that KnoBo with 135 concepts has an accuracy below 50 for OOD X-ray datasets? As the performance of OOD datasets decreases with the increased number of concepts, these two results seem contradicted. 
3. Typo in the title of Algorithm 1 in Section 4.2. Should be 'retrieval'
limitations: limitations: As explained in the Weaknesses and Questions sections.
paper_summary: paper_summary: The presented paper addresses the challenge of domain shifts in medical image classification, where conventional neural networks often lack effective priors for medical datasets. The authors introduce KnoBo, a novel class of concept bottleneck networks that integrate medical knowledge priors to enhance neural network performance in medical image classification tasks. KnoBo has three primary components: 1) Structure Prior:  leverages medical documents to construct a knowledge bottleneck; 2) Bottleneck Predictor:  maps images onto concepts; 3) Parameter Prior: imposes constraints on learning the linear layer's parameters, ensuring that the network's predictions remain consistent with the established medical knowledge. The proposed method of KnoBo is evaluated using two medical datasets, ISIC and X-ray, under both confounded and unconfounded settings.
strength_weakness: strength_weakness: The paper's primary weakness is its reliance on content generated by LLMs to create new medical concepts. Although GPT-4 is one of the most advanced models for generating concepts, it can still produce hallucinations, especially when the generated concepts are loosely aligned with common terms such as hair, skin color/tone, gel bubble, and rules (in the context of skin lesions).
questions: questions: - Whats is the main reason to define $ \Delta = | ID - OOD |$ as robustness measure (row 257)? In my opinion, this metric is more related to some "fair predictions" concept, since it measures how ID and OOD performances differ.  
- Do the authors know why the diversity values for skin lesion datasets (Table 3, last column) are much lower than those for X-rays?
limitations: limitations: Not a limitation, but I recommend incorporating a filtering scheme into the concept generation process to enhance the paper. While the authors have conducted human evaluations, relying solely on such assessments may not be feasible due to the limited availability of human resources. Refining the concept generation could be achieved by improving the prompts or employing a medical foundation model to assess the quality of the generated concepts about the target image.
paper_summary: paper_summary: The authors noticed the domain-shift issue of current medical dataset, and after taking inspiration from medical training, they propose giving deep networks a prior grounded in explicit medical knowledge communicated in natural language. The proposed network can incorporate medical knowledge priors to help model make decisions. Basically, the authors assumed that “medical concepts” are robust enough to against the domain shift. Valid assumption, interesting work.
strength_weakness: strength_weakness: It is very HARD to read and interpret. Please raise examples for better reading experience. Maybe I have some questions after the clearance. Other points could be seeen under the Questions section.
questions: questions: 1. “In 5 such constructed confounds per modality, covering scenarios of race, sex, age, scan position, and hospital, we find models unable to generalize well, dropping over 63% on average over an in-distribution (ID) evaluation.” What are the tasks? Please specify.
	2. “As a baseline, we extract a subset of d pixels directly from the image as the feature without any model-based priors, represented as xp ∈ Rd. We compare the classification performance using x versus xp to probe the efficacy of the vision backbone’s priors.” I didn’t get the point. Why did you want to compare the effectiveness of x and x_p? 
	3. It is very hard to read the 4.2 section. I cannot fully understand the algorithm 1. Please raise examples for better reading experience.
	4. Table 1, why the KnoBo always has not the best, or sometimes the worst performance in in-domain dataset?
	5. Is the Structure Prior learnt? Or the questions like “Is there ground-glass opacity?” were pre-defined? How ist he connection between “Is there ground-glass opacity?” and the top-right red box in Figure 1?
	6. Is the training end-to-end? Can you please provide an example for 1) how to pretrain using medical books 2) and given an X-ray image, how the workflow is to get the last result? I hold positive thought towards the study, but it is really hard to understand the workflow.
	7. Is the parameter prior a part of the bottleneck predictor? In figure 1 it seems like parallel, but in the main text it seems like existing sequentially. Hard to understand.
	8. How to determine the best number of concepts?
	9. How did you determine the preferred correlation between the label y and concept c?
limitations: limitations: N/A
paper_summary: paper_summary: This paper addresses the issue of deep networks' sensitivity to domain shifts in medical image analysis, particularly for chest X-rays and skin lesion images. The authors propose Knowledge-enhanced Bottlenecks (KnoBo), a concept bottleneck model that incorporates explicit medical knowledge from resources like textbooks and PubMed to improve generalization. KnoBo leverages retrieval-augmented language models to create a clinically relevant concept space and demonstrates a significant performance improvement—32.4% on average—over fine-tuned models across 20 datasets, highlighting PubMed as a particularly effective knowledge resource for mitigating domain shifts.
strength_weakness: strength_weakness: Lack of Application to 3D Medical Imaging:
Issue: The paper primarily focuses on 2D medical imaging modalities, such as chest X-rays and skin lesion images, without addressing the broader and more widely used domain of 3D medical imaging.
Impact: This limitation raises concerns about the generalizability and applicability of the proposed KnoBo model to 3D imaging scenarios, such as CT scans or MRI, which are critical in many clinical contexts. Including evaluations on 3D medical images would significantly enhance the relevance and impact of the research.
Insufficient Comparative Analysis with 3D Domain Adaptation Methods:
Issue: There is a lack of comparative analysis with existing domain adaptation and generalization methods specifically applied to 3D medical imaging.
Impact: The absence of such comparisons makes it difficult to gauge the relative effectiveness of KnoBo in a comprehensive manner, particularly given the significant advancements in 3D domain adaptation techniques. Addressing this gap would provide a more complete evaluation of KnoBo's capabilities and limitations.
questions: questions: Application to 3D Medical Imaging:
Question: Have you considered extending the KnoBo framework to 3D medical imaging modalities such as CT scans or MRIs? If so, what challenges do you anticipate, and how might they be addressed?
Suggestion: Expanding your research to include 3D medical images would significantly increase the applicability and impact of your work. Discussing potential adaptations and preliminary results for 3D imaging could provide valuable insights and demonstrate the versatility of KnoBo.
Comparative Analysis with State-of-the-Art Methods:
Question: How does KnoBo compare with recent state-of-the-art domain adaptation and generalization methods, especially those tailored for 3D medical imaging?
Suggestion: Including a detailed comparative analysis with advanced domain adaptation techniques, particularly those used in 3D imaging, would strengthen your paper. This would provide a clearer benchmark and highlight the unique advantages and limitations of KnoBo.
limitations: limitations: No

Comments: title: Thanks for all of the feedback and suggestions!
Thanks for interesting discussion! Extension to detection
Thanks for the clarification
Grounding Function Data Requirements Clarification
Regarding Hallucinations
Response to the rebuttal
Has the rebuttal addressed your concerns?
Has the rebuttal clarified sufficiently?
Has the rebuttal addressed your concerns?
comment: Thanks for all of the feedback and suggestions!
It would be great if you could add a hint of discussion about potentially applying KnoBo to segmentation/detection settings, and I think it would increase the paper's influence. If space not allowed, it could be put in the appendix. I hope the author can manage to include all the committed changes (to my review and others' review) in the final version. 

I feel like the rebuttal and discussion adequately address my questions and concerns. Therefore, I'm happy to raise my rating of presentation from 2 to 3, my rating of contribution from 2 to 3, and my overall rating from 5 to 7. Congrats on the work!
Thanks for the reply. We appreciate the engagement and feel this discussion is making the work much better.

In general, we believe classification is often an important sub-component of medical image analysis pipelines and KnoBo shows how to make that aspect controllable with priors via interpretable design. Segmentation (and related problems like detection and registration) is an important problem, although we have not thoroughly explored it yet. In the case of segmentation though, many model architectures decouple region prediction and classification (i.e. works like Mask R-CNN or Yolo have separate heads for predicting bounding boxes and their classifications) and KnoBo almost directly applies. In terms of making the spatial part of predictions interpretable and having appropriate priors, we may need a different mechanism than concept bottlenecks. It would also be interesting to analyze if confounding errors for segmentation are coming from the spatial or classification components. We can include a discussion of this in the future work / limitations section.
Thank you for the reply and clarification. I was confused about the number of examples for grounding functions after receiving the first rebuttal and, therefore, did not update my score. This clarification helps a lot. 

Before the end of the discussion and providing my revised score, I have one last question: The title of this manuscript contains a broad term of medical image analysis, while the current version merely focuses on classification. Given the fact that the ISIC dataset also provides a segmentation mask, I wonder if the authors have explored applying KnoBo in the context of segmentation and OOD/confounded generalization.
Apologies, we wanted to clarify the question of data efficiency in the grounding function further (and there was a small typo in the images required before). The mechanism for constructing them is:

1. given a concept name (i.e. ground class opacities for chest x-rays), and a pretraining corpus of (image,report) pairs (i.e. MIMIC), reports are sorted by S-BERT embedding similarity to the concept. The list is truncated to a fixed number of the most similar ones (i.e 2k in the manuscript, but as few as 100 below). 
2. An LLM is used to predict if the report positively or negatively affirms that the concept is present in the image, and the output is used as a label. 
3. A linear classifier is trained given a balanced subsample of positive and negative instances for the concept

The data requirements are that of training a linear classifier on a fixed feature representation and so practically is very small. Furthermore, between the set of all classifiers that need to be trained, there may be correlations between which reports likely mention information relevant to their construction. 

In the paper, we train 150 concept classifiers, so this means that there would be a maximum of 2000 * 150 = 300k image-report pairs required for training.  Practically, the actual number of samples is much lower because of correlations, 50k. Below is an updated table containing the number of samples required to train concept classifiers. With as few as 100 reports /concept, requiring at most 15k samples (although practically 9k), performance decreases very moderately.  

We acknowledge that this level of detail was not clear from the original manuscript and is an important part of how the method works and so will expand it in any future revisions. We will include a small "algorithm" describing the process. 

|reports / concept | maximum samples | actual samples|Confounded|Unconfounded|Overall
| -------- | ------- | -------- | ------- | -------- | ------- |
|100 | 15,000 | 9,060 | 72.5 | 71.7 | 72.1 |   
|250	| 37,500 | 10,603 | 73.7|	72.1	|72.9| | | 
|500| 75,000 | 19,288 | 73.1|	73.0|	73.1| | | 
|1000|150,000 | 33,113 | 73.1|72.8|73.0| | | 
|1500|225,000 | 43,229 | 73.5|73.2|73.4| | | 
|2000|300,000 | 50,071 | 74.3|73.1|73.7| | |
Thank you for your response. I would keep my score.
Thanks for the reply! Here, we reorganize your questions and answer them as follows:

**Q1. About the examples per grounding functions, does this part correspond to lines 177-179?**

Yes. Sec 4.3 explains the method.

**Q2. how many grounding functions are used for each dataset, how they are selected, and how is the # of grounding functions confirmed?**

150 grounding functions are used for each modality. As explained in lines 717-720, we initially use our retrieve-augmented generation algorithm to generate 200 concepts. However, some concepts may not have enough examples in the corpus to support training the grounding functions. Therefore, we only select the top 150 grounding functions ranked by their concept prediction accuracy on the validation sets.

 **Q3. If you use 22k examples in x-ray, you should have (22k/2k=11) grounding functions? Is that different from the skin dataset?**

Each example can be reused to train multiple grounding functions, as each clinical report can contain information on different aspects. For instance, example A can be used as a positive example for concepts 1, 2, and 3 and serve as a negative example for concepts 7, 8, 9, etc. Therefore, considering the overlaps, we use 22k examples in total. The skin lesion dataset uses a similar number of examples.

**Q4. And why does each x-ray grounding function use 1,523 examples when you actually supply 2,000 examples per grounding function?**

We set 2000 as the upper bound for examples, but when training each binary grounding function, we want to balance positive and negative examples through subsampling. Therefore, the actual number of training samples is 2$\times$min(positive examples, negative examples), which is smaller than 2000.

**Q5. For the failure case, would it be possible to supply some failure examples of KnoBo in Fig. 7 (i.e., examples without the green checkmarks)?**

This is definitely possible, and we will include them in the next version.

Thanks again for raising those questions! We will clarify those details in the next version of the paper.
I just want to add on here as I believe RAG is quite an effective and reasonable measure to prevent hallucinations by far. Plus, the number/stats they reported in the rebuttal seem pretty convincing.
I thank the authors for their time and effort in preparing the rebuttal. The rebuttal resolved some of my questions, and I appreciate the explanation and revision for Figure 4 and the supplement of failure concepts in the attachment. I have some remaining questions: 

1. About the examples per grounding functions, does this part correspond to lines **177-179**? Could the author point out or supply how many grounding functions are used for each dataset, how they are selected, and how is the # of grounding functions confirmed? I tried but failed to find these details in the manuscript. If you use 22k examples in x-ray, you should have (22k/2k=11) grounding functions? Is that different from the skin dataset? And why does each x-ray grounding function use 1,523 examples when you actually supply 2,000 examples per grounding function? 

2. For the failure case, would it be possible to supply some failure examples of KnoBo in Fig. 7 (i.e., examples without the green checkmarks)? 

Thank you very much, and I am looking forward to your reply. Great work!
Thank you for resolving all my questions. Now I have no more questions. I have adjusted my ratings. Nice work!
Hi, 

Do you feel we have provided enough detail about possible extensions of our work to 3D? We will add some of this discussion to the limitations and future work section given extra space. Would you consider updating your rating or are there other weaknesses that we can discuss now during the discussion period?

Thanks!
Hi,

Has our rebuttal clarified sufficiently that you feel you can judge our work? Are there other aspects that we can clarify for you to make an assessment now that we have provided answers to your question?

Thanks!
Hi, 

Has our rebuttal addressed the weakness you felt the paper had? In particular, we strongly feel our method is not data-hungry and we have a fairly extensive discussion of problems (currently in the appendix) that could be moved up given extra space. Would you consider updating your rating or are there other weaknesses you feel we can address during this discussion period?
Thank you for your response.

The authors have adequately addressed my primary concerns, and I have no further questions. I will maintain my previous rating.