Debugging Concept Bottleneck Models through Removal and Retraining
Authors: Eric Enouen
sainyam galhotra
Venue: ICLR 2026 Poster (2026)
Keywords: concept bottleneck
prototypical part network
interpretability
human debugging
Primary area: interpretability and explainable AI
TL;DR: Human debugging of interpretable concept bottlenecks

Abstract: Concept Bottleneck Models (CBMs) use a set of human-interpretable concepts to predict the final task label, enabling domain experts to not only validate the CBM's predictions, but also intervene on incorrect concepts at test time. However, these interventions fail to address systemic misalignment between the CBM and the expert's reasoning, such as when the model learns shortcuts from biased data. To address this, we present a general interpretable debugging framework for CBMs that follows a two-step process of *Removal* and *Retraining*. In the *Removal* step, experts use concept explanations to identify and remove any undesired concepts. In the *Retraining* step, we introduce **CBDebug**, a novel method that leverages the interpretability of CBMs as a bridge for converting concept-level user feedback into sample-level auxiliary labels. These labels are then used to apply supervised bias mitigation and targeted augmentation, reducing the model’s reliance on undesired concepts. We evaluate our framework with both real and automated expert feedback, and find that **CBDebug** significantly outperforms prior retraining methods across multiple CBM architectures (PIP-Net, Post-hoc CBM) and benchmarks with known spurious correlations.

DEBUGGING CONCEPT BOTTLENECK MODELS THROUGH REMOVAL AND RETRAINING
Eric Enouen
Sainyam Galhotra
Cornell University
Coreell University
enouenêcs. cornell edd
sgêca www.elorolll..oooo
ABSTRACT
Concept Bottleneck Models (CBMs) set of numan-interpretable concepts to predict the final task label, enabling domain experts to not only validate the CBM's predictions, but also intervene on incorrect concepts test time. However, these interventions fail to address systemic misalignment between the CBM and the expert's reasoning. such as when the model learns shortcuts from biased data. To address this, we present general interpretable debugging framework for CBMs that follows a two-step process of Removal and Retraining. In the Removal step, experts use concept explanations to identify and remove any undesired concepts. In the Retraining step, we introduce CaDebug, a novel method that leverages the interpretability of CBMs as bridge for converting concept-level user feedback into sample-level auxiliary labels. These labels are then used to apply supervised bias mitigation and targeted augmentation, reducing the model's 's reliance undesired concepts. We evaluate our framework with both real and automated expert feedback, and find that CBDebug significantly outperforms prior retraining methods across multiple CBM architectures (PIP-Net, Post-hoc CBM) and benchmarks with known spurious correlations.
INTRODUCTION
Concept Bottleneck Models (CBMs) (Koh et al., 2020) have emerged as powerful architecture for interpretable vision classification. CBM consists of two stages: a concept extractor first predicts a set of human-understandable concepts, which are then passed to an inference layer to produce the final label. This intermediate representation allows domain experts to inspect the model's reasoning process and verify whether it aligns with their own. This capability is crucial in high-stakes domains, such as healthcare or scientific analysis, where errors are costly (Makary & Daniel 2016) and expert validation is essential (Lekadir et al. 2025).
Beyond passive validation, CBMs enable test-time interventions (Koh et al. 2020 Espinosa Zarlenga al. 2023). An expert review the predicted concepts and directly correct them influence the final prediction. For example, if radiologist corrects a mispredicted concept, such as marking a lesion as present when the model missed it, the diagnosis may shift from benign malignant. Such interventions elevate the expert from a passive auditor to an active participant in decision making. However, the effectiveness of these interventions hinges on a critical assumption: the learned concepts must align with expert knowledge. In practice, this alignment is often fragile. Data quality issues, sampling bias, and incomplete concept vocabularies can lead models to exploit spurious correlations or overlook key factors (Torralba & Efros) 2011] Geirhos et al., 2020). Because test-time edits address only surface errors, the same reasoning flaws inevitably reappear on new samples.
Existing approaches fall short of achieving reliable alignment and are prone to such issues. Supervised CBMs attempt to enforce alignment by sharing a concept vocabulary with annotators, but they require costly per-sample labels and remain vulnerable to concept leakage, which can obscure global misalignment (Havasi et al. 2022 Srivastava et al. 2024). Unsupervised CBMs (Chen et al. 2019 Debole et al.] 2025) reduce labeling demands by discovering concepts from data or leveraging foundation models, yet this flexibility increases the risk that the learned concept set diverges from expert understanding.

---

C
Refined Concept Bottleneck
C Sspur
Remove spurious concepts for task: Waterbird vs. Landbird
{Ø, h}
Retrain
h'
X, Y)
Cspur
(a) Removal. User removes undesired concepts Capur-
(b) Retraining.
Update encoder ó and simple layer / based on Capur-
Figure I: Our debugging framework for incorporating a domain expert's knowledge into a concept bottleneck. Removal (a): The user inspects concept explanations and selects undesired concepts to remove, such as background concepts bird classification. Retraining (b): The concept extractor and inference layer are retrained based on this feedback, updating the CBM remove dependence on undesired concepts while maintaining reliance on task-relevant concepts.
In this work, we present a general debugging framework for CBMs that enables experts to globally only accurate but
edit model's reliance undesired concepts, ensuring that its predictions also right for the right reasons (Ross et al.) (017), and aligned with the domain expert's reasoning. This framework follows two-step process of Removal and Retraining (Figure I
In the Removal step (Figure Ta experts evaluate concept explanations and eliminate those spuriously correlated with the label. For example, an ornithologist may remove background concepts unrelated to bird species (Rao et al., 2024). However, removal alone is insufficient: remaining concepts may still carry signals from the removed ones, and task relevnnt concepts may have been ignored in favor of spurious ones. To address this, we introduce Retraining step (Figure [b] that leverages expert feedback to guide the model toward an expert-aligned concept set.
To implement our retraining step, we propose CRDebug (Concept Bottleneck Debugger), which op- erationalizes interpretable debugging by treating expert feedback as causal intervention. CaDebug leverages the interpretability of CBMs as a bridge to first convert the expert feedback into sample-level auxiliary labels. Then, using the estimated auxiliary labels, performs reweighting and augmentation scheme approximate the counterfactual distribution where the undesired concepts have no effect on the label. In summary, we make the following core contributions:
We present interpretable debugging framework for CBMs, extending to general architec- ture and enabling domain experts to globally edit model reasoning.
We introduce CBDebug, retraining approach that first approximates sample-level auxiliary labels from concept-level feedback, then reweights and augments the dataset to reduce reliance on undesired concepts and better align the model with expert reasoning.
We validate framework across multiple CBMs (PIP-Net, Post-hoc CBM) and datasets with known spurious correlations. CBDebug most effectively leverages user feedback on spurious concepts, outperforming prior work ProtoPNets and improving worst-group accuracy by up to 26% over the original model, with strong results when feedback is automated with an LLM.
RELATED WORK
We review prior work concept bottleneck models, interpretable debugging, and bias mitigation, and position our approach at their intersection.
Concept Bottleneck Architecture. Concept bottleneck models (CBMs) Kohh et al. 2020) decompose prediction into a concept extraction stage and an inference stage. While supervised CBMs require concept annotations to ensure alignment with human-defined attributes, such labels are costly to obtain and cannot be assumed for spurious concepts. We focus on recent unsupervised CBMs, which learn concepts directly from data and allow experts discover and address undesired shortcuts.
2

---

These models fall into two main architectural families. ProtoPNets (Chen et al., 2019) and their extensions (Nauta et al.] 2023b; Ma et al. [2024] Carmichael et al. 2024). learn prototypical patches from the training set to make predictions. In this family, concept- explanations take the form of representative image patches from the training data. VLMMCBMs (Debole et al. 2025) use a CLIP backbone to score the presence of textual concepts, as explored in many approaches (Yuksekgonul et al. [2023] Oikarinen et al., 2023; Srivastava et al., (202)) where concept-level explanations are the textual descriptions of each concept. Our framework can also extend post-hoc XAI methods (e.g., interpreting neurons from class activation maps (Zhou et al., 2016) as concepts). In this work and evaluate one representative from each family: PIP-Net (Nauta et al. 2023b) and Post-hoc CBM (Yuksekgonul et al. 2023
we focus on concept bottlenecks, which are interpretable-by-dee
Interpretable Debugging. The goal of interpretable debugging is to enable a domain expert to interact with an interpretable model to detect and correct undesired behaviors. Early work focused on explanatory interactive learning (Ross et al.) 2017 Teso & Kersting 2019 [chramowski et al. 2020), with subsequent extensions to neuro-symbolic models (Stammer et al. 2022 2021). Bontempelli ct al. (2023) advanced these approaches, focusing on ProtoPNets and removing the need for fixed concept vocabulary. Building on top of these works, we focus on a generalized unsupervised CBM architecture (Poeta et al. 2023).
Within unsupervised CBMs, spurious concept removal has been explored in both ProtoPNets and VIM-CBMs: Nauta et al. (2023a) study removal in PIP-Net on medical datasets, while Yuksekgonul
et al. (203)) Rao et al. (2024); Chauhan al. (2023) evaluate similar strategies for VLMM-BMs..
Retraining efforts have focused mainly on ProtoPNets, such as adding object segmentation maps for extra supervision (Barnelt et al., 2021) or ProtoPDebug (Bontempelli al. 2023) which applies a forgetting loss on class-specific subsets of user-identified spurious concepts. Donnelly et al. (2025) targets task-specific spurious concepts and bypasses the need for retraining, but is limited to random patch selection for learning new concepts. For VLM-CBMs, |Bontempelli et al. (202T) outline debugging strategies but lack empirical evaluation, while et al. (2024) supports concept removal without the need for retraining through influence functions. We build on these directions with a general debugging framework for CBMs and an effective retraining approach, CBDebug, that leverages novel connection between interpretable debugging and bias mitigation to remove undesired task-specific concepts and better align models with expert feedback.
Bias Mitigation. Interpretable debugging and bias mitigation related but distinct. Debugging aligns a model with an expert's 's reasoning through explicit feedback, while bias mitigation aims to improve robustness by reducing reliance on spurious correlations. We further explore connections to two main groups of bias mitigation: supervised methods, which incorporate auxiliary labels, and unsupervised methods, which estimate spurious correlations directly from data.
Supervised methods require auxiliary labels to reduce the impact of spurious correlations (Zheng
& MMkar 2022 Sagawa et al., 2019 Makar et al. 2022 Kirichenko et al., 2022). While these approaches cannot be directly applied to unsupervised CBMs, CBDebug utilizes a crucial connection between interpretable debugging and these methods. By leveraging the model's ' interpretability. we can bridge concept-level human feedback to sample-level auxiliary labels, effectively removing the reliance on any concept marked as undesired by the domain expert. This is accomplished by collecting the activation scores for each concept on every sample, which can then be used to form the auxiliary labels. Specifically, we instantiate our approach with permutation weighting (Arbour et al. 2021), which can handle high- bigh--diensional,, real-valued auxiliary labels. This allows to use concept activations directly, unlike methods such as GroupDRO (Sagawa et al., 2019) that require discrete groups and would necessitate additional clustering step to convert activation scores.
Unsupervised methods have similarly been proposed that relax the requirement for auxiliary labels.
These approaches either automatically estimate spurious groups (Sohoni et al. 2020 Seo et al. 2022
Chakraborty et al.) 2024) or reweight samples based on assumptions about how spurious correlations
learned during training (Liu et al. [022]] Nam et al. 2020 [Espnosaa Zarlenga et al. 2024 Wu ct al., 2023). Instead of relying on underlying assumptions about model training dynamics, our method focuses only on removing concepts marked directly by a domain expert. While the two approaches can overlap, interpretable debugging offers a distinct and complementary advantage: it gives the expert fine-graine control over what should be removed. For example, an expert may wish to keep certain 'spurious' concepts if they know they will perform well in practice, or may wish to remove 'core' concepts to debug what the model would use instead, This ensures the resulting model
3

---

is aligned with the domain knowledge of the expert, which is critical for interpretable models that part of hwwan-mmchhaei team. Further discussion and results can be found Appendix C.44
3 CONCEPT BOTTLENECK DEBUGGING FRAMEWORK
In this section, we formalize our interpretable debugging framework, for leveraging expert feedback to eliminate undesired concepts and aligning the model's reasoning with the expert's preferences. We first define the general class of concept bottleneck models our framework supports. Then, we outline our two-step debugging process: how concept-level user feedback is collected during the removal step and the formal goal of the retraining step.
3.1 CONCEPT BOTTLENECK
We denote a concept bottleneck model as pair {á, h). The concept extractor & X + R maps an input x E X' a vector of m concept activation scores, and the inference layer I + y, typically sparse linear layer, maps these activations to the output label. The core requirement for concept bottleneck is that each concept has a corresponding tuman-interpretable explanation, and we
review recent work that falls under this definition in Section 2 (Concept Bottleneck Architecture).
Specifically, we focus on unsupervised CBMs, which eliminate the need for auxiliary labels and are thus applicable to broader range of real-world settings. These models enable automatic concept discovery, making them scalable, but they are also prone to learning concepts that are entirely spurious irrelevant from the perspective of domain expert. Our framework is particularly well-suited to address this challenge, as it provides a mechanism for experts to inspect, debug, and guide the concepts learned by these scalable models. Furthermore, standard CBMs are not immune to shortcut learning: unsupervised CBMs simply make these shortcuts more explicit, rather than hidden among other concepts, enabling users identify and remove them.
3.2 REMOVAL
We focus on the removal of concepts that encode biases undesirable for the classification task, as identified by the domain expert. For instance, when classifying birds, the expert may wish to remove confounded concepts that capture background information rather than features of the birds themselves. Our framework does not assume specific structure representation for the underlying concepts. Instead, it operates in general setting where each concept is associated with an explanation, allowing our method to be applied across a variety of concept discovery approaches.
To guide concept removal, we adopt simple binary feedback mechanism that assumes spuriosity is defined at the task level: each concept either retained marked for removal, based on expert input. This minimal supervision design ensures that our approach remains broadly applicable and easy to integrate into real-world workflows, where experts may have limited time or domain knowledge to provide detailed annotations, but does not directly account for class-specific spurious concepts.
We illustrate our Removal process in Figure la We assume a trained CBM {Ø, h} with learned concept set C = (ct,.... Cm A domain expert inspects the learned concept set by interacting with the concept explanations and identifies a subset Capur C C to remove. We remove all concepts in Cspur from the concept set. Then, pass the edited CBM (ó. h] and Capur to the retraining step.
3.3 RETRAINING
There are two main failure modes when removing Cspar from C. First, any remaining concepts partially encode information from the undesired concepts, removal alone may not fully climinate their influence. Second, the model relied too heavily on the undesired concepts, removal can leave the model unable to perform well. These limitations motivate the need for retraining, which adapts the CBM to maintain high performance while avoiding reliance on the marked undesired concepts.
Having obtained human feedback in the forn of set of concepts Cepur deemed spurious for the task, we face crucial challenge: how can we most effectively use this feedback to improve the model's reasoning and performance?
4

---

C eppur
X
Y
Sppar
X
Label
Reweight
Augment
X aug
U
Y
Ocean: 1.7
Ocean: 1.7
F 0.96
Retrain
Coourr
Beach: 3.3 y == Waaterbirrd
0.6
Beach: 3.3
Tree: 0.9
Tree:
g 0.04
H',h' h!
@Copur
Grass: 2.4
> 9.4
Grass: 2.4
Y= "Waterbird
Figure 2: Overview of CBDebug (Concept Bottleneck Debugger), which consists of three main steps. First, the encoder Φ computes the concept activations for undesired concepts Cepur to generate the approximated auxiliary label V. Second, permutation weighting utilizes V and the class label Y to compute the odds of the sample being drawn from the unconfounded distribution, generating weights U. Third, augmentation is performed X based on the undesired concepts Cepur and weights U to generate Xaug. Finally, we retrain {ø, h on (Xaug: Y) weighted by and return {Ø', h'}.
We illustrate our Retraining process in Figure Tb The retraining algorithm is given the trained concept bottleneck {ø, h), the training dataset (X. )), and the set of undesired concepts Cspar. The goal of this step is to return an updated concept bottleneck {Ø', h'} that maintains high task performance by leveraging other, more task-relevant concepts instead of Cspur.
4 CBDEBUG: CONCEPT BOTTLENECK DEBUGGER
To achieve this goal, we introduce CBDebug (Figure 2 which operationalizes interpretable debug- ging by treating expert feedback as a causal intervention. Intuitively, CRDebug treats undesired concepts observed confounders and approximates the counterfactual distribution where those -n-thhe--ooooppp approach allows experts to explicitly
confounders have no effect on the label. This select or refine the set of undesired concepts, giving them transparent mechanism to steer the model, unlike methods that rely on unsupervised group discovery (see Bias Mitigation in Section
In practice, CBDebug consists of three main stages:
Couvert concept-level feedback into sample-level auxiliary labels by scoring each sample with the CBM. The activation scores V approximate the true auxiliary labels V for undesired concepts.
Label (Section 4.1):
Apply permutation weighting (Arbour et al. 2021 on (V, Y) to compute sample weights U that approximate the unconfounded distribution.
Reweight (Section 4.2
Augment (Section 4.3 Use the sample weights to selectively augment bias-aligned samples, yielding dataset Xang that further reduces shortcut reliance.
We then fine-tune the concept bottleneck with the augmented dataset (Xaug: Y) weighted by sample weights U and return the refined concept bottleneck {Ø/, h/}.
4.1
LABEL
The first stage of our approach (Label in Figure generates sample-level auxiliary labels from the expert's feedback. For example, user may mark background concepts like 'beach' 'grass' as undesired for the task of classifying birds. We then utilize the trained CBM's concept extractor to collect the activation scores for the marked concepts on the entire training dataset.
Formally, the labeling step takes the trained concept extractor Φ, the training samples X, and the spurious concept set Capur as input. returns V, a matrix of dimension x (Capurl, where the number of samples.
V
[pc (x:)) )
5

---

where (xx;)) are the concept activation scores of all concepts in C for sample 24, and @Copur (2:4) denotes the subselection of those concept activation scores for only the concepts in Capur.
4.2 REWEIGHIT
The second stage of our approach (Reweight in Figure 2) utilizes supervised bias mitigation approach to reweight the training dataset to reduce the correlation between V and Y. For example, if backgrounds spuriously correlate with class label, the reweighting scheme may assign low weight to waterbird on 'beach' background and a high weight to waterbird on a 'grass" background, forcing the model learn features that generalize beyond the undesired background concept.
We adopt permutation weighting (Arbour et al. 2021. later applied to shortcut removal by Zheng Makar p022), to perform reweighting. Unlike group-based reweighting approaches such as GroupDRO (Sagawa et al. 2019), which require discrete group labels and often struggle with groups that have low support (necessitating clustering of V), permutation weighting naturally accommodates multi-dimensional, continuous auxiliary labels. By directly enforcing independence between V and Y. it provides a more general and stable mechanism.
Formally, the reweighting step takes the approximated auxiliary labels V and the class labels Y as input. It returns sample weights U.
Given V and Y, we first construct two datasets. A dataset D is constructed as V concatenated with Y representing the confounded distribution, where there exists a correlation between the label Y and auxiliary label V. Then, create dataset D' by randomly permuting the label Y in the original dataset. This naturally breaks any correlation between and V, representing the unconfounded distribution.
We then train a binary predictor 1) Y x V - fo, 1} to predict the probability of a sample belonging to the unconfounded dataset D' compared to the confounded dataset. Finally, compute weight u; for each sample
17(35,, vi)
U2 7733/1 Uk))
where (1k, 0;) denotes the estimated probability that a sample belongs to D'. To ensure robust weights, we perform K-fold cross validation and average over multiple permutations. See Ap- pendix C. for further analysis of effect of the number of folds and number of permutations on assigned sample weights per subgroup.
4.3 AUGMENT
The third stage of approach (Augment in Figure 2 aims to further reduce the correlation between V and Y through augmentation. While reweighting is effective, it can lead to unstable training when the spurious groups are highly imbalanced, weights. Augmentation offers a more robust way to mitigate bias in these scenarios by generating new samples for underrepresented groups. For example, we augment the image of waterbird on water background with an image of bamboo from the concept bank, while leaving the image of a waterbird on a land background untouched.
few samples are given very large
Formally, the augmentation step takes the training samples X, sample weights U, and the concept Capur as input. It returns new training samples Xaug that further reduce the correlation between undesired concepts and the class label. Importantly, because these concepts were explicitly marked as undesired by the user, we do not change the label Y.
Samples assigned a low weight are more likely to be aligned with the bias we want to remove, and so we would like to focus our augmentation on these samples. To accomplish this, we can convert each sample weight u; into augmentation probability (zng(I;). We first invert the sample weight u; by subtracting each weight from the maximum sample weight assigned. Then we normalize the resulting values to [0, 1] to convert them into probabilities, and raise the probabilities to power to increase contrast and reduce the likelihood augmenting useful samples. See AppendixC.3] for further analysis the impact of y on augmentation probabilities and downstream performance.

---

Table I: Average and Worst-Group Accuracy on MetaShift and Waterbirds with PIP-Net and Post-hoc CBM. Best in bold, second best underlined. Average and standard deviation reported over the three initial seeds for Original and over the six debugging sessions for removal and all retraining approaches. CBDebug consistently improves worst-group accuracy across models and datasets.
Method
PIP-Net
Post-hoc CBM
Waterbirds
MetaShift
Waterbirds
MetaShift
Average
Worst
Average
Worst
Average
Worst
Average
Worst
Original
92.3+0.3
71.942.7
80,9
52.4
63,5
25.843.0
92,9+0.
84,5+22
Remove
92.6+0.4
74.4
81 41..6
55.0+2.6
61.2+18.8
13.9
89.0+4.
73.9
Retrain
92.4
72.5+1.0
81..+1..
53.3
66.9
33.2
93.1
84.4
ProtoPDebug
10.5
71.611.9
80.9
52.4
Our
Reweight Only
93.210.4
74.214.
81.8114
56.111.33
80.0
55.6115.2
93.1
87.3
Augment Only
92.4+0.6
75.5+2.9
82,2+1.7
55.643.3
64.5+448
25.9+11.4
92.6+1.7
86.3
CBDebug
93.7+0.7
79.4+4.3
82.3+1.7
57.3
73.646.3
51.9+66.2
93.4+1.0
89.3+1.8
We then augment each sample with probability (xa). Our augmentation strategy depends on the concept representation:
ProtoPNets: Since ProtoPNets operate prototypical patches, first randomly select i spurious concepts from Cepur and perform CutMix (Yun et al. (019)) for each selected concept. To determine
the actual patch used, we randomly choose from the top-ten highest-activated prototypical patches for that concept, following Nauta et al. (2023b)ss is approach for visualizing concepts.
VI.M-CMs: Since VLMM-BMs use text-based concepts, we leverage text-to-image- -o----iigg-ggggertteedd concept bank, following DISC (Wu ct al. 2023). We randomly select a single spurious concept from Cspur and perform Mixup (Zhang 2017) with image of that concept randomly sampled from the concept bank.
This approach ensures that bias-aligned samples are more likely to be augmented, creating richer training distribution that reduces spurious correlations without altering the true labels.
5 EXPERIMENTS
To evaluate our approach, we aim to answer the following questions: Q1: Quantitatively, how does CBDebug perform on both real [Section5..] and automated (Section feedback sources? Q2: Qualitatively, does CBDebug effectively remove dependence on undesired concepts and lead to more robust concept (Section 5.3 We also explore additional ablations of our method and comparisons to unsupervised baselines in Appendix C
Datasets. We datasets with known spurious correlations: Waterbirds (Sagawa et al. 2019).
MetaShift (Liang & Zou| 2022), CelebA (Liu et al., 2015), and ISIC (Codella et al.) 2019). These datasets provide concrete testbeds for assessing how well CBDebug reduces reliance undesired concepts, as their group structures allow performance to be measured directly across subpopulations.
Models. We evaluate representative ProtoPNet (PIP-Net (Nauta et al. 2023b)) and VLM-CBM (Post-hoc CBM (Yuksekgonul et al. 2202)) on these datasets. PIP-Net uses a ConvNeXt-tiny backbone, while Post-hoc CBM uses CLIP-ViT-L-Ie backbone for all datasets except ISIC where BioMedCLIP (Zhang et al. 2023) is used, and both are trained following the authors' original implementation. For Post- hoc CBM, we use combination of synthetic concepts from Wu al.) (2023) and curated high-quality concepts
DOikarinn et al. (2023) (Appendix B Each model trained with three random seeds per dataset, and we report average- and
accuracy averaged across seeds (see Appendix A.I A for additional training details). For ISIC we follow Wu ct al. (2023) and report test AUROC since there are 2 distinct groups
Setup. After training the original models, we collect feedback from real automated domain expert to identify spurious concepts (Appendix A.2).. Since the expert feedback aligned with the known spurious correlations, utilize the robustness of the model measure for the effectiveness of each retraining algorithm. For PIP-Net, we fine-tune the entire model for half the original training epochs. For Post-hoe CBM, we freeze the backbone and retrain only the linear layer.
7

---

Table Average and Worst-Group Accuracy for Automated Feedback on Post-hoc CBM. Average and standard deviation reported over the three initial seeds. CBDebug consistently outperforms the original model and standard retraining.
Method
Waterbirds
MetaShift
CelebA
ISIC
Average
Worst
Average
Worst
Average
Worst
AUROC
Original
63.5
25.843.0
92.940.4
84.5
76.240.8
8.7
39.3
Remove
64.6
2.541.1
90.5
79.6
19.949.1
6.5
41.7+16.9
Retrain
69.0.2.2
38.0+5.5
92.4
83.0
79.9
22.2
37.7
Ours
Reweight Only
10.0
61.9 15.8
9..
84. #5.2
73.9
53.3
52.6
Augment Only
67. 27.4
32.9
92.0
84.4
71.5
38.9
18.6
CBDebug
76.0
58.346.0
93.0
87.5
68.744.1
51.3
58.0
Baselines. We compare C3Debug against the following baselines (Appendix AA3):
Removal. Removes undesired concepts without further retraining.
Retraining, Takes the model after removal and fine-tunes on the training dataset, following standard fine-tuning protocol.
ProtoPDebug (Bontempelli al. 2023). Collects image patches in input space representing undesired concepts into a forget set, penalizes the encoder for activating on forget set patches.
Seweight/Augmen Only. These ablations evaluate our main components in isolation: the Label and Reweight step (without augmentation) and the Augment step (without reweighting).
5.1 CAN CBDEBUG EFFECTIVELY RETRAIN BASED ON USER FEEDBACK?
To answer this question, we run debugging sessions with six real users. Each user performed the removal step for each of the four tataset-moded combinations evaluated in Table (Appendix (A.2). For both models, users are instructed select spurious concepts. For PIP--Nt, users are shown the top ten most activated patches from the training dataset and optionally three example images showing which patch the concept activates on. Post-hoe CBM, users are shown the full set learned concepts and they can select concepts that seem spurious for the task. We fine-tune according each user's 's feedback on each model, making each session an end-to-end debugging run.
[
Our results shown in Table For PIP-Net, our removal baseline provides modest boost to worst-group accuracy, improving performance by 2.5% on Waterbirds and 2.6% on MetaShift. In contrast, for Post-hoc CBM, removal substantially reduces worst-group accuracy. We hypothesize that this is because Post-hoc CBM's more limited concept set (roughly 10-30) causes it to ignore other zask-relevant concepts in favor of the dominant shortcut, making it more sensitive removing bias-aligned concepts than PIP-Net, which learns far more concepts (around 100-200) (Appendix A.2)). For both models, there remains significant gap between the worst-group and average-gron group accuracy, indicating that spurious correlations were not fully eliminated. While concept removal can yield incremental improvements, it cannot by itself encourage the model to discover new, robust concepts and is insufficient for fully addressing shortcut reliance.
Baselines.
The Retrain baseline further illustrates this point. For PNP-Net, it performs worse than Removal, while for Post-hoc CBM, it improves performance Waterbirds but not on MetaShift. These results suggest that even when spurious concepts are explicitly removed, retraining on the biased dataset can cause the same correlations leak back into the model's representations, highlighting the need for more targeted retraining approach.
CBDebug. In contrast to these naive baselines, CBDebug improves worst-group accuracy by 7.5% on Waterbirds and 4.9% on MetaShift for PIP-Net, and by 26.1% on Waterbirds and 4.8% on MetaShift for Post-hoc CBM. CBDebug surpasses the previous state-of-the-a -hte interpretable debugger, ProtoPDebug, while integrating its component steps into a framework that delivers more stable gains across settings.
8

---

Table Top five concepts for Post-hoc CBM before retraining, after retraining normally, and after retraining with CBDebug. Retrain learns new background concepts (highlighted in blue and green) to replace the ones removed. CBDebug effectively removes background concepts, replacing them with more robust concepts.
Class
Original
Retrain
CBDebug
hooked scabird beak
beach
duck-like body
sea
gull-like body
hooked scabird beak
Waterbird
harbor
water
orange wings
lake
hooked seabird beak
orange eyes
gull-like body
duck-like body
orange nape
olive crown
olive upper tail
olive upper tail iridescent
tree-clinging-like body
bamboo
Landbird
forest
green primary color
blue upper tail olive crown
olive upper tail tree
tree-clinging-like hody
olive breast
hawk-like body
(a) Concepts before retraining
(b) Concepts after retraining with CBDebug
Figure 3: The six most highly activated concepts for the Original model trained on Waterbirds and the model after retraining with CRDebug. CBDebug removes both concepts representing bamboo from the concept set and replaces them with more robust concepts representing bird features.
These consistent improvements across architectures and datasets highlight CBDebug as a reliable and effective method for debugging based on real user feedback. We additionally compare CBDebug to unsupervised group-robustness methods in Appendix C.4] which rely on assumptions about how spurious correlations are learned, showing the benefit human-guided debugging for performing bias mitigation.
5.2 CAN CBDEBUG EFFECTIVELY RETRAIN BASED ON AUTOMATED FEEDBACK?
As our framework incorporates a domain expert in the loop, a natural question is whether this feedback can be automated with recent advances in foundation models. To explore this, we use LLMs to provide automated feedback for Post-hoc CBM on Waterbirds and MetaShift. Automation reduces both human effort and cost, enabling us to further extend experiments to CelebA (Liu et al., 2015) and ISIC (Codella et al.] (011)). For Post-hoc CBM, the automated "user" provides a binary judgment on the spuriosity of each text-based concept. We set the temperature to zero to ensure deterministic responses. Additional details on the prompt design and comparisons to real user feedback can be found in Appendix A.2
Baselines. As shown in Table 2) while the Reweight Only baseline achieves superior worst-group
performance on Waterbirds and CelebA, its results are less stable across datasets, underperforming MetaShift. Similarly, the Removal method proves highly volatile on ISIC and MetaShift, where occasionally performs well but frequently collapses below the original model's performance.
CBDebug. In contrast, CBDebug offers more reliable and robust solution, consistently outper- forming the original model across all tested benchmarks, with gains of up to 42.6% over the original model on CelebA.

---

5.3 DOES CBDEBUG EFFECTIVELY REMOVE DEPENDENCE ON UNDESIRED CONCEPTS?
We visualize the concepi bottleneck before and after retraining with CBDebug on Waterbirds to better understand the impact of approach the concept set. For Post-hoc CBM, results in T that while baseline retraining still finds new spurious correlations to replace the removed ones, CBDebug effectively removes these concepts from the representation and replaces them with task-relevant concepts. Similarly, Figure shows that for PIP-Net, two land concepts that previously dominated the predictions were effectively removed and replaced with more robust bird concepts. In both scenarios, CBDebug effectively removes dependence on spurious attributes. Additional visualizations are provided in Appendix C.2
CONCLUSIONS
address the problem misalignment between a model's behavior and domain expert reasoning, often caused by shortcuts from biased data. We propose general interpretable debugging framework and introduce CBDebug, which leverages the interpretability of the model to convert high-level concept feedback into sample-level labels. Empirical results show that CBDebug outperforms prior retraining methods across multiple CBMs, datasets, and both real and automated feedback sources.
7
LIMITATIONS
While CBDebug demonstrates consistent improvements across datasets and model architectures, several limitations remain. First, observe substantial variance in performance metrics, particularly when using Post-hoc CBM as the backbone, where results are highly sensitive to initialization and training stochasticity. Although CBDebug reliably improves performance across settings, the effect sizes should be interpreted with this variability in mind.
Second, CBDebug's effectiveness depends the quality and consistency of concept feedback: inaccurate or adversarial feedback can undermine performance gains. Finally, our framework assumes task-level spurious concepts and therefore does not directly mitigate class-specific spurious correlations, which may limit applicability in settings where spuriosity varies across classes.
8 ACKNOWLEDGEEMENTS
This research was supported by a gift to the PalkedIn-Coredd Bowers Strategic Partnership. ARO grant W91 INF-25-111111 1-0254, BSF grant 2024101 and grant from Infosys. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsors.
10

---

REFERENCES
David Arbour, Drew Dimmery, and Arjun Sondhi. Permutation weighting, In International Confer- ence on Machine Learning, pp. 331-341 PMLR, 2021.
Alina Jade Barnett, Fides Regina Schwartz, Chaofan Tao, Chaofan Chen, Yinhao Ren, Joseph Y Lo, and Cynthia Rudin. case-based interpretable deep learning model for classification mass lesions in digital mammography. Nature Machine Intelligence, 3(12):1061-1070 2021.
Andrea Bontempelli, Fausto Giunchiglia, Andrea Passerini, and Stefano Teso Toward unified framework for debugging concept-based models. arXiv preprint arXiv:2109.11106 2021.
Andrea Bontempelli, Stefano Teso, Katya Tentori, Fausto Giunchiglia, Andrea Passerini, et al. Concept-level debugging of part-prototype networks. In Proceedings of the The Eleventh Interna- tional Conference on Learning Representations (ICLR 23). ICLR 2023, 2023.
Zachariah Carmichael, Suhas Lohit, Anoop Cherian, Michael Jones, and Walter J Scheirer. Pixel- grounded prototypical part networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4768-4779, 2024.
Rwiddhi Chakraborty, Adrian Sletten, and Michael C Kampffmeyer. Exmap: Leveraging explainabil- ity heatmaps for unsupervised group robustness spurious correlations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12017-12026, 2024.
Kushal Chauhan, Rishabh Tiwari, Jan Freyberg, Pradeep Shenoy, and Krishnamurthy Dvijotham. In- teractive concept bottleneck models. In Proceedings of the aaai conference on artificial intelligence, volume 37, pp.
2023.
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019.
Jihye Choi, Jayaram Raghuram, Yixuan Li, Suman Banerjee, and Somesh Jha. Adaptive concept bottleneck for foundation models. In ICML 2024 Workshop Foundation Models in the Wild.
Noel Codella, Veronica Rotemberg, Philipp Tschandl, Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv preprint wwXiv::/02.033355 2019.
Nicola Debole, Pietro Barbiero, Francesco Giannini, Andrea Passerini, Stefano Teso, and Emanuele Marconato. If concept bottlenecks are the question, are foundation models the answer? arXiv preprint arXiv:2504. 19774, 2025.
Jon Donnelly, Zhicheng Guo, Alina Jade Barnett, Hayden McTavish, Chaofan Chen, and Cynthia Rudin. Rashomon sets for prototypical -part networks: Editing interpretable models in real-time. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 4528 4538, 2025.
Mateo Espinosa Zarlenga, Katie Collins, Krishnamurthy Dvijotham, Adrian Weller, Zohreh Shams, and Mateja Jamnik. Learning receive help: Intervention aware concept embedding models. Advances in Neural Information Processing Systems, 36:37849-37875, 2023.
Mateo Espinosa Zarlenga, Swami Sankaranarayanan. Jerone TA Andrews, Zohreh Shams, Mateja Jamnik, and Alice Xiang. Efficient bias mitigation without privileged information. In European Conference on Computer Vision, pp. 148-166. Springer, 2024.
Robert Geirhos, Bethge, and Felix Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 1):6656673, 2020.
Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias
Marton Havasi, Sonali Parbhoo, and Finale Doshi- Velez. Addressing leakage in concept bottleneck models. Advances in Neural Information Processing Systems, 55:23386-2339) 2022.

---

Lijie Hu, Chenyang Ren, Zhengyu Hu, Hongbin Lin, Cheng-Long Wang, Zhen Tan, Weimin Lyu. Jingfeng Zhang, Hui Xiong, and Di Wang. Editable concept bottleneck models. In Forty-second International Conference Machine Learning, 2024.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.0299 2022.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pp. 5338-5348. PMLR, 2020.
Karim Lekadir, Alejandro F Frangi, Antonio R Porras, Ben Glocker, Celia Cintas, Curtis P Langlotz, Eva Weicken, Folkert W Asselbergs, Fred Prior, Gary S Collins, et al. Future-ai: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare. bmj, 388, 2025.
Weixin Liang and James Zou. Metashift: a dataset of datasets for evaluating contextual distribution shifts and training conflicts. In International Conference on Learning Representations, 2022.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11976-11986, June 2022.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Chiyu Ma, Jon Donnelly, Wenjun Liu, Soroush Vosoughi, Cynthia Rudin, and Chaofan Chen. Interpretable image classification with adaptive prototypebbssed vision transformers. arXiv preprint arXiv:2410.20722. 2024.
Maggie Makar, Ben Packer, Dan Moldovan, Davis Blalock, Yoni Halpern, and Alexander D' Amour. Causally motivated shortcut removal using auxiliary labels. In International Conference Artificial Intelligence and Statistics, pp. 739-766. PMLR, 2022.
Martin A Makary and Michael Daniel. Medical error-the the third leading cause of death in the us. Bmj, 353, 2016.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jacho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 04220673-20684 2020.
Meike Nauta, Johannes H Hegeman, Jeroen Geerdink, Jörg Schlötterer, Maurice van Keulen, and Christin Seifert. Interpreting and correcting medical image classification with pip-net. In European Conference on Artificial Intelligence, pp. 198 215. Springer, 2023a.
Meike Nauta, Jörg Schlötterer, Maurice Van Keulen, and Christin Seifert. Pip-net: Patch-based intuitive prototypes for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2744-2753, 2023b.
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui- Wei Weng. Label-free concept bottleneck models. In The Eleventh International Conference on Learning Representations, 2023.
Elconora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. Concept- based explainable artificial intelligence: A survey. CoRR, 2023.
Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-nane Task-agnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision, pp. 444-461. Springer, 2024.
12

---

Andrew Slavin Ross, Michael C Hughes, and Finale Doshi- Right for the right reasons: Training differentiable models by constraining their explanations. arXiv preprint arXiv: 103.03777.. 2017.
Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint 1tX.v:/91/..07700 2019.
Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Georg Luigs, Anne-Katrin Mahlein, and Kristian Kersting. Making deep neural networks right for the right scientific reasons by interacting with their explanations. Nature Machine Intelligence, 2(8):476-486, 2020.
Seonguk Seo, Joon- Young Lee, and Bohyung Han. Unsupervised learning of debiased representations with pseudo-attributes. attributes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2022.
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse- coarse-grained classification problems. Advances in Neural Information Processing Systems, 33: 19339939-10 19339-19352, 2020.
Divyansh Srivastava, Ge Yan, and Lily Weng. Vlg-cbm: Training concept bottleneck models with vision-language guidance. Advances in Neural Information Processing Systems, 37:79057-79094, 2024.
Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3619-3629, 2021.
Wolfgang Stammer, Marius Memmel, Patrick Schramowski, and Kristian Kersting, Interactive disen- tanglement: Learning concepts by interacting with their prototype representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10317-10328, 2022.
Stefano Teso and Kristian Kersting. Explanatory interactive machine learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 239 -39-245, 2019.
Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In CVPR 2011, pp. 1521-1528. IEEE, 2011.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
Shirley Wu, Mert Yuksekgonul, Linjun Zhang, and James Zou. Discover and cure: mitigation of spurious correlation. In International Conference on Machine Learning, pp. 37765- 37786. PMLR, 2023.
Concept-aware
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In The Eleventh International Conference on Learning Representations, 2023.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo.
Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6023-6032, 2019.
Hongyi Zhang. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 170.00/11 2017.
Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint 203.v:2303.00915, 2023.
Jiayun Zheng and Maggie Makar. Causally motivated multi-shorteut identification and removal. Advances in Neural Information Processing Systems, 35:12800- 12212, 2022.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921-2929, 2016.
13

---

A EXPERIMENTAL DETAILS
A.1 INITIAL TRAINING
We evaluate two models, each representative of a major family of concept bottlenecks:
PIP-Net (Nauta al. 2023b): A patch-base mutch-based concept bottleneck that learns prototypes from the training data using self self-supervised losses to make predictions.
Post-hoc CBM (Yuksekgonul et al. 2023 A text-based concept bottleneck that maps the embed-
ding space of a model to similarity scores to textual concepts with concept activation vectors or CLIP embeddings.
We evaluate these two models on three popular subpopulation shift benchmarks:
2019): A synthetic dataset where the background (water vs. land) acts as the spurious attribute. contains 4795 training samples.
Waterbirds (Sagawa et al.
MetaShift (Liang & Zou 2022): A natural dataset where the spurious attribute is the scene context (indoor vs. outdoor). We use version derived from the COCO dataset rather than Visual Genome, containing 2738 training samples.
) A face attribute dataset where gender (female male) serves as the spurious attribute. It includes 162770 training samples.
CelebA (Liu et al.
ISIC (Codella et al. 2019): A medical imaging dataset for skin lesion classification into benign or malignant. The spurious attributes are 'dark corners', 'hair', 'gel borders', 'gel bubbles', 'ruler', 'ink markings/staining! and 'patches'. We follow the setup from DISC (Wu et al. 2023) with five different splits testing reliance on spurious features.
All experiments were conducted on compute node with 1 112 CPU cores, 1 TB of RAM, and 2 2xNVIDIA RTX 6000 Ada GPUs. the next three sections, we explain the experimental details for the components of approach. Section[A.] describes how train the original models, Section A.2 describes our user or automated debugging of the original models, and Section A.3 describes our retraining approaches based on the user feedback.
In our experiments we first train each model with three random seeds. For both models we utilize the hyperparameters recommended in their work.
For PIP-Net, we utilize a ConwNeXt-tiny (Liu et al., 2022) backbone. We pretrain for epochs, then train in the second stage for 60 epochs. We utilize batch size of 128 for pretraining and 64 for training and a learning rate of 0. 0.0005 for the backbone, and 0.05 for the linear layer. For Post-hoc CBM,
CLIP-ViT-L--1 backbone. The first step is to initialize concept bank, which
describe in detail in Section
B
We utilize a Asparse 0.02 for all datasets.
A.2 USER DEBUGGING SESSIONS
In this section detail our user study. We run six different debugging sessions with computer science graduate students. Each debugging session consisted of the user marking concepts spurious on four different tasks: Waterbirds PIP-Net, Waterbirds Post-hoc CBM, MetaShift Pet-Net. MetaShift + Post-hoc CBM).
We first show the task description given to study participants, and then provide examples of the user interface for selecting concepts spurious or not.
Before beginning our scall-scale user study, we sought IRB guidance. As the first step, we contacted the IRB office affiliated with the authors' institution, providing description of our planned study design to determine the appropriate next steps. The compliance assistant responded that, because the research focused the debugging method and did not involve collecting any user information, "I can confirm based on the information you't ve provided that we would not consider this project to meet the regulatory definition of human participant research, and therefore you do not need to submit an application to conduct the work as you have described it." Based on this determination, we did not proceed with formal application.
Participants were computer science graduate students who voluntarily chose to take part in the debugging sessions. The study not part of course requirement, and participation not tied
14

---

to grades, credit, other obligations. No personal data sensitive information collected, and the activities involved brief, task lask-focused feedback on visual model explanations. Participation was entirely voluntary, and compensation provided. Participants were informed of the study's purpose and that their contributions would be used a research paper.
User Study Task Description
this study, you will help improve two state-of-the-an interpretable vision classification models: PIP-Net and Post-hoc CBM. These models aim to explain their predictions using human-underrsandabble concepts.
However, these interpretable models still sufter from shorteut learning, where they latch to spurious correlations that wolves that mistakenly learns to associate the presence of snow in the background with the wolf class, because most training images of wolves happened to include snowy scenes.
hold robustly the real world. A classic example a model trained to recognize
In this study, give you the opportunity to improve these models by identifying and removing such spurious concepts.
MODELS
PIP-Net learns visual concepts. will be shown visual features model has identified as important. Each concept has both its top- 10 image patches visualized as well as three images where this prototype is marked as active that can he optionally viewed. Mark concepts that do not focus the correct object for the classification task.
Post-hoc CBM uses text-based concepts. We' 'e seeded its concept bank with some potentially spurious candidates in addition to the core concepts, and you' see which concepts the model relied on. Mark those that seem irrelevant non-causal for the prediction.
DATASETS AND TASKS
You will perform this analysis across three datasets:
Waterbirds Classify images as either a waterbird (e.g. Albatross, Auklet, Gull) or landbird (e.g.
Woodpecker, Hummingbird, Warbler).
MctaShift Classify images as either dog
TASK DETAILS
For each dataset, you will:
1. Review the concepts learned by each model and how they relate to the prediction labels.
Flag any concepts you believe are misleading, spurious, or unrelated to class being predicted.
You will repeat this process for both models. Your input will help teach the model which concepts to unlearn to build
robust concept set.
We also automate the feedback for Post-hoc CBM with large language model, GPT-3.5-turbo. show below the task description prompt used, with the specific baasificttio__tak__deesrrppiplp dependent on the dataset being used.
Automated User Study Task Description
You are helpful assistant that classifies visual concepts either SPURIOUS or NOT SPURIOUS. The classification task is: (classification_ stansatttakkaaeeaa bisk_description) concept is considered SPURIOUS if:
is NOT a physical or anatomical attribute the ohject itself. It may correlate with the label due to dataset bias (c.g., hackground scenery or co-occurring objects), but is not causally related to the object's identity.
Respond only with SPURIOUS or NOT SPURIOUS a brief justification.
Classification Task Descriptions
Waterbirds: "distinguish between WATERBIRDS and LANDBIRDS."
MetaShift: "distinguish between common animal categories such as CATS and DOGS."
CelebA: "distinguish between people with BLONDE HAIR and DARK HAIR."
15

---

(a) Example our user interface for patch-based models
(b) Example of interface for text-based models
Figure 4: Participants are shown concepts learned by the model and asked to flag those that are spurious for the classification task.
We then show the number of initial concepts used by the model compared to the number removed by the users. For the user results, we average the initial concepts over the random three seeds, the removed concepts over the six debugging sessions. For the automated results, average over the three random seeds.
fairly consistent results across the six users and three random seeds, showing that users generally agree on which concepts are spurious. Since we have two users annotating each model,
For PIP-Net (Figure S
also compute the average agreement. On Waterbirds the average agreement is 97.9 97.9, and on MetaShift the average agreement is 82.4%.
For Post-hoc CBM (Figure 6) the users again seem remove around the number of concepts, although the agreement scores vary much more with average agreement on Waterbirds being 51.4% and average agreement MetaShift being 44.8%. However, point out that even though the agreement is not high, retraining can still work well across users we show in our main results.
Finally also show our automated results Post-hoc CBM (Figure 7 showing that removes more concepts on average than the real users.
Additionally, while focus on automating text-based models in this work, multi-modal models could be utilized to extend the automated results to patch-based models and we leave exploration of this to future work.
16

---

Arg. intall Concepts Arg. Namoved Cuncapts
175
152
Number oi Conoapo
200
52
Waterbints
Mete
Figure 5: Number of concepts marked spurious during the debugging sessions for real users PIP-Net.
Avg, Iriial Concests
Avg. Remoued Concepts
Concepta
Watnedia
Mesdait
Figure 6: Number of concepts marked as spurious during the debugging sessions for real users on Post-hoe CBM.
Aug. Concapts
Avg. Removed Concepts
20-
Number of Coreatt
15-
10
Watartinds
Headbut
Careat
Figure 7: Number of concepts marked as spurious during the debugging sessions for automated users on Post-hoc CBM.
17

---

A.3 RETRAINING
We then describe our retraining process. For PIP-Net we fully fine-tune the original model for an additional 30 epochs. We use batch size of 64 with reduced learning rate of le-S for the backbone, and 0.05 for the linear layer. For Post-hoc CBM, we follow the original work and keep the backbone and concept layer frozen, only retraining the linear layer. We keep Asparse 0.02.
We then retrain according to each of the following retraining algorithmss:
Remove: Remove the concepts from the concept bottleneck (e.g. zero weights in linear layer).
Retrain: Perform removal, then continue fine-tuning without performing any intervention for PIP-Net, and retrain linear layer for Post-hoc CBM.
ProtoPDebug: Add forgetting loss for spurious concept activations directly to the loss term. ProtoPDebug cannot be applied to Post-hoc CBMs because the loss is added to the concept bottleneck layer, but Post-hoc CBM only the final layer is retrained.
Augment: Augment the training dataset and then retrain. For PIP-Net, we randomly select one of the top ten patches for each concept and randomly paste five patches within each image. For Post-hoc CBM, we utilize the synthetic concept bank from Wu al. (2023) to perform Mixup with, but text-to-image model could create concept bank for any text text-based concepts used. We augment with a fixed value, keeping 0.75 of the original image and 0.25 of the spurious concept image for Mixup.
Reweight: Reweight all samples in the training dataset according to our permutation weighting scheme and then retrain. For PIP-Net these weights are applied to the classification loss, and for Post-hoc CBM these weights are applied when retraining the final linear layer.
CBDebug: Combine both Reweight and Augment and then retrain. Augmentation probabilities computed by squaring the normalized inverted weights. Reweight all samples according to our permutation weighting scheme, and augment samples that were assigned low weights by our scheme further reduce the impact of the spurious attribute. We use 7 2 for all datasets except ISIC, where we use 5.
A.4 DEFINITIONS
Permutation Weighting: A technique that reweights training samples to reduce reliance on a given auxiliary variable. Each sample receives a weight reflecting its likelihood under permuted version of the auxiliary variable, encouraging model to focus less on spurious correlations.
Worst-Group Accuracy (WGA): Measures the model's performance on the group of data which it performs the worst. This metric highlights whether the model fails on minority or atypical subgroups.
18

---

B FULL CONCEPT BANKS
We begin this section by motivating our methodology for concept bank creation. Prior work (Choi a on the distributional robustness of concept bottlenecks uses synthetic concepts from Wu et al. (2023) as the concept bank for training Post-hoc CBMs. We first note that their subselection of certain concepts requires knowledge of the underlying classification task and so in our work we do not subselect categories, except for ISIC where we only use texture concepts as defined in Wu et al. (2023). Additionally, as can seen Figure of Choi et al., there are no concepts that distinguish well between waterbirds and landbirds. Since we are working with real users, model that learns exclusively spurious concepts where all concepts are removed by the user is not useful.
Below we present the concept bank for each dataset used in this paper. Each dataset utilizes the synthetic concepts from Wu et al. 2202)), and then also utilizes our set of curated concepts. We first present the synthetic concept set utilized on all datasets.
Synthetic Concept Set
blackness, blueness, greenness, redness, whiteness, concrete, granite, leather, laminate, metal, blotchy, blurriness, stripes, polka dots, knitted, cracked, frilly, waffled, scaly, lacelike, grooved, stratified, gauzy, marbled, flecked, stained, braided, matted, meshed, cobwehbed, spiralled, dotted, crosshatched, wrinkled, woven, potholed, crystalline, paisley, veined, fibrous, studded, hubbly, pleated, grid, perforated, porous, interlaced, smeared, honeycombed, sprinkled, chequered, lined, banded, bumpy, zigzagged, swirly, pitted, freckled, bamboo, beach. bridge, bush, canopy. earth, field, flower, flowerpot, fluorescent, forest, grass, ground. harbor, hill, lake, mountain, muzzle, palm, path, plant, river, sand, sea, snow, water, awning, base, bench, building, earth, fence, field, ground, house, manhole, path, snow, streets, air-conditioner. apron, armchair; back-pillow, balcony, bannister, bathrooms, bathtub, bed, bedelothes, bedrooms, cabinet, carpet, ceiling, chair, chandelier, chest-of- drawerss, countertop, curtain, cushion, desk, dining dining-rooms, door, door-frame, double-door, drawer, drinking-glass, glass, exhaust-houd hood, figurine, fireplace, firepae,, floor, flower, flowerpot, fluorescent, ground, handle, handle-bar, headboard, headlight, house, jar, lamp, light, microwave, mirror, ottoman, oven, pillow, plate, refrigerator, sofa, stairs, toilet, bird, cat, cow, dog, horse, mouse, paw, arm, back, body, ear, eye, eyebrow, female-face, leg, male-face, foot, hair, hand, head, inside-arm, knob, mouth, neck, nose, outside-arm, ashcan, airplane, bag, bus, beak, bicycle, blind, board, book, bookcase, bottle, bowl, box, brick, basket, bucket, bumper, can, candlestick, cap. cardboard, ceramic, chain-wheel, chimney, clock, coach, coffee-tabie column, computer, counter, cup, desk, engine, fabric, fan, faucet, flag, floor. food. foot-board. frame, glass, keyboard, lid, loudspeaker. minibike, motorbike, napkin, pack, painted, painting, pane, paper, pedestal, person, pillar, pipe
To add more useful concepts, we add the attributes from CUB (Wah et al., 201I], translated to natural language, to the concept bank for the Waterbirds dataset. For ISIC, utilize the eight concepts from Yuksekgonul et al.] (202)). For MetaShift and CelebA, we instead curate a set concepts using a large language model similar Oikarinen al. (2023) with simpler prompt and more powerful model, GPT-4o. We also perform some manual pruning to ensure the concepts are useful for the given task (for example, removing cat and dog from the synthetic concept bank for MetaShift, or removing background concepts from the curated concept bank).
We utilize these concept banks proof of concept that CBDebug helps text-based models, and leave further exploration of the impact of different concept banks to future work.
Curated Concept Set Prompt
You are a concept generation assistant. Generate list clear and concise concepts are im- portant visual features for "laas_nname' Generate a list of concepts, with each concept appearing on separate line. Do not include any extra formatting, descriptions, or explanations- -jutt the raw concepts.
Concepts:
Then, show the curated concept set for each dataset.
19

---

Waterbirds Curated Concept Set
curved beak, dagger beak, hooked beak, needle beak, hooked seabird beak, spatulate beak, all all-purpose beak, cone beak, specialized beak, blue wings. brown wings, iridescent wings, purple wings, rufous wings, grey wings, yellow wings, olive wings, green wings, pink wings, orange wings, black wings, white wings, red wings, buff wings, blue upperparts, brown upperparts, iridescent upperparts, purple upperparts, rufous upperparts, grey upperparts. yellow upperparts. olive upperparts. green upperparts. pink upperparts. orange upperparts. black upperparts, white upperparts. red upperparts, buff upperparts. blue underparts, brown underparts, iridescent underparts, purple underparts, rufous underparts, grey underparts, yellow underparts, olive underparts, green underparts, pink underparts, orange underparts, black underparts, white underparts, red underparts, buff underparts, solid breast, spotted breast, striped breast, multi-colored breast, blue back, brown back, iridescent back, purple back, rifous back, grey back, yellow back, olive back, green back, pink buck, orange buck, black back, white back, red buck, buff back, forked tail tail, rounded tail tail, notched tail tail, fan-shaped tail tail, pointed tail tail, squared tail tail, blue upper tail, brown upper tail, iridescent upper tail, purple upper tail, rufous upper tail, grey upper tail, yellow upper tail, olive upper tail, green upper tail, pink upper tail, orange upper tail, black upper tail, white upper tail, red upper tail, buff upper tail, spotted head, malar head, crested head, masked head, unique pattern head, eyebrow head, eyering head, plain head, eyeline head, striped head, capped head, blue breast, brown breast, iridescent breast, purple breast, rufous breast, grey breast, yellow breast, olive breast, green breast, pink breast, orange breast, black breast, white breast, red breast, buff breast, blue throat, brown throat, tridescens throat, purple throat, rufous throat, grey throat, yellow throat, olive throat, green throat, pink throat, orange throat, black throat, white throat, red throat, bufl throat, blue eyes, brown eyes, purple eyes, rufous eyes, grey eyes. yellow eyes, olive eyes, green eyes, pink eyes, orange eyes, black eyes, white eyes. red eyes, busl eyes, about the same as heud bill, longer than head bill, shorter than head bill, blue forehead, brown forehead, iridescent forehead, forehead, olive forehead, green forehead, pink forehead, orange forehead, black forehead, white forehead, red forehead, buff forehead, blue under tail, brown under tail, iridescent under tail, purple under tail, rufous under tail, grey under tail, yellow under tail, olive under tail, green under tail, pink under tail, orange under tail, black under tail, white under tail, red under tail, buff under tail, blue nape, brown nape, iridescent nape, purple nape, rufous nape, grey nape, yellow nape, olive nape, green nape, pink nape, orange nape, black nape, white nape, red nape, buff nape, blue belly, brown belly, iridescent belly, purple belly, rufous belly, grey belly, yellow belly, olive belly, green belly, pink belly, orange belly, black belly, white belly. red belly, buff belly, sounded-wing. wings, pointed-wing wings, broad-wing wings wings, tapered-wing wings wings, long-wings wings, large size, small size, very large size, medium size, very small size, upright-perching water-like body, chicken-like-marsh body, long-leggeddlike body, duck-like body, owl-like body, gull-like body, humminghird body, pigeon-like body, tree-clinging like body, hawk-like body, aandpppe-likke body, upland-ground ground-like like body, swallow-like body, perching-like body, solid back, spotted buck, striped buck, mulli-colored back, solid tail, spotted fuil, striped tuil, multi-colored tail, solid belly, spotted belly, striped belly, multi-colored belly, blue primary color, brown primary color, iridescent primary color, purple primary color, rufous primary color, grey primary color, yellow primary color, olive primary color, green primary color, pink primary color, orange primary color, black primary color, white primary color, red primary color, buff primary color, blue legs, brown legs, iridescent legs. purple legs, rufous legs, grey legs, yellow legs, olive legs. green legs, pink legs, orange legs, black legs, white legs, red legs, buff legs, blue bill, brown bill, iridescent bill, purple bill, rufous bill, grey bill, yellow bill, olive bill, green bill, pink bill, orange bill, black bill, white bill, red bill, buff hill, blue crown, brown crown, tridescent crown, purple crown, rufous crown, grey crown, yellow crown, olive crown, green crown, pink crown, orange crown, black crown, white crown, red crown, buff crown, solid wing, spotted wing, striped wing, mulli-colored wing
forehead, rufous forehead, grey forehead, yellow
MetaShift Curated Concept Set
Long snout, Short snout, Floppy ears, Upright ears, Round eyes, Slit pupils, Curled tail, Straight tail, Stocky body, Slim body, Wide muzzle, Narrow muzzle, Large nose, Small noxe, Broad paws, Small paws, Short, dense fur. Fine, fur, whiskers, Expressive face, Neutrul face, Square upright posture, Crouched or perched posture
fur, Simple or spotted coat, Striped or marbled coat, Short whiskers. Long
CelebA Curated Concept Set
Light color, Dark color, Low contrast, High contrust, Warm, yellow tones, Cool, brown tones, Light eyebrows, Dark eyebrows, Light lashes, Dark lashes, Finer hair, Thicker hair, Soft texture, Coarse texture, Less visible roots, More visible roots
20

---

ISIC Curated Concept Set
blue-white veil, regular dots and globules, irregular dots and globules, regression structures, irregular streaks, regular streaks, atypical pigment network, typical pigment network
21

---

C ABLATIONS
C.I PERMUTATION WEIGHTING
Permutation Weighting utilizes main hyperparameters that we ablate in this section. The first is the number of folds K in K-fold cross-validiee voss-validation. We perform K-fold cross-validation in order to utilize all of our training data train the classifier while evaluating on unseen data. We also average the weights over multiple random permutations of the dataset to get more robust estimate of the weights.
We then evaluate the impact of these hyperparameters the assigned sample weights. In Table 4 and Table we evaluate different combinations the number of folds and number of permutations, and report the average weight assigned to each subgroup in the Waterbirds dataset. We found that the average weights per group is fairly robust to these hyperparameters, but as you increase the number of folds and decrease the number of permutations, the average weights for the minority subgroups increase. For our main results, we select five permutations and five folds to balance computational cost and robustness..
Table 4: Effect of number of folds (permutations fixed at 5) on average weights for each Waterbirds subgroup for randomly selected user.
Class (y)
Landbird Land
Landbird Water
Waterbird Land
Waterbird Water
Background (a)
# Training Samples
3498
184
56
1057
Average Weight (2 folds)
1.1
2.6
5.5
0.6
Average Weight (5 folds)
1.1
2.9
7.1
0.6
Average Weight (10 folds)
1.1
3.2
8.3
0.7
Table Effect of number of permutations (folds fixed at 5) on average weights for each Waterbirds subgroup for randomly selected user.
Class (y)
Landbird Land
Landbird Water
Waterbird Land
Waterbird Water
Background (a)
Training Samples
3498
184
56
1057
Average Weight (I permutation)
1.0
2.8
7.7
0.6
Average Weight permutations)
1.1
2.9
7.1
0.6
Average Weight (10 permutations)
1.1
2.8
6.6
0.6
22

---

C.2 DOES CBDEBUG EFFECTIVELY REMOVE DEPENDENCE ON SPURIOUS ATTRIBUTES?
We obtain the same visualizations of concepts before and after retraining for all the rest of the datasets and model combinations we test We similar results before, showing that CBDebug can effectively remove spurious concepts. In Table 6 the original model learns 'bookcase" which correlates highly with being indoors, and the retrained model learns 'bedroom' which correlates highly with being indoors as well. For CBDebug, none of its top five concepts correlate highly with the spurious attribute (indoor vs. outdoor). Figure [8] we see that the concepts learned by PIP-Net are not as well disentangled on MetaShift compared to Waterbirds (see our discussion in Section A.Ip. Finally, in Table 7 the original model learns the concept 'male face' for dark hair and 'female face' for blonde hair, but both baseline retraining and CBDebug remove the reliance on these main spurious concepts. CBDebug also learns 'Dark color', which better correlates with dark hair than 'building'.
(a) Concepts before retraining
(b) Concepts after retraining with CBDebug
Figure 8: The six most highly activated concepts for the Original model trained on MetaShift and the model after retraining with CBDebug. PIP-Net learns less disentangled concepts MetaShift, making the intervention less clear visually.
Table 6: Top five concepts for Post-hoc CBM before retraining, after retraining normally, and after retraining with CBDebug on MetaShift.
Class
Original
Retrain
CBDebug
Curled tail Long whiskers
Curled tail
Long whiskers
Long whiskers
Short whiskers
Cat
Short whiskers
Short whiskers
Slit pupils
mouse
bedroom
bird
bookease
Short, dense fur
Curled tail
Floppy ears
Floppy ears
Short snout
Wide muzzle
Wide muzzle
Long snout
Dog
Short snout
Short snout
Floppy ears
Narrow muzzle
Narrow muzzle
Wide muzzle
Long snout
Long snout
Narrow muzzle
Table 7: Top five concepts for Post-hoc CBM before retraining, after retraining normally, and after retraining with CBDebug on CelebA.
Class
Original
Retrain
CBDebug
male face
building counter ceiling
Dark color
blackness
granite
Hair
box building
mirror
eychrow
eyebrow
pillar
house
Less visible roots More visible roots female face
Less visible roots More visible roots freekled
Less visible roots More visible roots
Blonde Hair
freekled matted
23

---

C.3 AUGMENTATION PROBABILITIES
To convert the sample weights into probabilities, we first substract each from the max and then normalize them to [0, I]. This ends up with extremely right-skewed distribution, we also add a hyperparameter to control this skew by taking the augmentation probabilities to the power of
y. We plot the histogram for ? and 7 in Figure with 100 bins on the Waterbirds dataset. Additionally, this hyperparameter enables simple interpolation between our two approaches, because
as 7 Do you do not augment any samples and recover normal permutation weighting.
Disribution Augmentation Probabilties
Distribution of Augmentation Probubilities
101
1.5
CONFE 10P
127
Ausmentation Probabiity
Augmennttion Probatiity
(a) Augmentation Probabilities with y = 1.
(b) Augmentation Probabilities with yy 2.
Figure 9: Augmentation Probabilities computed on Waterbirds, with counts plotted log-scale. Squaring helps reduce the extreme right skew of the probabilities, reducing the probability that non-spurious samples get augmented.
Table & shows worst-group accuracy (WGA) on Waterbirds for E 21, 10). While there is some variability across seeds, WGA remains relatively stable, indicating that CBDebug is robust to the exact choice of y. For Waterbirds, MetaShift, and CelebA, reduce augmentation intensity and avoid distribution shifts caused by applying diffusion diffusion-generatte concepts medical images. Extensive tuning of this hyperparameter is left for future work.
2. For ISIC,
7 5 to
Table 8: Worst-Group Accuracy across values of y (augmentation probability) for Post-hoc CBM on Waterbirds.
Average
Worst
75.3 16.3
60.2
112.7
7
76.0 #2.8
58.3 +6.0
7 5
77.6 +6.0
58.9 +9.1
7 = 10
77.8 26.0
62.5 +11.4
24

---

C.4 COMPARISONS TO UNSUPERVISED BIAS MITIGATION APPROACHESS
While CBDebug offers a distinct approach from popular unsupervised bias mitigation pipelines: giving direct control domain expert who interacts with the downstream machine learning model instead of relying on training dynamics to guess what spurious correlations might be present, we do test our approach on bias mitigation datasets, where unsupervised bias mitigation pipelines can serve as useful benchmark for the effectiveness of C3Debug.
We evaluate unsupervised bias mitigation approaches Waterbirds and MetaShift with PIP-Net: Just train twice (Liu et al., 2021) (ITT) and Learning from Failure (Nam et al. 2020) (LfF). Following Espinosa Zarlenga al. (2024), we perform hyperparameter tuning using average validation accuracy. to avoid leaking privileged information about the underlying groups. For JTT, we select the number of epochs T from (1, 5, 25) and the upweighting term Aup from (10, 25, 50), and select (T, Aup) = (10, 25) for Waterbirds and (10.5, 5) for MetaShift. For LiF we select the bias amplification term q from (0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95) and select q = 0.9 for Waterbirds and q = 0.95 for MetaShift.
Our results are shown Table 9 While JTT shows no meaningful improvement, LfF does improve the worst-group accuracy compared to the original model. CBDebug demonstrates stronger ability to mitigate bias, improving worst-group accuracy over both of these unsupervised pipelines. This highlights CBDebug's 's effectiveness in leveraging expert feedback spurious concepts to fine-tune the model.
Table 9: Average and Worst-Group Accuracy Waterbirds and MetaShift with PIP-Net.
Method
Waterbirds
MetaShift
Average
Worst
Average
Worst
Original
92.3
71.9
80.9
52.4
Remove
92.610.4
74.4
81.4
55.0
Retrain
92.4
72.5
81.2=1.6
53.3+2.1
ProtoPDebug
92.5+0.1
71.6-1.9
80.9+1.
52.4+1.4
JTT
91.8
71.7
80.7
51.9
LFF
92.8 10.2
75.4
81.5
56.0
Ours
Reweight Only
93.2+..4
74.2
81.8
56.1
Augment Only
92.410.6
75.5
82.2
55.6
CBDebug
93.7
79.4
82.3
57.3
25


Reviews:
[{"review_id": "6sTEN49wnS", "reviewer": "Reviewer_16NR", "summary": "The authors introduce an interactive debugging algorithm designed for concept-based neural networks. In essence, a user provides a list of concepts that they think should not be used for prediction, the algorithm applies two strategies for updating the model so that it does not rely on said concepts anymore. The strategies include permutation reweighting to identify likely affected examples, and augmentation (either CutMix or Mixup, based on the architecture). Experiments compare the proposed technique against naive baselines and ProtoPDebug on four datasets.", "questions": "Please feel free to comment on the weaknesses I pointed out.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "**Originality**: the idea of debugging concept-based models is not novel, as readily acknowledge by the authors, and neither are the individual deconfounding strategies.  The specific mix proposed here is however novel.  This is good enough for me.\n\n**Quality**: A benefit of relying on pre-existing strategies is that we know that they have been validated empirically already, which is good. Regardless, the algorithm as a whole is intuitively appealing.  The experimental setup is also good: four confounded datasets and a reasonable selection of competitors.  The research is empirical in nature.\n\n**Clarity**: The text is well written, well structured, and easy to follow.  The related work is quite comprehensive.\n\n**Significance**: This work contributes to ongoing efforts for making self-explainable models more reliable, and as such it is significant.\n\nOverall, I like this paper.", "weaknesses": "**Clarity**:  Some aspects of the work should be described in more detail in the main text, chiefly the augmentation strategies.  Right no, Section 4.3 doesn't tell the reader how augmentations are obtained exactly.  This unnecessarily complicates understanding.\n\nFurthermore, the main competitor (ProtoPDebug) is designed for interactive usage, in the sense that it actively seeks cheap supervision that can quickly correct the model after few interaction rounds. Here the experiments consider a passive setup instead: there are not explicit interaction rounds. As such, it is unclear how much supervision the proposed method requires to \"beat\" ProtoPDebug.\n\n**Significance**: If I understand correctly, the proposed method works under the assumption that the same set of spurious concepts applies to all inputs and labels. This is not necessarily the case. (I don't think ProtoPDebug makes this assumption.) This limits applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "21u0snV5AK", "reviewer": "Reviewer_SPzc", "summary": "The paper introduces CBDebug, a methodology designed to leverage expert (or automated) feedback about concepts that are spuriously correlated with the target task. The goal is to debias the model and promote learning of the correct relationships between concepts and labels. The approach involves three main steps: (i) collecting feedback to identify spuriously correlated concepts, (ii) reweighting samples based on the activation of those concepts, and (iii) augmenting the dataset according to these weights to reduce the influence of spurious correlations. The model is then retrained on the augmented and reweighted dataset. Through experiments on four datasets and across two model architectures, the authors demonstrate performance gains, particularly for the most challenging (worst-performing) classes.", "questions": "- Why was the evaluation limited to the Post-hoc CBM architecture? It would be valuable to test the proposed method on additional CBM variants such as CEM or Probabilistic CBM. This could further demonstrate the general applicability and robustness of the approach, especially if similar improvements are observed.\n- Could the authors clarify how the Remove baselines were implemented? If the spurious concepts were simply removed without further fine-tuning, how was the predictor adjusted to handle the missing concepts?\n- As I understand it, the core idea of the proposed method is to reweight samples containing spurious correlations and further perturb them through data augmentation to reduce the bias. However, this process may also downweight correctly predicted concepts present in the same samples. Could the authors comment on this potential side effect and its practical implications?\n- How would the proposed approach compare to a simpler baseline that removes all samples with weights below a given threshold, rather than reweighting or augmenting them?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The authors address an important problem (debiasing and aligning Concept Bottleneck Models (CBMs)) from a dataset-centric perspective. By leveraging expert or automated feedback, their approach moves toward the goal of making models \"right for the right reasons\". Although the experiments are not extensive in terms of model diversity, the results are promising and supported by strong qualitative analyses.", "weaknesses": "- The paper is somewhat difficult to follow, as it draws upon multiple research areas and alternates between them several times, particularly in the first part of the paper. A clearer narrative structure would help the reader grasp the main contributions more easily.\n- The proposed approach is evaluated on only two models (one CBM and one prototype-based), limiting the scope of the empirical validation. It would strengthen the paper to include additional CBM-like baselines (e.g., CBM, CEM, Probabilistic CBM). This would better demonstrate the generality of the proposed method. Moreover, it should be clarified whether the Post-hoc CBM was used with the residual setting (which leverages side information) or in its standard form, as the latter is conceptually equivalent to a CBM and thus a reasonable baseline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FxqMzmJ3fM", "reviewer": "Reviewer_AMnB", "summary": "This paper introduces CBDebug, a human-in-the-loop debugging pipeline to improve the underrepresented generalization of interpretable concept-based models (e.g., CBMs). CBDebug is a general framework in which human experts first identify concepts that a CBM spurious uses for downstream predictions (removal step). The model is then retrained by removing these effects through causal rebalancing and data augmentation (retraining step). The work’s evaluation suggests that CBDebug improves worst-group accuracy and helps models avoid spurious shortcuts across different baselines and underlying architectures.", "questions": "Balancing the strengths and weaknesses of this work, I am leaning towards accepting this work. However, below I include some questions identifying minor concerns/aspects that I think, if appropriately addressed, could improve this submission:\n\n1. **[Critical]** Could you please elaborate on my concern regarding the significance of the observed results? Why is this not discussed anywhere or, at the very least, acknowledged?\n2. **[Minor]** Could it be possible that the “Retraining” baseline is affected by being exposed to the spurious concepts in its original training? If so, then using a baseline model trained from scratch without any of the identified spurious concepts may give us a good sense of how much the second step of CBDebug matters compared to what one would see if the concepts were removed from the start.\n3. **[Major]** Could you please discuss why the results of Appendix C.4 are not discussed anywhere in the main body of the paper? I believe they are both interesting and insightful results (particularly for those coming from the fairness community). Therefore, at the very least, I would suggest discussing their takeaways in the main body of the paper. More generally, I would recommend at least summarizing the main takeaways of any experimental appendix you include in your paper in the main body (otherwise, people may not learn their importance unless they go the extra mile of reading the entirety of the appendix).\n4. **[Major]** How do augmentation-specific hyperparameters affect the observed results (e.g., $\\gamma$, $k$, the use of top-*ten* activated patches for ProtoPNets, text-to-image model for VLM-CBMs)? Appendix C.3 very superficially touches upon $\\gamma$, but there are no other ablations on these hyperparameters elsewhere. \n\n### Minor Suggestions and Typos\n\nI found the following potential minor issues/typos, which may be helpful when preparing a new version of this manuscript:\n\n1. **[Missing References, Minor]** I would suggest adding relevant references/citations for the introduction claim that “This capability is crucial in high-stakes domains, such as healthcare or scientific analysis, where errors are costly and expert validation is essential.”\n2. **[Potential Typo, Nit]** The reference “Ross et al. (2017)” in line 73 is dangling. It likely needs to be “(Ross et al., 2017)” instead. A similar comment applies to the reference of “Zhou et al. (2016)” in line 113.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Thank you so much for submitting this work! I enjoyed reading this paper, learned a lot from it, and appreciate the time taken to write it up and submit it to ICLR. Below are what I believe are this paper’s main strengths:\n\n1. **[Significance, Critical]** This work lies at the intersection of fairness, interpretability, and human-in-the-loop AI. It achieves its goals by introducing a new interactive debugging pipeline that can be applied to **any** unsupervised concept-based model (and can be fully automated using foundation models as arbitrers). Hence, I believe this work may be of importance to a large fraction of the ICLR community and can have a potentially large impact, laying the groundwork for further papers on interactive debugging and concept-based interpretability.\n2. **[Quality, Critical]** The overall method is very well motivated and sound. Moreover, the evaluation includes automated and human-driven experiments, both key to fully understanding the proposed pipeline, along with quantitative and qualitative results. Considering all of this, the paper’s methodology and evaluation are strong.\n3. **[Originality, Major]** Overall, the authors do a great job at identifying a key gap in the literature, that of lacking a general interactive debugging method for any unsupervised CBM, and proposing a working approach for that gap. This is certainly potentially impactful and novel. \n4. **[Clarity, Major]** The paper is very well-written and easy to navigate. On top of that, the figures are easy to follow, clear, and helpful in understanding the overall contribution and results.", "weaknesses": "In contrast, I believe the following are some of this work’s limitations:\n\n1. **[Quality, Major]** My primary concern is that the results in Table 1 seem to be extremely noisy (some standard deviations are very large). More importantly, notice that this is also evident in Table 2, so it may not be just due to the human aspect of some of this evaluation, as those results are based on the foundation-model-based selection. Therefore, it is very difficult to judge the significance of the observed differences, and the lack of discussion of this key limitation anywhere in the paper is a bit troublesome, as it omits a clear potential issue in the results.\n2. **[Clarity, Minor]** Although the appendix has several empirical studies, none of the key results of these appendices are discussed in the main body of the paper (meaning people will likely never see them). For example, the takeaways from Appendix C.4, which would be of interest for the fairness community, are not even mentioned in the main paper.\n3. **[Novelty, Minor]** The only reason why I didn’t mark the novelty above as critical is that the methodological aspect of the proposed pipeline, although very sensible and interesting, does not introduce any new methodological elements. Instead, it (very sensibly) builds on work in other areas (e.g., permutation weighting, CutMix, MixUp, etc.). Nevertheless, in my opinion, this is not a severe limitation, and it shouldn’t significantly diminish the novelty of the proposed pipeline, as it has novelty in its own right by combining diverse methodologies to achieve a specific goal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VEbRoEZaQK", "reviewer": "Reviewer_jWmY", "summary": "This paper introduces a framework to analyze and mitigate interpretability failures in Concept Bottleneck Models (CBMs). The authors observe that CBMs can behave unreliably even when concept predictors achieve high accuracy, due to a misalignment between concept representations and their intended semantics. To address this, the paper proposes a _debugging_ process that diagnoses which concepts lead to faulty predictions and retrains or replaces them. The analysis combines synthetic and real datasets to demonstrate that concept-level reliability does not guarantee faithful concept use by the model.", "questions": "1. Related to W1, could the authors clarify in more concrete terms how their approach differs from prior works on causal or concept-editing methods in CBMs?\n\n2. How do the authors expect their findings to generalize to other modalities or model families, such as large language models that incorporate concept bottlenecks?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper addresses an important limitation in CBMs (i.e., the gap between interpretability claims and true concept usage). This is timely and relevant given the increasing adoption of CBMs in interpretable ML research.\n- The diagnostic method proposed is systematic and empirically supported by several experiments. The experiments convincingly show that high concept accuracy does not imply semantic reliability, which is an important insight.", "weaknesses": "- (W1) While the framing of _debugging_ CBMs is conceptually appealing, the proposed methodology appears closely related to existing approaches on causal or concept-level editing in CBMs (e.g., [1], [2]). It would be helpful if the authors could elaborate on the specific algorithmic contributions that distinguish their method from these prior studies. The originality claim would be more convincing if the paper introduced novel theoretical insights.\n- (W2) Several technical terms (e.g., permutation weighting, worst-group accuracy) are mentioned without formal definitions. Providing clearer descriptions or mathematical formulations would help readers understand how these quantities are computed and how they contribute to the experimental evaluation.\n\n**References**\\\n[1] Chauhan, K., Tiwari, R., Freyberg, J., Shenoy, P., & Dvijotham, K. (2023, June). Interactive concept bottleneck models. In Proceedings of the aaai conference on artificial intelligence (Vol. 37, No. 5, pp. 5948-5955).\\\n[2] Hu, L., Ren, C., Hu, Z., Lin, H., Wang, C. L., Xiong, H., ... & Wang, D. (2024). Editable concept bottleneck models. arXiv preprint arXiv:2405.15476.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)