Title: An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations
Venue: NeurIPS 2025
Accepted: yes
Keywords: concept bottleneck models

label noise
Abstract: Concept bottleneck models (CBMs) ensure interpretability by decomposing predictions into human interpretable concepts.
Yet the annotations used for training CBMs that enable this transparency are often noisy, and the impact of such corruption is not well understood.
In this study, we present the first systematic study of noise in CBMs and show that even moderate corruption simultaneously impairs prediction performance, interpretability, and the intervention effectiveness.
Our analysis identifies a susceptible subset of concepts whose accuracy declines far more than the average gap between noisy and clean supervision and whose corruption accounts for most performance loss.
To mitigate this vulnerability we propose a two-stage framework.
During training, sharpness-aware minimization stabilizes the learning of noise-sensitive concepts.
During inference, where clean labels are unavailable, we rank concepts by predictive entropy and correct only the most uncertain ones, using uncertainty as a proxy for susceptibility.
Theoretical analysis and extensive ablations elucidate why sharpness-aware training confers robustness and why uncertainty reliably identifies susceptible concepts, providing a principled basis that preserves both interpretability and resilience in the presence of noise.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the impact of concept-level noise in Concept Bottleneck Models (CBMs), focusing on a realistic scenario where the supervision signal on intermediate concepts is imperfect. The authors conduct a systematic study on how concept noise affects downstream prediction performance, interpretability, and intervention effectiveness. To mitigate the model’s vulnerability to noisy concepts, they propose two complementary techniques: sharpness-aware minimization during training, and uncertainty-based intervention during inference. Experimental results across multiple datasets demonstrate the effectiveness of the proposed solutions.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts:\n\n1. The paper offers a systematic and well-structured investigation into how concept noise influences various aspects of CBMs.\n\n2. The writing is clear and logically organized, making the paper easy to follow.\n\n3. The proposed methods are simple, practical, and effective in improving model robustness under noise.\n\nWeaknesses:\n\n1. The assumptions regarding the noise generation process are arguably too simplistic. In Section 2.1, this process ignores potential dependencies between concepts and labels.\n\nQUESTIONS:\nThe noise simulation process appears oversimplified. As described in the paper: \"each binary concept label $c_i \\in {0,1}$ is independently flipped with probability $\\gamma$ to simulate concept noise $\\hat{c}$. For target noise $\\hat{y}$, each class label $y$ is randomly changed to a different label, uniformly sampled from the remaining classes, also with probability $\\gamma$.\"\n\nHowever, as the authors themselves note in lines 31–33, \"changing concepts can shift the label.\" This indicates that concept and target label noise are not necessarily independent in real-world scenarios. Given this, how do the authors justify the assumption of independence between noisy concepts and noisy labels in their experimental setup? Would the proposed methods remain effective under a more realistic, correlated noise model?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides an empirical analysis, backed up by some theoretical arguments, of how concept bottleneck models (CBMs) are affected by noisy concept annotations. The authors identify noisy concept supervision in training data as a major source of downstream performance degradation. They claim that this effect is primarily driven by a subset of so-called “susceptible concepts.” To mitigate the issue, the authors propose two strategies: sharpness-aware minimization (SAM) during training and uncertainty-guided interventions at inference. Experiments on two real-world datasets support their conclusions.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. The paper presents a systematic empirical study of the effects of noise in CBMs, including detailed analyses of prediction performance, interpretability, and intervention effectiveness.\n2. It offers practical insights for working with noisy concept annotations.\n3. The study is timely, especially given the growing use of LLM-generated concept labels, which are prone to noise.\n4. The theoretical derivations connecting susceptibility and predictive uncertainty provide additional support and motivation for the intervention strategy.\n\n### Weaknesses\n\n1. **Lack of Novel Methodology.** The paper does not introduce any fundamentally new methods. Both SAM and uncertainty-based interventions are existing techniques that are simply applied to this setting. The novelty is limited to empirical observations and analysis, which—though valuable—do not represent a significant conceptual advance.\n2. **Limited number of datasets.** For a paper whose main contributions are empirical, the use of only two datasets (CUB and AwA2) is limiting. The findings would be more convincing if demonstrated on a broader range of settings.\n3. **Predictable Findings.** Several results—such as the degradation of interpretability and performance under noise, or the benefits of targeting uncertain predictions—are somewhat expected. While clearly and rigorously shown, these insights are incremental.\n\nQUESTIONS:\n1. The theory in Appendix B compares $f(\\hat{c}_i)$ with $f(c^*_i)$, focusing on the error in predicting concept $i$. However, how does this prediction error relate to the *training noise rate*, which is the central focus of this paper? The setup seems to conflate inference-time uncertainty with training-time corruption—could the authors clarify this connection?\n2. Related to W1), how do the authors position their findings in light of prior work (e.g., Shin et al., Penaloza et al., Sheth & Kahou), which also studied intervention strategies and robustness in CBMs? What distinguishes this work?\n3. The definition of susceptible concepts is based on a greater-than-average drop in accuracy under noise. But what if a concept suffers a large decline yet contributes negligibly to the final prediction (e.g., in a linear bottleneck model, if its weight is close to zero)? Shouldn't the uncertainty-aware intervention take into account concept relevance?\n\nLIMITATIONS:\nThe paper lists several limitations which are fair.\n\n---\n\nOverall, this is a well-written and timely paper that provides a useful empirical analysis of CBMs under noisy supervision. However, in my opinion, it falls short of the NeurIPS bar in terms of novelty and conceptual contribution. The main techniques used are existing ones, and the findings, while insightful, are largely expected. I would encourage the authors to consider submitting to a workshop or resubmission after strengthening the theoretical and/or methodological contributions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a comprehensive study on how noisy concept annotations affect Concept Bottleneck Models (CBMs), which are designed to enhance interpretability in machine learning by predicting human-understandable concepts before making final predictions. The authors demonstrate that even moderate levels of noise in concept labels can significantly degrade CBM performance across three dimensions: predictive accuracy, interpretability, and the effectiveness of human interventions. They identify a subset of concepts that are disproportionately affected by noise and are primarily responsible for the observed performance degradation.\n\nTo understand the mechanisms behind this vulnerability, the authors analyze how noise impacts individual concept predictions and their alignment with target outputs. They find that noise does not affect all concepts equally. A small group of concepts suffers severe accuracy drops, often due to semantic ambiguity or low frequency in the dataset. These “susceptible concepts” also tend to be the most influential in determining the final prediction, meaning their corruption has an outsized impact. Furthermore, noise disrupts the alignment between concept representations and target predictions, leading to misinterpretations and reduced model reliability.\n\nTo mitigate these effects, the authors propose a two-stage framework. During training, they apply Sharpness-Aware Minimization (SAM), which encourages the model to find flatter minima in the loss landscape, thereby improving robustness to noisy labels. SAM is shown to significantly improve both concept and task accuracy, especially for the susceptible concepts. During inference, they introduce an uncertainty-guided intervention strategy that prioritizes correcting the most uncertain concept predictions. This approach leverages entropy as a proxy for susceptibility, enabling targeted corrections that restore much of the lost performance with minimal intervention.\n\nThe paper provides both theoretical and empirical support for the proposed methods. The authors derive formal justifications showing that predictive uncertainty correlates with susceptibility under reasonable assumptions. Empirical results across multiple datasets and CBM variants confirm that combining SAM with uncertainty-guided interventions consistently outperforms baseline methods. Notably, correcting just a few highly uncertain concepts can recover performance close to that of models trained on clean data, even under high noise conditions.\n\nThis study offers a principled and practical framework for enhancing the robustness of CBMs in noisy environments. It highlights the importance of identifying and protecting noise-sensitive concepts and demonstrates that targeted training and inference strategies can preserve both interpretability and predictive power. The authors also discuss limitations, such as the reliance on binary concept labels and linear predictors, and suggest future directions including extensions to hierarchical or continuous concepts and the integration of semi-supervised learning. This work lays a strong foundation for deploying interpretable models in real-world settings where annotation noise is inevitable.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n-\tThis is the first comprehensive study that systematically investigates the impact of noisy concept annotations on CBMs.\n-\tThe paper is well written with clear language and a clear articulation of the problem the authors seek to address.\n-\tThoughtful experiments by the authors lead to insightful results around the subset of concepts responsible for degrading performance.\n-\tThe mitigation strategies proposed are both theoretically sound and empirically validated.\n-\tThe authors consider the practical implications and limitations of their work.\n\nWeaknesses:\n-\tThe paper only gives consideration to one modality, vision, and does not touch on generalizability to other modalities.\n-\tThe noise is simulated and random, which may not reflect real world noise in annotations.\n\nQUESTIONS:\nWhat are the main factors beyond ambiguity and low prevalence in the dataset that make certain concepts more susceptible?\n\nHow does SAM improve the robustness of CBMs compared to other noise mitigation techniques?\n\nWhy do you think predictive entropy is a good surrogate for susceptibility? Can you explain the intuition?\n\nHow do you think your results might generalize to other modalities?\n\nHow do you think real world noise might differ from the simulated noise in this study, and how might that alter your results?\n\nLIMITATIONS:\nLimitations are thoughtfully considered and addressed within the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper examines how label noise, encompassing both concept label noise and task label noise, impacts Concept Bottleneck Models (CBMs). Specifically, this work shows that (1) a CBM’s task and concept accuracies are highly susceptible to noisy label annotations, (2) concept noise is the primary source of degradation, (3) concept-label-noise-induced degradation can be attributed to small set of “susceptible” concepts, which can then be targeted during intervention to improve the effectiveness of an intervention, and (4) noise-aware losses such as Sharpness-Aware Minimization (SAM) can alleviate some of the effects of label noise. This work evaluates all these elements primarily for traditional CBMs across two datasets: CUB and AwA2. This empirical evaluation is then complemented with two theoretical results on the effect of introducing SAM to a CBM’s training and on using concept uncertainty as a proxy for identifying the set of susceptible concepts.\n\nSTRENGTHS AND WEAKNESSES:\nThank you for this very interesting work; I enjoyed reading it and found it easy to follow (thanks!). Below, I describe what I believe are this work’s main strengths and weaknesses:\n\n### Strengths\n\n1. **(S1) [Significance, Major]** This paper examines a highly popular XAI framework, namely concept bottleneck models (CBMs), from the perspective of robustness. As such, it has the potential to be of interest and use to a large subset of the NeurIPS community.\n2. **(S2) [Quality, Major]** The sheer number of evaluations in the entire paper is very large, although a significant chunk of them are left entirely in the appendix (and are not discussed at all in the main body of the paper). Within those experiments discussed in the main body, the figures and tables are, for the most part, really easy to parse, read, and understand.\n2. **(S3) [Originality, Minor]** Although, as argued below, this work’s originality could be stronger (as it mostly evaluates results that are to be expected given previous works), the theoretical results introduced in the appendix are interesting and new. This is particularly true for the results in Appendix B, as those in Appendix A are new but appear to be a simple readjustment of previous work by Baek et al.\n4. **(S4) [Clarity, Minor]** The paper is very clearly written and well-organized.\n\n### Weaknesses\n\n1. **(W1) [Originality + Significance, Critical]** Although it is always important to verify expected claims empirically and theoretically, in my opinion, most of the results presented in this work are to be expected based on well-known limitations of DNNs (not just CBMs). For example, if dataset noise increases, it is well-known and expected that we will struggle to learn a classifier (which, if used as the input of yet another classifier, as in CBMs, that new combined downstream classifier will also struggle). Similarly, if noise increases on features such as concepts, then it is expected that, in the limit, all concepts will become indistinguishable from each other.  Therefore, as noise increases, it is not unexpected that all concepts become equally (although spuriously) important to any model trained on them (as seen in Section 3). Similar points apply to the other observations/results of this paper (including intervention results and the mitigating experiments, which use an already-existing mitigation approach for DNNs to improve robustness in CBMs). Because of this, I have some hesitations about the potential novelty and contribution of this work vis-à-vis already well-established results.\n2. **(W2) [Quality, Critical]** The study of label noise in the main body of the paper is restricted uniquely to traditional CBMs. This leaves aside a wealth of better-performing, highly popular methods that have emerged since, including CEMs, ProbCBMs, ECBMs, Post-hoc CBMs, Label-free CBMs, Stochastic CBMs, and others. These are only discussed in the appendices. This seems like an odd decision given that they are important considerations. More importantly, however, looking at the SAM results for these baselines casts some doubts on whether the results improvements from SAM observed in CBMs in the main body of the paper properly generalize. Please see below for specific questions regarding this.\n3. **(W3) [Quality, Critical]** The evaluation of the proposed mitigation approaches is lacking a non-trivial evaluation baseline in the main body (i.e., not discussed in the appendix). This is particularly important if the main contribution for this mitigation pipeline is applying an already existing method to CBMs (if that's the case, then the novelty could be in explaining WHY one should use SAM over any of the other existing approaches to deal with label noise). Moreover, the SAM experiments would significantly benefit from explaining how SAM compares against recently proposed concept-specific methods that aim to handle the exact same problem (e.g., [1], although I am aware this paper might've come concurrently with this work). Similarly, the concept intervention experiments for the proposed policy in Figure 9 do not include any non-trivial intervention policies (e.g., Coop [1] or UCP and CCTP). These are later discussed in Figure 11, but they appear to be discussed as if these results were previously unknown. However, the efficacy of UCP w.r.t. a random baseline and CCTP was well-known from the paper that introduced both. What would be more relevant for this paper is to see how it compares to the policy “susceptibility”-based policy introduced here.\n4. **(W4) [Clarity, Major]** Many key results are included and discussed only in the appendix (e.g., comparisons for different CBMs, against other mitigation strategies, etc). Yet, several of these appendix experiments are critical and more important than those expected from previous works, such as those in Section 2 and Figure 11. Therefore, I would strongly suggest that the authors reconsider the structure and discussion of these results in the main paper. At the very least, the main takeaway of the results discussed in the appendices should be included in the main paper (currently, there is no easy way to navigate the appendices other than reading them end-to-end, as there are no mentions of the main messages of each appendix anywhere in the main body of the paper). \n5. **(W5) [Quality, Major]** Some of the experiments, for example, the results summarized in Figure 1 and the visualizations reported in Figure 4, are hard to interpret or are missing key details. See below for specific questions.\n\n### References\n\n1. Penaloza et al. Preference optimization for concept bottleneck models. In ICLR Workshop on Human-AI Coevolution, 2025.\n2. Chauhan et al. \"Interactive concept bottleneck models.\" *Proceedings of the AAAI Conference on Artificial Intelligence*. Vol. 37. No. 5. 2023.\n\nQUESTIONS:\nI am currently leaning towards rejecting this work, given that I am not entirely convinced that the results presented in this work, as they are currently framed, are substantially different from those discussed in previous works. Below, I list some questions whose answers I believe could potentially add clarity or further improve this manuscript. If I have misunderstood anything at any point, please do not hesitate to let me know (sorry in advance if so!).\n\n1. **[Significance + Originality, Critical]** I have some serious concerns regarding the novelty and significance of this paper. To help with this, it would help if you could elaborate on the following points: (1) Why are the results introduced in Section 2 not to be expected given known issues of training DNNs with noisy inputs? (2) Why are the results in Section 3.2 surprising, given that, as argued above, it is expected that in the limit, all noisy concepts will become indistinguishable? (3) Why are the SAM results in section 4.1 unexpected, given that SAM is a general-purpose optimization technique that can be applied to most DNNs, including CBMs, without much modification? (4) Why are the results in Figure 11 something unexpected, given that Shin et al. already discuss that in detail in their work? (5) How does the UCP baseline compare to the “susceptibility”-based intervention policy proposed earlier in the paper?\n2. **[Quality + Clarity, Critical]** Could the authors please explain why comparisons against other relevant CBM variants or mitigation pipelines were entirely pushed to the appendix without any discussion of them in the main paper’s body? For example, if results for other potential mitigation approaches are already in the appendix, why should one not discuss them in the main body, where SAM is evaluated? Similarly, if other CBM variants have been evaluated as part of the study, why not include those results in the main body? The decision to include these in an appendix that is barely discussed in the main paper seems unusual, as they are important baselines that one would expect to have as comparisons in the main figures, not hidden in the appendices. Because of this, I believe that this work may benefit from a potentially significant rewrite of the experimentation section, focusing more on unexpected aspects.\n3. **[Quality, Critical]** I have some concerns about whether SAM works in CBMs for the reasons claimed in this paper. For example, why do we not see similar improvements in robustness for other CBM variants (e.g., CEMs, ECBMs, etc) when we use SAM to train them (as seen in Table 6 of the appendix)?\n4. **[Significance + Quality, Major]** Why is the goal of this work different from that of Penaloza et al. [1], as argued in Section 5? That work also seems to propose a way for mitigating label noise in CBM models. I am aware that they don’t conduct a complete study of label noise on CBMs (instead, they assume that CBMs are sensitive to this noise), but the aim to mitigate the effect of this noise seems to be shared between this paper and [1]. Given that [1] may be considered concurrent to your work, however, I only mark this as a major concern rather than a critical concern (however, at the very least, [1] should be adequately characterised in this work).\n5. **[Quality, Minor]** What are the “embeddings” used in Figure 4? Are these CEM embeddings? If so, all of these important details should be immediately clear from the text.\n6. **[Quality, Minor]** Why are interventions done at a per-concept level rather than at a group-level for datasets such as CUB, where there are groups of known mutually-exclusive concepts? This approach deviates from standard practices when evaluating models on this dataset, so it would benefit from further clarification.\n\n### Minor Suggestions\n\nRegarding suggestions for the presentation, the following are possible very minor errors and typos that could be addressed before a camera-ready version:\n\n1. **[Nit, Potential Typo, Very Minor]** It is unclear what “it” refers to in “it first predicts…”  (line 24). If it is “CMBs”, then it should be “they” or, to make it even clearer, explicitly “CBMs” or “a CBM”.\n2. **[Nit, Presentation]** The use of a circular plot in Figure 1 for representing a noise vs accuracy relationship, rather than a standard/traditional simple scatter/line plot, is a bit unusual. This may be just my personal preference, but it is much easier to read a standard scatter plot in a horizontal fashion than in a radial fashion. As such, I would reconsider the choice of plot for this figure to make the paper easier to parse and read. A similar comment applies to Figures 12 and 13 in the appendix (the former could also benefit from a larger font for the labels).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work investigates how label noise corrupts concept bottleneck models (CBMs), degrading their prediction accuracy, interpretability, and intervention utility. The authors identify that performance loss is caused by a vulnerable subset of concepts, those most sensitive to annotation errors. To address this, the authors introduce (1) sharpness-aware training to stabilize concept learning and (2) inference-time entropy ranking to detect and correct only the most noise susceptible concepts. The framework preserves CBM reliability under noise while maintaining full interpretability, supported by theoretical analysis and empirical validation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Investigating noisy labels in CBMs presents a meaningful study to learning with noisy labels.\n2. The work conducts extensive and thoughtful experiments, thoroughly analyzing the impact of noise across multiple dimensions.\n\nWeaknesses:\n1. Since SAM improves performance even on clean data, its specific role in mitigating noisy impacts remains unclear.\n2. The noisy impact is highly dependent on hyperparameters (e.g., epochs, learning rate) and noise type. These settings should be explicitly clarified in Sections 2–3 for reproducibility.\n3. Experiments were run across three different GPUs, making replication difficult. At minimum, the same hardware should be used for comparable experiments, and results should specify the GPU used.\n4. The observed noise effects resemble Subclass-Dominant Label Noise (SDN) [1], where intra-class attributes influence robustness. A direct comparison would strengthen the analysis.\n\n[1] Subclass-Dominant Label Noise: A Counterexample for the Success of Early Stopping\n\nQUESTIONS:\n1. If only a small subset of concepts is susceptible noise, does this limit the practical impact of label noise on CBMs if they do not occur on susceptible noise?\n2. Figure 10 shows unclear separation between susceptible and non-susceptible concepts. Are these patterns consistent across different datasets. How many datasets have the authors evaluate?\n3. Is noise applied independently to each concept or the entire dataset? This significantly affects interpretation of the results.\n4. For globally applied noise, does the model consistently identify the same concepts as susceptible across different noise instantiations, or does susceptibility vary randomly? I suggest the authors track these susceptible concepts.\n5. Target noise does not affect the performance, even at 40% corruption, which seems odd. Why?\n6. In Figure 7, why do many initially low frequency concepts show frequency rise (2000x+) after noise injection?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}