Title: Deferring Concept Bottleneck Models: Learning to Defer Interventions to Inaccurate Experts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Concept Bottleneck Models

Learning to Defer
Abstract: Concept Bottleneck Models (CBMs) are interpretable machine learning models that ground their predictions on human-understandable concepts, allowing for targeted interventions in their decision-making process. However, when intervened on, CBMs assume the availability of humans that can identify the need to intervene and always provide correct interventions. Both assumptions are unrealistic and impractical, considering labor costs and human error-proneness. In contrast, Learning to Defer (L2D) extends supervised learning by allowing machine learning models to identify cases where a human is more likely to be correct than the model, thus leading to deferring systems with improved performance. In this work, we gain inspiration from L2D and propose Deferring CBMs (DCBMs), a novel framework that allows CBMs to learn when an intervention is needed. To this end, we model DCBMs as a composition of deferring systems and derive a consistent L2D loss to train them. Moreover, by relying on a CBM architecture, DCBMs can explain the reasons for deferring on the final task. Our results show that DCBMs can achieve high predictive performance and interpretability by deferring only when needed.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces deferring concept bottleneck models (DCBMs). This method combines the idea of learning to defer with the intervention procedure of concept bottleneck models. In that, DCBMs improve upon default interventions in two ways: Estimating when interventions are required and estimating whether deferring to the human would actually be helpful. To achieve this, DCBM introduces a new loss to train a CBM with deferring in mind.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper introduces a novel method to incorporate deferring to humans into CBM training, tackling an important problem of practicability of interventions.\n- The introduced loss is formally derived and justified\n\nWeaknesses:\n- The implications of some of the results is not clear and missing some discussions.\n\nQUESTIONS:\nClarifications for the experimental evaluations:\n- Setup:\n\t- How are the uncertain concept annotations for CUB constructed? Specifically, what is the randomness in the human concept labels?\n\t- What is the task accuracy of the human for cifar10-h?\n\t- Are the black-box baselines the same as the models used to compute the concept embeddings?\n- RQ2:\n\t- In my understanding, deferring of concepts cannot help to mitigate concept completeness. Thus, the only possible deferring to handle concept completeness is deferring on the task. However, when the task is deferred to the human, than the concept space does not matter (i.e. also whether it is complete or not). Maybe I am missing the point here, but I do not really understand why the concept of completeness is particularly interesting in this context. I would appreciate if the authors could elaborate more on why it is not expected that task deferring should be independent of the concept space.\n\nGeneral questions for the proposed method:\n- Deferral cost of concepts and task labels: In the conclusion, it is stated that the same costs are assumed for deferring at a concept or task level. I agree that different costs should probably be explored in the future, but I am also curious how this affects the current results: Is the deferring of concepts dependent on the number of concepts compared to the (singular) task label? Or is this irrelevant, even given the large differences in numbers of concepts between Cifar10 and CUB?\n- Coverage of the concepts is the percentage of concept predictions done by the model. However, particularly in CUB there is a large number of concepts the model predicts correctly (~95%). A concept coverage of 80% thus means that the model queries at least four correct concepts for one incorrect concept in the best case, is this correct? And given that, the model still performs quite badly (in Fig 3a) in these situations compared to previous strategies that investigate interventions (e.g. LCP in [1]), when comparing only interventions (or deferring) on the concepts. Can the authors discuss this case (and the relation to previous work on interventions) in more detail?\n- As DCBM allows for deferring of both the task label and the concept label, is there any accounting for the fact that deferring at the concept level could already be sufficient and make deferring on the task label for this sample unnecessary?\n- Is there an argument for not training the concept predictor fully based on the input when using DCBM training (as e.g. the original independent scheme from Koh et al.)?\n\nLimitations:\nThe limitations of this work are not sufficiently discussed in the paper. I recommend either the creation of a separate limitations subsection or having them clearer and structured as part of the conclusion.\nI see the following limitations/discussion points:\n- (As briefly mentioned in the paper): Deferring costs are the same for concepts and tasks. While task-level deferring might be in line with the previous learning to defer setup, the main focus of deferring on CBMs should be concept-level, as task-level deferring is assumed to be substantially more expensive. A different approach would be showing that the predicted concepts make task-level deferring easier for humans than doing the same for a black-box model and thus more applicable, but this would require more experiments.\n- The approach only works reliably as long as the expert performs the same between training and testing. This also implies that the model cannot necessarily adapt to a different human with other strengths. While I understand the argumentation why and where learning to defer improves upon interventions, the current method loses many of its benefits when this assumption is not given anymore.\n\n\nMinor points:\n- Eq. 4: Large braces would help to clarify what the sum covers, and the final ',' seems unnecessary\n- Clarity of figures and captions:\n\t- As Figs. 3 and 4 are quite information dense; it might be helpful to mention the key messages in the caption to improve reader understanding. Additionally, the lines overlap a lot, which makes it quite difficult to follow the results sometimes. \n\t- The caption of Fig. 5 and the in-figure descriptions could be improved to make the figures easier to understand: In particular, the in-figure header of b could better reflect the content of the plot (i.e., the probability for deferral.) Additionally, I recommend not using red and green to improve readability for color-blind people.\n\n\n[1] Shin, Sungbin, et al. \"A closer look at the intervention procedure of concept bottleneck models.\" _International Conference on Machine Learning_. PMLR, 2023.\n\nLIMITATIONS:\nThe limitations are not sufficiently discussed (see above).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis works introduces a Learning to Defer (L2D) framework into Concept Bottleneck Models, proposing Deferring CBMs that can learn when an intervention is needed through a composition of deferring systems. The authors show that intelligent deferring offers not only improved performance, but also paves the way for future work on explaining deferrals for better human-machine interaction.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The paper is clearly written.\n- Exploring human-in-the-loop interactions with interpretable systems is highly useful for real-world deployment of these models.\n\nWeaknesses\n- Lack of baselines. The paper presents a Learning to Defer approach, training a model to autonomously defer on its intermediate concepts. This seems like it could be easily compared to other intervention strategies [1], to show how well the model can improve task/concept performance across a consistent number of interventions. However, the authors claim that \"works on concept interventions fundamentally differ from our L2D-based approach in that they assume that experts themselves trigger a correction in a model’s concept predictions.\" It is unclear what the authors mean by this.\n- Overall the results do not seem to adequately support the main claims of the paper. Both the main claim that human experts may introduce errors that \"muddle the effects and dynamics of the human-AI collaboration expected when using CBMs\" (harm performance? harm something else?) or that their proposed DCBMs adequately handle this human uncertainty, especially compared to other intervention strategies.\n\nTypos\n- Line 14? \"DCBMs can explain why defer occurs on the final task\"\n- Line 198 \"syntethic\"\n\n[1] Shin, Sungbin, et al. \"A closer look at the intervention procedure of concept bottleneck models.\" International Conference on Machine Learning. PMLR, 2023.\n\nQUESTIONS:\n- The accuracy-interpretability claim seems unsubstantiated by the cited paper (CEM), which states that \"embedding-based CBMs (i.e., Hybrid-CBM and CEM) can achieve competitive or better downstream accuracy than DNNs that do not provide any form of concept-based explanations\". Could the authors explain what they mean by accuracy-interpretability trade-off in this case?\n- There seems to be a distinction that exists between the initial concept annotations being inaccurate, and an actual human-in-the-loop being inaccurate, but they seemed to be conflated in the text. For example, the difference between an Oracle and Uncertain human in Figure 3 is unclear, and the authors only state that \"we use them to produce random human concept labels as done by Collins et al. [2023].\" Could the authors elaborate this experimental setup further?\n- Similarly, do DCBMs utilize this concept uncertainty information directly? It seems the human predictions utilize this inherent uncertainty, but it is unclear how this extends to real-world tasks.\n- Could the authors compare to recent work on informed intervention strategies or explain how their work cannot be compared?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a framework for addressing concept bottlenecks in deferring systems. The core contribution is a method to improve interpretability and robustness in such systems by explicitly modeling deferral mechanisms. The authors formalize the problem using theoretical guarantees and validate their approach on synthetic and benchmark datasets. The work bridges technical rigor with practical considerations for deploying deferring systems in real-world scenarios.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths\n  - The paper is clear and easy to follow. And the writing is concise.\n  - First work to combine deferral with concept bottlenecks, enabling interpretable L2D. The coverage-interpretability-accuracy trade-off is a novel perspective.\n\n- Weakness\n  - The multi-variable loss assumes concept independence, violating real-world hierarchical/causal structures. Independent deferral decisions may propagate errors when parent-child concept relationships exist.\n\nQUESTIONS:\n- In the paper, the expert error is assumed to follow a fixed distribution, but real-world expert errors may correlate with input data complexity (e.g., higher misjudgment rates for rare cases). How can we validate the robustness of DCBM under dynamic error scenarios? Additionally, if human experts exhibit malicious or strategic behavior, does DCBM’s deferral mechanism collapse?\n- The delay cost (λ) is treated as a global hyperparameter, but in practical scenarios, the delay costs for different concepts or tasks may vary significantly, for example, correcting \"tumor size\" in medical diagnosis is more urgent than revising \"blood pressure measurements\". Can we design a locally adaptive cost function that accounts for task-specific urgency or contextual priorities?\n\nLIMITATIONS:\n- The experiments only measure coverage but do not quantify decision latency. In real-world systems, human response delays could cause prediction chain blocking in DCBM. Perhaps an asynchronous deferral mechanism could be introduced: allowing the model to emit provisional predictions while awaiting human responses.\n- To increase coverage, DCBM must reduce deferrals, but deferred concepts are often critical explanatory factors with high model uncertainty. This leads to degraded explanation quality at high coverage rates.\n- The computational overhead of the joint training strategy remains unquantified.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nCBM are a family of models that are interpretable by design by conditioning on human-understandable concepts. This paper introduces: Deferring CBM, a type of CBM that allows deferring of uncertain concepts to humans.  Inspired by Learning to Defer, DCBM is trained for the cases when human intervention seems necessary to the model.  The authors perform experiments on CUB and CIFAR-h to show the effectiveness of deffering to humans.\n\nSTRENGTHS AND WEAKNESSES:\n# Strength\n\n- This is a great work that attempts to add human-in-loop setting for CBMs.\n\n- The paper is generally well-organized.\n\n- The authors theoretically motivate and bridge the gap between L2D and CBMs.\n\n# Weakness\n\n\n- I am struggling to understand the reason to defer to humans for the task prediction. The original and many of the following works of CBMs were motivated by the fact that concepts might be easier to intervene on than the task prediction. Hence, your setting to defer concepts is well motivated, but I dont understand, what is the motivation of intervene on the task? This is one of my main concerns. \n\n- There are already many works that suggest that concepts aren't independent (and there is leakage between them), and they are also cited in the paper. I assume concept independence is, hence a strong assumption. \n\n- I suppose deferring is another way of intervention on concepts. It would be natural to ask for the same about of interventions, how does the performance fair?\n\nQUESTIONS:\n- I didnt understand why task accuracy reduces when there is more defering? L222\n\n- In Fig 5, it seems that some of the concepts are not easy to grasp, and realistically if a human cannot identify the concepts, defering is futile. Whats the genuine application of DCBM?\n\n- Why do you defer tasks as well?\n\nLIMITATIONS:\nLimitations are not mentioned. \n\n- Human study to validate", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}