Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this work, authors analyze the current trade-off between label and concept accuracy in concept bottleneck models when the number of concepts reduces. They prove this trade-off to be inherent to standard CBM  models both theoretically and empirically. Based on their observation, they propose a new approach named DCBM, which allows the model to learn additional "implicit" concepts to improve label classification performance when concepts are missing while ensuring good concept classification. To avoid models to bypass explicit concepts in the decision-making of their downstream task, the authors also ensure low JS divergence between explicit concept predictions and label predictions. They show DCBM superiority in a low concept regime over previous CBM models on two benchmark datasets. They also show that their model performs well under forward intervention and their newly introduced backward rectification task. 
strength_and_weaknesses: Strengths: 
- The problem of performance decrease with the number of concepts is highly relevant to the field of interpretability given CBM's popularity.
- The authors have valuable experiments and theoretical findings supporting the existence of limitations for CBM in low concept regime
- The method and metrics are well explained making the paper easy to follow.
- Contrary to other works, the authors consider the possible issue of shortcut learning through implicit concepts and propose a solution for it. 
- While their method is clearly better in all classification aspects compared to CBM, the authors also perform an analysis of the computational overhead.  
- Authors provide experiments on the impact of both introduced parameters, JSD weight $\beta$ and concept-label trade-off term $\alpha$. (but in the appendix, see below) 

Weaknesses:
- The experiments section is sometimes hard to follow, there are some key aspects of the experiment that are unclear to me:
  -  What is the value of $\beta$ in Table 2 for DCBM ?
  - How did the authors select $\tilde{d}$, the implicit concept dimension ? What is the impact of this parameter on performance?
  - What do the authors mean by "computational overhead" in table 5, is it compared to CBM alone? If yes, what is the relative comparison? Is this time per epoch or over an entire training process?
- The authors did not provide code yet which forbids any conclusion with respect to reproducibility.
- The impact of JSD weight $\beta$ is core to this work, as it's the only way to avoid shortcut learning through implicit concepts. Thus the content of figure 7 belongs in the main paper and should be further discussed. 
- Results from Figure 7 (c,d) and Table 3 seem to be inconsistent. Indeed in Figure 7, for CUB 10% and 20% DCBM($\beta$ = 0.01) relies mainly on implicit concepts with an accuracy below 50% using explicit concepts only. However, in Table 3, forward intervention (which only affects explicit concepts) has a greater impact than for CUB and CUB 50% where DCBM($\beta$ = 0.01) does achieve good classification accuracy from explicit concepts. The authors never discuss this matter. What are the underlying reasons?


This is not a weakness given the recentness of the work, but I believe authors should consider citing "Clinical outcome prediction under hypothetical interventions – a representation learning framework for counterfactual reasoning" (2022) that propose partial concept bottleneck (PCB). They also propose to learn another discrete representation to alleviate the lack of concepts. However, their work is more limited and does not consider the crucial issue of shortcut learning from implicit concepts making interpretability impossible. 
clarity,_quality,_novelty_and_reproducibility: All sections except the results one are perfectly clear and well-written. The result section would gain a bit of clarity by moving certain details to the appendix while bringing ablation on $\beta$ to the main paper. This work shares ideas with a concurrent work (see comment above), but is of a higher quality and additionally addresses shortcut learning problems with implicit concepts. Thus, I believe the idea to be novel. The problem and conducted experiments are well-motivated but providing extended details about the experimental setting and baseline is needed. Reproducibility could be improved by providing code. 
summary_of_the_review: Overall the authors propose a simple and efficient method for a highly relevant problem in the field. Their work is clearly structured and written. Their experiments show strong results and are well-motivated. However, crucial details surrounding these experiments and discussions are missing to back the authors' claims, in particular around the JSD weighting parameter $\beta$ and its impact.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper examines concept bottleneck models and focuses on the case where these models have unexpected behaviour when there is insufficient concept information. The authors propose a strategy for decoupling CBMs into explicit and implicit concepts while retaining high predictive performance and interpretability. They devise two algorithms based on mutual information to automatically correct labels and trace back to incorrect concepts. Based on this a human may intervene to correct incorrect concept definitions.
strength_and_weaknesses: Overall the work addresses a very relevant problem and presents a good solution to the proposed issue. The method is well motivated and the authors make several comparisons on CUB and DERM7PT data sets.

There are definitely several shortcomings though. Perhaps the most critical of these is the fact that the authors are missing several crucial citations that can potentially impact the quality of the analysis and significance of the results. In particular, the authors never mention works such as Mahinpei et al 2021 (https://arxiv.org/pdf/2106.13314.pdf) who describe some of the major pitfalls of CBMs. Mahinpei et al 2021 explicitly talk about this unexpected behaviour of concept bottlenecks in terms of leakage and several followup works from the same group e.g Havasi et al 2022 present some solutions with a side channel using mutual information to separate the concept information into two subsets to overcome this leakage issue.  I would like to see some comparisons here as to how does the issue of having insufficient concept information relate to leakage? And similarly, how does the approach compare against Havasi et al 2022.
 
Mahinpei et al show that while Concept models encourage concept dimensions to be uncorrelated, this does not completely
prevent information leakage – each concept dimension can still encode for multiple concepts and that the
concept dimensions can nonetheless be statistically dependent. They state that CBM training should explicitly
minimize mutual information between concept dimensions – both aligned and unaligned - if concepts are believed to be independent. Similar views are presented in Klys et al 2018. How does the proposed method compare to these suggestions? 

Finally, I would also like to see some discussion and analysis on how having the human-in-the-loop affects the concept definitions. Is it expected that the human intervenes on the concepts akin to interventions in the causal sense?


clarity,_quality,_novelty_and_reproducibility: The work is well written but parts are incomplete in the sense that several crucial comparisons are missing and as a result the work doesnt feel as novel.


summary_of_the_review: Overall the paper is well written but the experimental section needs a lot of work in terms of comparing with existing works tackling similar issues and a significantly better job needs to be done in terms of contextualising the contributions of the paper compared to existing works. 

The work seems to lack novelty for the most part.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This is a strong paper that proposes methods that right issues arising from concept bottleneck models. The authors show how to split concepts into explicit and implicit information, and then demonstrate how to debug concept and label errors.
strength_and_weaknesses: Strengths
- The paper's presentation is strong. Figures are immaculate!
- The method addresses serious issues with CBMs. I would have preferred a strong motivating (running) example that naturally suggests the need for implicit and explicit concepts.
- The theory seems correct; however, the authors could do a much better job at motivating its need (perhaps using the aforementioned example). At the moment, it is unclear how the proposed theory impacts the subsequent algorithms/experiments.
- The notion of forward intervention and backward rectification are sensible and interesting. 

Weaknesses
- While the paper is quite a whirlwind and looks thorough, there are some clarity issues. Instead of easing readers into a clear, convincing narrative, the authors elected to inundate the readers with results without much interpretation. 
- There is a nomenclature issue with Section 6. It's a stretch to call the proposed system as a human-machine system. There is no human subject experiment to validate this. It's unclear if humans will intervene and rectify sensibly. The experiments presume orcale-esque behavior from the human. Instead of walking through the results, the authors make it difficult to follow the results and their implications. Supporting prose can be of much help to all.
clarity,_quality,_novelty_and_reproducibility: Clarity: Discussed above.
Quality: Adequate
Novelty: Adequate
Reproducibility: Adequate but there was no code release for review.
summary_of_the_review: This was a very interesting paper that clearly has legs to improve CBMs. However, the lack of clarity in message makes it hard to accept as is. The density of the paper could be reduced to provide the readers clear motivation for the proposed methods.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Concept Bottleneck Model (CBM) has the advantage of interpretability. One of the issues that limits their use is insufficiency of high-level concepts, that is, available concepts (explicit concepts) do not encode sufficient information to predict class labels. Hence, the paper suggested to use additional concepts (implicit concepts) which are essentially additional neurons in the bottleneck layer. Implicit concepts are introduced to avoid pollution of explicit concepts during training. The paper proposes Decoupling Concept Bottleneck Model (DCBM) which is a concept-based model decoupling heterogeneous information into explicit and implicit concepts. Moreover, they proposed two methods (forward intervention and backward rectification,) which can automatically correct labels and trace back to wrong concepts.

strength_and_weaknesses: +The theoretical arguments about the tradeoff between concept prediction and label prediction accuracy when concepts are not sufficient to predict class labels are interesting and insightful. 

+The paper focuses on decoupling implicit and explicit concepts which is an interesting solution to keep the explicit concept prediction accuracy intact and maintain high label accuracy.

+Maintaining high interpretability in CBMs is important and the authors have tried to preserve it by using a JS Divergence term.

+Leveraging mutual information minimization to enable forward and backward intervention on explicit concepts is an interesting approach that the authors have employed.

----------------
-Backward intervention is poorly described. Figure 4 ---which is about forward and backward intervention--- is not discussed in the text at all. The related equations are not properly described and are deferred to the appendix A but that appendix contain only a sudo-code with no additional explanation about the backward intervention formulation. What are the intuitions behind eq 4, 5 and 6?

-DCBM is claimed to be an interpretable model, but because the final decision is made based on both human-friendly concepts (explicit concepts) and implicit concepts (not interpretable), it seems that the entire model becomes not interpretable. Although JS Divergence loss term is applied to reduce the impact of implicit concepts on the prediction, the final decision still depends on both implicit and explicit concepts. 

-It was mentioned that joint concept/label training in CBM suffers from concept distortion and the concept decoupling model is proposed to mitigate this problem. When the weight of JS Divergence goes to infinity, DCBM converges to CBM. When using less number of explicit concepts, it is expected that the label accuracy of CBM (as well as DCBM with large JS weight) decreases. But we don’t see such a pattern in Figure 7. Looking at sub-figures a-d they all have a similar label accuracy of around 80%. This is counterintuitive. According to Figure 7, CUB%10 and CUB have almost similar CBM label accuracy. Same for Exp-CBM with the largest JS weight. 

-A similar counter-intuitive observation can be made in Table 3. Label accuracy reported for CUB, CUB%50, CUB%20 and CUB%10 all are in the range of 80%. I expected that all methods perform much worse on CUB%10 dataset. How this similar performance can be justified?


-Another counterintuitive behavior in Figure 7 is about the label accuracy for IMP-DCBM. My understanding is that as the JS weight increases, implicit concepts become less informative about class labels (harsher decoupling) and the label accuracy of IMP-DCBM should decrease. But we don’t see such a pattern in any of the sub-figures of Figure 7. In Figure 7, by increasing JS weight, label accuracy of IMP-DCBM increases and then decreases.  How the initial increase can be justified? 

-How can we guarantee that the backward intervention captures wrong predicted concepts and not correct concepts?

-What are the exact loss terms used as L_Y and L_C. Are they conventional cross-entropy? 

-In Table 4 that the concept accuracy on samples with wrong label prediction is reported, what is the proportion of the misclassified samples that the authors do intervention on? Did you rectify all the samples with a wrong assigned label?
-It would be interesting to have an experiment to show how well the decoupling network is able to reduce the degree of mutual information when different number of explicit concepts are used.

clarity,_quality,_novelty_and_reproducibility: Please see my comments above
summary_of_the_review: It seems that there are interesting ideas but not properly described. More importantly, I have major concerns about the experiments and I hope the authors can clarify.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors introduce a new version of the Concept Bottleneck Model (Koh et al., 2020) that dims to be more accurate and interpretable w.r.t. other Concept-based models, especially in regimes where concept supervision is limited.
 The proposed method is inspired by a theorem depicting the trade-off between the concept error and the label error in the Joint Concept-Bottleneck model (CBM), due to missing concept information.  Hence, the authors propose a new version (DCBM) of the original model to decouple the information between explicit and implicit concepts. They conduct an extensive experimental comparison with other known CBMs and a standard black-box model, achieving overall better label/concept accuracy performances. 
Along with a novel strategy to train the DCBM, they also propose a new two-stage human-machine interaction, addressing both the correction of the wrong concepts and of the misclassified labels. This proves effective and improves previous methods, without requiring costly time computations.


strength_and_weaknesses: This paper addresses the important problem of the accuracy/interpretability trade-off of Concept-based models. In this respect, the authors prove a theorem, based on known results in Random Matrix Theory, which reveals an inherent trade-off between the concept error and the label error.  While the assumptions are a bit restrictive (the weights are assumed to be sampled independently from a gaussian distribution, which excludes real-world scenarios where correlations among weights are important) the lower bound relates the expected error on the label to that on the concepts, which is a first, relevant theoretical result for Concept-based models. Another limitation is inherited by the dimensionality of the classification space, which is assumed to be $k < \min(d_1, d_2)$, where $d_1$ and $d_2$ refer to the dimension of the explicit and implicit concepts, respectively. It seems that the CUB dataset violates the assumptions of the Theorem, being not applicable in that scenario (in fact, in the CUB-100% dataset d_1=112, where k=200). 

Following, motivated by the theorem, the authors suggest that decoupling the explicit concepts and implicit concepts should alleviate the overall label/concept distortion. Upon carefully analyzing the result of the theorem, it is still not clear to me how this claim is effectively supported. In fact, the quantities in Eq. (10) are neither evaluated nor accounted for in the formulation of the model.
As it stands, the theorem yields an interesting result but it seems not so related to the model proposed by the authors nor to the experimental evaluation. 

The experimental evaluation of the label/concept accuracy of the model supports the major performances of the model w.r.t. other Concept-based proposals. 
For what concerns the interpretability of the model, a key requirement of CBM is the prediction to depend mostly on the interpretable (in this case explicit) concepts. 
As stated by the authors, an important parameter of the DCBM is the weight $\beta$ of the Jensen-Shannon (JS) divergence, which regulates the influence of the implicit concepts for the prediction. To retain interpretability, the prediction should depend mostly on explicit concepts and, in principle, high values of $\beta$ should enforce this property. This holds as long as the problem of predicting $Y$ given $\hat C$ is linearly separable. Does this condition hold with the selected concepts in the CUB experiments? If not, the explicit concepts themselves would not suffice to predict correctly the label; hence optimizing the JS divergence would naturally deteriorate the model accuracy. When unsupervised concepts are relevant for prediction, they should be somehow interpretable, otherwise, the interpretability of the model falls off. For example, in Concept Bottleneck Model With Additional Unsupervised Concepts (in short CBM-AUC) by Sawada and Nakamura, IEEE (2022), additional unsupervised concepts are extracted by a self-explainable neural network (Alvarez-Melis and Jaakkola, NeurIPS (2018)), which guarantees in principle their interpretability. Since CBM-AUC is a natural competitor of DCBM, it should be clear why it is not considered in the empirical evaluation

In the experiments with CUB-20% and CUB-10%, it seems that any $\beta \leq 1 $ favors the intrinsic concepts to be highly informative of the label, refer to Fig. 7. Then, how can the interpretability of the predictions be guaranteed? For the results in Table 2, which $\beta$ do they refer to? This is not clear from the text, as in Sec. 5.2 the reader is referred to Appendix D, where there is only the study in relative performances upon changing $\beta$. In Appendix C.4 it is reported that: “the best model is chosen by either the best label accuracy or the best concept accuracy on validation set ”. Are those DCBMs with $\beta=0.1$ and $\beta=1$? Conversely, in the intervention experiment, $\beta=0.01$ and $\beta=0.1$ were used (I think there is a typo in the caption of Table 4, where there is $\beta=1.0$ but in the main text, it is said $\beta=0.1$). In this case, the implicit concepts are highly informative of the label, as clear from Figure 7.

Overall, it must be evaluated the relative importance of $\tilde f \circ \tilde g$ on the prediction w.r.t. $f \circ g$ to assess how the implicit concepts affect the downstream task. Without such a metric, interpretability cannot be guaranteed.

The proposal of the two-phase human-machine interaction is novel and interesting: it works effectively w.r.t. other interactive strategies, winning in all scenarios. Also, as a merit, it does not increase sensibly the computational cost. Here, a map $f_{DEC}$ is estimated to decouple the residual concepts from the explicit ones in the implicit layer. While I understood it is only used in the interactive section (correct me if I am wrong), it would be useful to adopt it also in the label prediction, since $\tilde c - f_{DEC} (\hat c)$ does not contain redundant information of the explicit concepts $\hat c$. 

clarity,_quality,_novelty_and_reproducibility: The paper is well-written and, overall, the exposition is clear. I have already addressed my perplexities on the quantities introduced in Theorem 1 and the issues with implicit concepts in the prediction. Further clarifications are required to assess the validity of the proposed model, that in turn would improve the quality and the novelty of the manuscript. In fact, the interpretability of the model cannot be assessed directly from the experiments and it seems not to hold in general. First of all, it should be clearer from the text which parameters $\beta$ in Table 2 have been adopted for the accuracy comparison and how the interpretability of the model is assessed. I found the experiments extensive, however, the natural competitor CBM-AUC was not taken into consideration.  The details in the appendix support reproducibility, and the code repository would be available upon acceptance.

summary_of_the_review: This paper presents a novel theoretical result (Theorem 1 for CBMs), which is relevant for future developments of interpretable concept-based models. Theorem 1 indicates an inherent trade-off between label and concept accuracy, up to some assumptions. However, it is not evident why the proposed model should address the limitations of CBMs, in light of the quantities introduced in the theorem. A decoupling phase is introduced in the interactive phase but is not present when giving the prediction, referring to Fig. 3. 
In the experimental investigation, the proposed model (DCBM) achieves better downstream and interventional label/concept accuracy than CBM, especially in the regime of reduced-concept supervision. 
Still, the claim of being interpretable is not fully supported by the experiments, whereas they indicate the implicit concepts (which are not interpretable) are highly predictive of the label in all regimes tested (exception made for $\beta=10$ in all datasets, Fig. 7).
Since the proposed model makes use of implicit concepts for the prediction, their relative importance has to be quantified somehow. Guaranteeing interpretability is necessary for DCBM, and for concept-based models in general, but it was just assumed by the authors. There is still the possibility that implicit concepts are not so relevant for the prediction, but it has to be shown. I am open to revising my score if this point is clarified.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: (a) The paper examines one issue for concept bottleneck models where there is insufficient concept information. They argue that in these cases, the concept bottleneck model may either have less predictive power or have concept distortion (only when concept and model are trained jointly). They thus propose to design a CBM model that has implicit concepts to retain high predictive performance, and also algorithms based on mutual information to automatically correct labels and trace back to incorrect concepts. Based on this a human may intervene to correct incorrect concept definitions. (b) It tackles one major issue of concept bottleneck model, and also propose an intervention where human can interact with the explanation (c) One weakness of the paper is that the implicit concepts does not address the concept leakage issue. Since this work attacks CBM that the explicit concepts may include information of implicit concepts (which leads to concept distortions), it is natural to ask the question: does the proposed DCBM's explicit concept also include information of implicit concepts. This question is not addressed by the author, which raises questions on the significance of the proposed method. Another issue is that the idea of separating concepts into supervised concepts and unsupervised concepts has been proposed by [Concept Bottleneck Model With Additional Unsupervised Concepts, IEEE, 2022 April]. While DCBM shows better performance on label accuracy, the implicit concepts in CBM-AUC is more interpretable than the implicit concepts in DCBM. The contribution of DCBM seems to overlap with the contribution of CBM-AUC.

justification_for_why_not_higher_score: The authors did not investigate the leakage between implicit concepts and explicit concepts, which might make the intervention result less reliable. Also, the idea of having implicit concepts to achieve higher accuracy is not novel, and the comparisons are not entirely fair (CBM-AUC; SAWADA et al.).
justification_for_why_not_lower_score: The motivation is good, and the experiments results are good as many baselines are compared.
Decision: Reject