Decoupling Concept Bottleneck Model
Authors: Rui Zhang
Xingbo Du
Junchi Yan
Shihua Zhang
Venue: Submitted to ICLR 2023 (2023)
Keywords: Interpretability
Concept-based Model

Abstract: Concept Bottleneck Model (CBM) is a kind of powerful interpretable neural network, which utilizes high-level concepts to explain model decisions and interact with humans. However, CBM cannot always work as expected due to the troublesome collection and commonplace insufficiency of high-level concepts in real-world scenarios. In this paper, we theoretically reveal that insufficient concept information will induce the mixture of explicit and implicit information, which further leads to the inherent dilemma of concept and label distortions in CBM. Motivated by the proposed theorem, we present Decoupling Concept Bottleneck Model (DCBM), a novel concept-based model decoupling heterogeneous information into explicit and implicit concepts, while still retaining high prediction performance and interpretability. Extensive experiments expose the success in the alleviation of concept/label distortions, where DCBM achieves state-of-the-art performances in both concept and label learning tasks. Especially for situations where concepts are insufficient, DCBM significantly outperforms other models based on concept bottleneck and respectively achieves error rates 24.95% and 20.09% lower than other CBMs on concept/label prediction. Moreover, to express effective human-machine interactions for DCBM, we devise two algorithms based on mutual information (MI) estimation, including forward intervention and backward rectification, which can automatically correct labels and trace back to wrong concepts. The construction of the interaction regime can be formulated as a light min-max optimization problem achieved within minutes. Multiple experiments show that such interactions can effectively promote concept/label accuracy.


Under review as a conference paper at ICLR 2023
DECOUPLING CONCEPT BOTTLENECK MODEL
Anonymous authors Paper under double-blind review
ABSTRACT
Concept Bottleneck Model (CBM) is kind of powerful interpretable neural net- work, which utilizes high-level concepts to explain model decisions and interact with humans. However, CBM cannot always work expected due the trou- blesome collection and commonplace insufficiency of high-level concepts in real- world scenarios. In this paper, we theoretically reveal that insufficient concept information will induce the mixture of explicit and implicit information, which further leads to the inherent dilemma of concept and label distortions in CBM. Motivated by the proposed theorem, we present Decoupling Concept Bottleneck Model (DCBM), a novel concept-based model decoupling heterogeneous infor- mation into explicit and implicit concepts, while retaining high prediction perfor- mance and interpretability. Extensive experiments expose the success in the al- leviation of concept/label distortions, where DCBM achieves state-of of-the-art per- formance in both concept and label learning tasks. Especially for situations where concepts are insufficient, DCBM significantly outperforms other models based concept bottleneck. Moreover, to express effective human-machine interactions for DCBM, devise two algorithms based mutual information (MT) estima- tion, including forward intervention and backward rectification, which can auto- matically correct labels and trace back wrong concepts. The construction of the interaction regime can be formulated as light min-max optimization problem achieved within minutes. Multiple experiments show that such interactions can effectively promote concept/label accuracy.
INTRODUCTION
Concept Bottleneck Model (CBM) (Koh et al., 2020; Losch et al., 2021) is interactive and interpretable AI system, which encodes the prior expert knowledge into the neural network and makes decisions according to correspond- ing concepts. In detail, CBM firstly maps input images into corresponding high-level concepts, and then utilizes these concepts for downstream tasks. Moreover, CBM gives the ante-hoc ex- planations for the model's predictions due to its end-to-end training regimes and has been widely applied healthcare (Chen et al., 2021; Rong et al., 2022), shift detection (Wijaya al., 2021), algorithmic reasoning (Xuanyuan et al., 2022), and so on.
Explicit Concepts
Implcct C Concepts
Mixed Comcepts
Expected C Concepts for CBM
CNN
(b)
Concepts
Concepts
O0
...
(c)
Mapping
Mixed Concepts Foecibly Ad
Outputs
OOO
O (d)
00
Figure 1:
Concept/label distortions in CBM. (a)
the main pipeline CBM. (b) CBM expects an ideal case with all explicit concepts. (c) Het- erogeneous concepts real-world scenarios. (d) Thus, CBM has to mix explicit/implici concepts achieve both concept/label learning.
However, high-level concepts in real-world sce- narios often suffer troublesome collection and commonplace insufficiency due to the con- sumption of large amounts of resources. Thus, the concept information is usually not sufficient to recover label information. In this circumstance, CBM cannot fit the ground-truth labels only cording to the clean but insufficient concepts. On the contrary, to learn better classifier. CBM has to inject extra information into the concept layer, i.e., sacrifice the concept accuracy (Fig. 1).

---

Under review as a conference paper at ICLR 2023
Forward Intervention
Backward Rectification
of this?
W hat kind of bird is this?
wings, solld breasts, plain head,
Is Groove-Biles Ani for black
Interaction
is Groove-Bi Billed Ani.
No, is Brewer Blackbird.
But I think its head not plain
Why did you make this mistake?
you are right, will more likely a Brewer Blackbird.
BM
Human
mistook head plain.
Two manane interactive tasks in DCBM. Forward Intervention: correcting labels according to the more accurate concepts given by human; Backward Rectification: tracking back to wrong concepts according to the labels updated by human.
Figure 2:
Inspired by such observations, we theoretically reveal that CBM cannot avoid the inherent trade-off between concept and label distortions due to the mixture of explicit/implici information.
Motivated by the theoretical results, we propose Decoupling Concept Bottleneck Model (DCBM), novel concept-based model, to decouple the heterogeneous information into explicit and implicit concepts. In detail, DCBM automatically allocates the implicit concepts to auxiliary neurons, as such the pollution of explicit concepts can be avoided. Furthermore, DCBM maximizes the inter- pretability of the models via the Jensen-Shannon (JS) divergence constraint during training.
Equipped with the DCBM, aim to conduct corresponding human-machine machine interactive tasks, garded as one the most important components for such concept-based models. However, the design of interaction algorithms for DCBM is not straightforward due the correlation between plicit and implicit concepts. To this end, we propose a phase-two decoupling algorithm for DCBM to peel out the explicit information via mutual information (MI) estimation (Belghazi et al., 2018). We formulate it into a light min-max optimization problem solved in minutes. After the phase-two decoupling stage, we consider two interaction tasks, including forward intervention and backward rectification, which can correct labels and trace back to wrong concepts automatically via human- machine interaction (Fig. 2).
In summary, we make the following contributions:
To our best knowledge, we are the first theoretically reveal and prove the existence of inherent concept/label distortions in CBM for the situation where high-level concepts are insufficient.
Motivated by the theoretical results, we propose Decoupling Concept Bottleneck Model (DCBM), novel concept-based framework, to alleviate the distortions by automatically decoupling the con- cepts into explicit and implicit ones during the training process.
Extensive experiments show that DCBM achieves state-of-the-ane results in both concept and label learning tasks. In particular, compared with other concept- concept-based based models, the concept/label distor- tions can be alleviated conspicuously for DCBM with inadequate concepts.
We devise a novel comprehensive numan-machine system, which can automatically correct labels and ascertain wrong concepts via feedback from human experts. The system designed to decouple heterogeneous information by solving a light MI-based optimization problem within minutes.
2
THEORETICAL RESULTS
We firstly introduce Concept Bottleneck Model (CBM) (Koh et al., 2020), and then present the dilemma of concept/label distortions CBM theoretically when the concepts are insufficient.
2.1 CONCEPT BOTTLENECK MODEL (CBM)
Given training triples (In, Cn. yn 1N
where T., E R D C2 E Rd.. yn E R respectively indi- cate imput, concept, and label vector. With g RD Rdd mapping from input space into concept space and f Rd # mapping from concept space into label space, CBM defines two loss func- tions, i.e., Ly and Lc, which measure the difference between the ground-truth labels (concepts)
2

---

Under review as a conference paper at ICLR 2023
(b)
(c)
Concept Prediction
Jensen Shannon Divergence
Mapping
(a)
Label
CNN
Prediction
Decouple
Mapping
Prediction
Concepts
O Outputs
Explicit Concept Values
Implicit Concept Values
Exp-Outputs
Imp-Outputs
Figure 3: Pipeline of the proposed DCBM. "Exp-Outputs" and 'Imp-Outputs' represent the outputs of explicit and implicit concepts, respectively. We jointly consider three objectives, including a) Label prediction, to optimize the label accuracy; b) Concept prediction, to optimize the concept accuracy; c) JS Divergence, force the model make predictions according explicit concepts. As such, the issue of forcible concept/label distortions in CBM can be automatically alleviated.
and the predicted ones, respectively. Note that CBM utilizes the supervision of concepts during training while having access them at test time. Specifically, CBM devises three constructions to learn the functions g and f,, i.e., Independent, Sequential, and Joint models, which are introduced in Appendix detail.
2.2 MAIN THEOREM
We state simplified version of our main theorem in this section. The full version and its corre- sponding proof can be seen Appendix C.
from Theorem (Informal). Consider a dataset {In, c,, yn Un generated from the regime X + C y with insufficient concept information. Let dı (d2) denote the number of known (missing) concepts, and Ey (Ec) denote the label (concept) error. Then, the following inequation holds for Joint CBM with high probability:
Ey 2 (d1,, d2) - 2yEc.
(1)
where 7 E IR+, and ((d,, d2) E R+ increases (decreases) with da (di). respectively.
The analysis of the main theorem consists of two steps: (1) analyzing the relationship between the number of missing concepts and the label еттог for Sequential CBM: (2) revealing the trade-off between concept and label error for Joint CBM via the perturbation theory of least square prob- lem (Higham, 2002). On the one hand, Theorem reveals that the weighted sum of the concept error (Ec) and label error (Ey-) are bounded by + On the other hand, ((d1, d2) increases with more missing concepts d2, thus the concept and label error of CBM will also increase correspondingly. Hence, Theorem reveals that the concept/label trade-off will get worse with the missing concepts.
METHODOLOGY
Based on the above theorem, we argue that CBM suffers from concept/label distortions. Thus, we embark devising Decoupled Concept Bottleneck Model (DCBM) to relieve such distortions.
3.1 DECOUPLING CONCEPT BOTTLENECK MODEL
According to the Theorem and its proof, CBM has to inject implicit information into the explicit to achieve low label error, which will obviously sacrifice the concept accuracy. To decouple the aforementioned heterogeneous information, we devise extra mapping f ÿ the implicit
3

---

Under review as a conference paper at ICLR 2023
Ground-truth Label
Intervention
Ground-truth Explicit Concepts
Rectification
Mapping
Decoupling
NN
Mapping
Prediction
C Concepts
Outputs
Explicit Concept Values
Exp-Outputs
Simulated Explicit Concepts
Implicit Concepts Values
Imp-Outputs
Residual
Forward Intervention Process
Backward Rectification Process
Figure 4: Forward intervention and backward rectification in DCBM. Based on MI estimation, im- plicit concept values are decoupled into two parts, i.e., simulated explicit concepts and residual, to extract the information of explicit concepts from implicit ones. In forward intervention, we both intervene explicit concepts and simulated explicit concepts through ground-truth explicit concepts, to promote the label prediction accuracy. In the process of backward rectification, DCBM can auto- matically trace back to the cause through a light optimization task.
information the concept layer of previous CBM (Fig. 3). consider two loss functions:
C, Ly [(f (( un], L2 Σ Lc (c(o(
where ÿ RRD + Rd maps x,, into implicit concept space and f Rdd R maps implicit concept values into the label space.
However, such designed loss functions cannot always protect the model's interpretability. since the predicted label might mainly depend on f regardless of f o To ensure the model interpretabil- ity, add auxiliary JS Divergence constraint as follows to force the model to make decisions according to the explicit concepts as far as possible:
L3 Σ Is [f f fz g(x,))
(3)
The overall loss is LDCBM = L1 + L2 B. La. where a and |3 are hyperparameters to control the weight of different objectives. Note that the classification accuracy of explicit parts, i.e., I g. can be guaranteed by appropriate 3, while the DCBM converges to CBM when 3 tends to co. However, is not that the higher the better, and further analyze the choice of 3 in Sec. 4.4.
Interpretability of DCBM. Even with extremely insufficient concept information, DCBM can guar- antee the relatively low concept/label error simultaneously, however, it is impossible to expect that all model decisions can be interpreted according to the limited concepts. Under this situation, we aim make full use of the concept information and give interpretations for many samples as possible. To this end, we utilize the JS Divergence as metric to evaluate the interpretability for each data point, which reflects the change of DCBM's prediction caused by implicit concepts, i.e., the black-box parts.
3.2 THE PROPOSED PHASE-TWO DECOUPLING ALGORITHM
In this section, we consider two human-machine interaction tasks for DCBM, including forward intervention and backward rectification (Fig. 2). The forward intervention aims correct the label
4

---

Under review as a conference paper at ICLR 2023
according to the ground-trute trouh concepts given by experts, which is proposed in (Koh et al., 2020) for the first time. Inspired by this task, we believe that a more accurate label will also help the identification of misclassified concepts, and thus, propose backward rectification, novel human- machine interactive task, in this paper. In detail, after DCBM makes decision ý according to the corresponding concepts e, the above two tasks can be formulated follows.
Forward Intervention. Human experts check the learned concepts è for DCBM, and correct them the right concepts c'. Then, how would the model update its decision according the more accurate concepts c²?
Backward Rectification, Human experts check the learned labels ý for DCBM, and tell the model that the right labels should be y². Then, how would the model trace back to wrong concepts accord- ing the more accurate labels y² given by human experts?
Despite being equipped with clear definitions, the algorithm design is not straightforward due to the correlation between explicit and implicit concept values, and thus, we cannot ignore the explicit information hiding in the implicit one when conducting two interaction algorithms. To this end, we first separate the implicit concepts into the following two parts: (1) simulated explicit concepts, i.e.. the information of explicit concepts, which can be represented as the output of a decouple function; (2) the residual, which has no or limited information of explicit concepts. The above decomposition can be obtained by MI-based optimization problem, and we borrow the tool from MI neural estimation (Belghazi et al., 2018) to overcome the computational cost for MI estimation.
Equipped with the pre-trained decoupling neural network, it is convenient to conduct the above two interaction tasks. For forward intervention, we only need to replace the predicted concepts with expert concepts in the explicit concepts layer and the input of the decoupling network, respectively, which is in line with the green line in Fig. 4. For backward rectification, the main idea is to adjust the explicit concept values make the model's decision agree with the expert label, which is in line with the red line in Fig. 4. The detailed description of the algorithm can be found in Appendix B.
4 EXPERIMENTS: CONCEPT/LABEL LEARNING
Referring to (Koh et al., 2020; Yuksekgonul et al., 2022), we perform experiments two benchmark datasets, including CUB (Wah et al., 2011) and Derm7pt (Kawahara et al., 2018), to show whether the concept/label distortions can be effectively relieved. Meanwhile, experiments variants of CUB are performed to show the existed concept/label distortions that CBM is suffering. We introduce the details of datasets and other baselines Appendix D.1 and Appendix D.2.
three
4.1 CONCEPT/LABEL DISTORTIONS IN CBM
We show that there indeed exists a forcible concept/label distortions in all CBMs obviously. Take CUB dataset for example, we execute concept and label learning tasks under several cases with different ratios of concepts, namely CUB%10, 10, CUB%20, CUB%50, as well as the original CUB with no concept discarded (Fig. 5). While = 0.01 is the best concept weight introduced by (Koh et al., 2020), we still present another one, i.e., a = 0.005, performing better on label learning task. As expected, all these models undoubtedly suffer a severe distortion in either or both tasks, especially label distortion for Independent and Sequential models, and concept distortion for Joint model with 0.005. This phenomenon inspires to think about the underlying cause and envision an ideal model structure that can alleviate concept/label distortions.
4.2
METRICS AND MODEL SETTING
We introduce the metrics and model setting in Appendix D.3 and Appendix D.4, respectively.
4.3
BENCHMARKING MODEL ACCURACY
DCBM is compared with concept-based models, including Independent, Sequential, Joint, Concept- based Model Extraction (Kazhdan et al., 2020) (CME), Concept Bottleneck Model with Additional Unsupervised Concpets (Sawada & Nakamura, 2022) (CBM-AUC), and some models without in- terpretability, including Standard models with/without bottleneck, and Multitask model (Table 1).
5

---

Under review as a conference paper at ICLR 2023
Label Learning on CUB
Concept Learning on CUB
80
95 B
Accuracy (%)
Accuracy (%)
60
90
IND
40
SEQ
IND SEQ
JOINT (a=0.01)
85
JOINT (a=0.01)
20
JOINT (a=0.005)
JOINT (a=0.005)
80
10%
20%
50%
100%
10%
20%
50%
100%
Concept Ratio
Concept Ratio
Figure 5: Label (left) / Concept (right) prediction accuracy CUB when concepts are randomly dropped. We compare different cases with 10% concepts. As expected, Indepen- dent (IND), Sequential (SEQ), and Joint models suffer an obvious distortion in either both tasks.
Though Joint model had little trade-off between label and concept when concepts are sufficient, we still find the amplified concept/label distortions with more concepts invisible. To verify this, give two groups of concept weights to ensure the best performance in either label or concept learning task, which are 0.01 /0.005 in Joint model and 0.1 0.100.01 in DCBM. The results in Table indicate that DCBM outperforms all other approaches, and even surpasses the models with no in- terpretability. Note that compared with other concept-based models, with more ratios of concepts discarded on CUB, the performance decline in both tasks is alleviated obviously for DCBM. Espe- cially for DCBM with a 0.1, the decline of label and concept learning tasks -1.09 % and 0.85% 85,, respectively, while it is 4.1 1..11% 1%/2.33% in Joint-0.01, 5.00%/1 1.57% in Joint-0.005. 42.93%/2. Independent model, 30. 1...65%%/...3... 13% in Sequential model, 40.41 %6/1.21% in CME, and 0. 0.70% 1.74% in CBM-AUC. Moreover, DCBM-0.1 significantly outperforms other concept-based models in both concept and label learning tasks on datasets with insufficient concepts, i.c., CUB20%, CUB 10%, Derm7pt.
in
The results of DCBM-0.1 in Table show a counterintuitive but interesting phenomenon that label accuracy even increases with higher ratios of concepts discarded on CUB, which is contrary to any other approach. To illustrate this, we together consider label accuracy of DCBM--01, DCBM- 0.01, and Standard model without bottleneck (abbreviated STAN). It is intuitive that with more ratios of concepts discarded on CUB, both DCBM-0.1 and DCBM-0.01 will lose interpretability and converge STAN, wonder the lahel accuracy of DCBM-0.1 and DCBM-0001 converges to the one of STAN, which is 82.54%. Therefore, the reason in our conjecture that the label accuracy of DCBM-0.1 decreases is because
0.1 is large that it forces the model learn concepts better while, to some extent, affecting the label accuracy. However, for DCBM-0001, = 0.01 is small and under this situation, the concepts can promote label accuracy.
4.4 WEIGHT OF JS DIVERGENCE
Since the prediction is based on both explicit and implicit outputs, it is interesting question how the explicit and implicit outputs themselves influence the final prediction. Fig. 6 shows the Label (a-e)/ Concept (f) accuracy for DCBM with 0.1 Derm7pt, CUB, and three variants of CUB, where EXP-DCBM and IMP-DCBM present the prediction results only harnessing explicit and implicit outputs, respectively. The results show that EXP-DCBM keeps a very low label accuracy when the weight of JS Divergence is close to 0 and gradually transcends with the rise of weight. By contrast, IMP-DCBM has rather higher label accuracy than EXP-DCBM when the weight of IS Divergence is close to O. and keeps stable or increases slightly with the rise of weight, but suddenly drops to a very low degree when the weight achieves a very high level, e.g., 3=10.0 on CUB and its variants. We argue that large weight of JS Divergence forces EXP-DCBM to be close to DCBM, and thus might damage the influence of IMP-DCEM. In Appendix E.4, we also illustrate counterintuitive phenomenon why the lable accuracy of IMP-DCBM rises first and then falls precipitously as shown in Fig. (a-c,e). However, though the label accuracy for DCBM's gradients fluctuates severely, the label accuracy for DCBM itself remains stable, with the exception

---

Under review as a conference paper at ICLR 2023
Table 1: Accuracy (%) of concept and label learning on CUB, Derm7pt, and three variants of CUB. We compare DCBM with interpretable (INTE.) baselines, including Independent (IND), Sequential (SEQ), Joint, CME, and CBM-AUC models. Meanwhile, series of black-box models are compared due to their high performance, including Standard models with bottleneck (STAN-BNK) / without bottleneck (STAN), and Multitask (MULTI) model. To display better performance on either task, we employ two groups of concept weights in Joint model (a /.01/0.005) and DCBM (a 0.1/0.01). The best results in each task are in bold. Our methods are marked brown.
DATASET CUB%20
TASK
INTE.
APPROACH
CUB
CUB%
CUB% 10
DERM7PT
IND
76.65
0.22
74.76
1.61
61.34 = 0.61
33.72
+ 2.12
71.28 + 1.32
SEQ
0.266
73.43 + 0.76
65.22 0.39
45.30
5.44
75.68 +: 0.39
JO1NT-0.01
0.11
79.71 +: 0.67
78.15 = 0.30
75.53 +: 0.29
83.02 + 0.29
JOINT-0.005
0.28
8 1.04
80.66 0.07
18.10 0.36
82.29 + 0.36
LABEL
CME
78.33
74.53
61.78
37.92
79,45 1.48
CBM-AUC
81,26 0.37
81.20 + 0.36
81.26 0.23
80.56 + 0.43
75.47 + 2.83
(ouns) DCBM-0.1
81.01 0.40
81.34 = 0.20
81,58 1
82.10 0.05
85,32 0.39
(ours) DCBM-0.01
0.12 83.95
83.53
0.17
82.87 0.33
82,73 0.31
1.04
STAN-ENK
8193 0.20
81,56
80,39 = 0.50
77.92 = 0.24
83.23 2.43
STAN"
82.54 0.17
82,54 +: 0.17
82.54 0.17
82.54 + 0.17
82.18 + 1.67
MULTI
83.43
82.80 + 0.14
81.35 = 0.08
78.01 + 0.54
84.80 +
IND
SEQ'
96.75 0.03
96.34 0.16
95.93 0.11
94.62 0.54
78.69 1 0.49
JOINT-0.01
96.89 + 0.03
96.60 0.01
95.87 0.03
94.16 0.07
73.83 + 2.72
CONCEPT
JOINT-0.005
94.30 + 0.26
93,53 + 0.73
91.12 - 0.78
82,73 + 0.99
72.54 + 3.45
CME
95.96 +: 0.06
95.78 +: 0.06
95.54 = 0.07
94.75 + 0.09
79.45 + 1.48
CBM-AUC
97.17
81.53
(ours) DCBM-0.1
96.92 = 0.04
96.90 + 0.07
96.66 1 0.03
96.07 - 0.02
80.70 +: 0.22
(ours) DCBM-0001
96.30 = 0.12
06.14-2 + 0.15
94,89 + 0.35
90.84 +: 0.88
76.85 1.91
X
STAN-BNK
91.34 + 0.15
90.37 0.05
87.73 0.16
82.20 + 0.35
77.06 + 0.47
MULTI
96.10 0.46
95.52 0.19
94.34 0.57
89.13
77.07 1.42
Standaadd models without bottlencck share the same structure regardless of the number of concepts. Independent and Sequential model have no difference in x - process. Standard model trained with concepts unsupervised, thus inject linear probe bottleneck predict the
concept values.
(a) CUB
(b) CUB$50
(c) CUB$20
Label Label Accuracy (%)
80
80
Label Aceumcc (96)
80
Label Accumcy
70
70
70
DCBM
DCBM
DCBM
60
EXP-DCBM
60
EXP-DCBM
EXP-DCBM
IMP-DCBM
IMP-DCBM
IMP-DCBM
Joint-0.01
Joint- 0.01
Joint-0.01
0.0
0.01
0.1
1.0
10.0
0.0
0.01
0.1
1.0
10.0
0.0
0.01
0.1
1.0
10.0
Weight of IS Divergence
Weight of JS Divergence
Weight of JS Divergence
(d) CUB%10
(e) DemmPp
(l) Concept
Concept Accuracy ((F)
Label Aceuncy (%)
Label Accurey (%)
80
70
96
DCBM
DCBM EXP DCBM IMP-DCBM
CUB
60
-8
EXP DCBM
60
94
CUB%55
IMP-DCEM
CUB%20
50
Joint-0.01
Joint-0.01
CUB$ 10
40
0.0
0.01
0.1
1.0
10.0
0.0
0.01
0.1
10.0
0.01
1.0
10.0
Weight of Divergence
Weight of JS Divergence
Weight of JS Divergence
0.1 on Derm7pt / CUB / CUB%50, %20,% %10 when the weight of JS divergence varies. EXP-DCBM and IMP-DCBM repre- sent the predictions only harnessing explicit and implicit outputs, respectively. Furthermore, we use dotted line to express the label accuracy of Joint-0.01.
Figure 6: Label (a-e) / Concept (f) prediction for DCBM with a
that the accuracy drops a little due to the nosedive of the label accuracy for IMP-DCEM. Under this perspective, we believe that the choice of is not arbitrary, and not always implies trade-off
7

---

Under review as a conference paper at ICLR 2023
between interpretability and aceuracy. For instance, the label accuracy for DCBM keeps stable or rises slightly. while the one of EXP-DCBM significantly improves when 3 varies from 0.0 to 1.0. In addition to this, shown in Fig. (f), the influence of concept accuracy changes little with different weights of JS Divergence, which indicates the concept robustness in terms of f. Furthermore, the dotted lines in Fig. reveal that the performance of DCBM convergences the one of CBM when 33 is sufficiently large (3- 10).
4.5 OTHER EXPERIMENTAL ANALYSIS
Actually, in addition the weight of JS Divergence, lots of other interesting analysis is worth discussing, which may either affect concept/label accuracy or interpretability and bring about some further findings. We display the relationship between the weight and values of Divergence in Appendix E.1 and how much the dimension of implicit concepts affects concept/label accuracy in Appendix E.2, Moreover, we compare the label accuracy predicted only by explicit concepts with the values of JS Divergence in Appendix E.3 and illustrate in Appendix E.4 why the label accuracy of IMP-DCBM rises first and then falls in Fig. 6.
5 EXPERIMENTS: HUMAN-MACHINE INTERACTION
As introduced in Sec. 3.2, two algorithms based on MI estimation, including forward intervention and backward rectification, are devised to automatically correct label predictions and trace back to wrong concepts, while they can be achieved only by few additional training costs. In Table 2 and Table 3, we experiment the corresponding tasks for DCBM with a = 0.1 and 3 = 0.01/1.0 on CUB, CUB%50, CUB%20, CUB%10 10, and Dptm7pt. The choices of 8 0.01/L.0 respectively represent the best model performance and the best interpretability. It is also worth mentioning that we do not interact with real-world human experts in the experiments but assume that the judgments given by human experts are consistent with the concept/label values in the dataset. Therefore, we only propose a feasible plan of human-machin interaction via DCBM and it requires further examination to demonstrate whether the system can work as expected.
Forward Intervention. In Table 2, we display the noninterventive model performance for Joint model with a = 0.01 and DCBM. Then, for CUB and its variants, we respectively intervene 11 (10% of the total 112) concept values for wrong predictions, while we intervene all the concepts related to pigment network for Derm7pt, to show the intervention promotion, see JOINT-INT and DCBM-INT. Concept values are respectively intervened to the 95th or 5th percentile of the ones over the training distribution for positive/negative samples, which is in line with (Koh et al., 2020). Besides, we perform novel forward intervention algorithm called DCBM-INT-DEC, as introduced in Sec. 3, and as expected, the label accuracy can be further promoted in most cases. Especially for the situation where concepts are insufficient, our proposed algorithm can still provide effective intervention instead of even severely reducing the accuracy Joint model does. As discussed in (Koh et al., 2020), one major reason for this phenomenon is that when the concept accuracy is low, the learned concepts are not necessarily aligned with the ground truth concepts. Similar CME, the label accuracy of DCBM promotes more when more concepts are discarded. The reason is that models with poorer concept accuracy may have larger room for label prediction improvement. Furthermore, we also conduct experiments to reveal how well the decoupling network is able to reduce the MI in Appendix E.5 and its effects during prediction in Appendix E.6.
Backward Rectification. Table 3 presents the unrectified model performance (DCBM) as well as the rectification without (DCBM-REC) with (DCBM-REC-DD Table 3 show the effectiveness of backward rectification, where DCBM-REC and DCBM-REC-DEC succeed in promoting concept accuracy in all cases, especially for DCBM-REC-DEC. Apart from accuracy promotion, we also give true positive rate (TPR) and false positive rate (FPR) to detail such promotion. The former represents the ability to discover wrong predictions while the latter illustrates the disturbance with correct predictions. Through in-depth observation, we find that though the TPR of DCBMM-REC-DDE
MI estimation. Results in
is low, overpasses DCBM- ACBM-DEC in prediction accuracy mainly due to a lower FPR, namely, the number of samples mistakenly corrected by RCBM-REC-JEE DEC is far less than that of DDDCCB-RER Furthermore, we need to declare that the backward rectification task is very difficult because the ratio of misclassified concepts is very low, which is less than 4% in general. Thus, our
8

---

Under review as a conference paper at ICLR 2023
Table 2: Forward intervention Derm7pt, CUB and its three variants. We present the label accuracy (%) for the original Joint model with = 0.01, CME, and DCBM with = 0.1, and then respec- tively intervene 10% concepts to show the promotion, see JOINT-INT, CME-INT, and DCBM-INT. Moreover, we intervene simulated explicit concept values decoupled from implicit concept values (abbreviated as DCBM-INT-DEC), where label accuracy can be further improved. The best results on each dataset are in bold.
DATASET
APPROACH
CUB
CUB%50
CUB
CUB%10
DERM7PT
IOINT (Koh et al., 2020)
79.64 1 0.18
79.71
= 0.67
7
75.53 1 0.229
83.02 1 0.29
JOINT-INT (Koh et al., 2020)
82.04 +: 0.39
83.06 = 0.10
80.99 +: 0.26
67.85 + 0.42
74.15 + 2.93
CME (Kazhdan et al., 2020)
78.33 +: 0.21
74.53 - 0.04 0.04
0.97
37.92 0.39
79.45 + 1.48
CME-INT (Kazhdan et al., 2020)
82.43 + 0.21
80.59 0
71.57 0.36
54.31
79.56 + 1.60
(ours) DCEM
30.82+0..1
81.49 = 0.32
81.38 + 0.44
82.43
84.28 + 1.78
(ours) DCBM-INT*
81.96 +: 0.09
83.39 0.77
84.76 0.24
86.23 0.01
86.06 + 0.59
(ouns) WCBM--NT-DDCC
82.52 +: 0.38
84.32 = 0.71
86.21 = 0.11
87.34 0.38
86.16 + 0.68
(ours) DCBM'
81.01 1: 0.40
81.34 = 0.20
81.58 1 0.51
82. 0.05
85.32 1 0.39
(ours) DCBM-INT
82.59 + 0.25
84.07 = 0.11
85.55 = 0.40
87.91 + 0.25
85.54 + 1.71
(ours) DCBM-INT-DES INT-DEC
82.80 1 0.32
84.46 0.18
85,82
0.53
87.75 1 0.23
86.16 0.77
DCBM with 8 0.01, typically displaying the best model performance.
DCBM with 8 1.0, owning high performance while retaining the best model interpretability.
Table 3: Backward rectification on Derm7pt, CUB and its three variants. We present the original concept accuracy (ACC) of DCBM with wrong. Then, we respectively show the accuracy promotion (ACC), true positive rate (TPR), and false positive (FPR) under an optimization regime without (DCBM-REC) with (DCBM-REC- DEC) Decoupling Neural Network. The best results for each /3 are in bold.
0.1 only samples whose label predictions
DATASET
APPROACH
METRIC
CUB
CUB%50
CUB% 20
CUB%I0
DERM7PT
DCRM
89.36 0.05
88.38
0.11
87.62
0.12
85.24 + 0.14
73.37 2.16
DCBM-REC*
90.02 0.05
89.73 0.12
89.63
0.34
85.49 + 0.47
75.39 = 3.01
DCBM-REC-DEC'
90.25 0.08
90.18 0.04
91.31
0.14
75.30 2.97
ACC (%)
DCBM'
89.23 1: 0.06
88.48 1 0.22
87.48
0.13
85.12
71.59 1 3.14
DCBM-RECC
0.11
90.32 = 0.08
91.22 0.42
90.03 + 0.45
72.13 + 3.89
DCBM-REC-DE DEC"
90.33 1: 0.15
90.48 - 0.05
91.35 11: 0.27
91.03 1: 0.80
72.83 - 4.09
DCBM-REC*
0
31.24 = 0.62
48.03 0.60
64.86 1: 1.07
53.41 = 10.77
DCBM-REC- DEC"
22.49 0.67
27.72 = 0.27
41.87 + 1.82
60.44 + 9.90
53.41 +: 10.77
TPR (%)
DCBM-REC'
24.73 1: 0.51
32.30 : 0.56
46.51 +: 1.31
68.99 1: 1.95
37.80 : 2.03
DCBM-REC-DECC
24.23 0.63
30.99 = 0.88
42.76 + 1.21
66.48 + 3.39
33.74 = 6.73
DCBM-REC*
2.17 + 0.42
2.58 + 0.28
4,49 = 0.41
10.89 + 0.31
22.22 = 2.74
DCBM-REC-DEC*
1.68 + 0.14
1.61 + 0.11
1.70 + 0.08
3.72
21.46 1.34
FPR (%)
DCEM-REC'
1.87 1 0.06
2.12 0.16
2.38 - 0.33
6,36 - 0.44
16.18 = 4.65
DCRM-REC-DEC"
1.69 + 0.07
1.77
0.21
1.70
0.12
4.74
1.21
12.55 6.87
DCBM with 8 = 0.01, typically displaying the best model performance.
DCBM with 8 = 1.0, owning high model performance while retaining the best model interpretability.
main intention is to choose several predicted concepts which are possibly wrong and then query them to the human experts.
DISCUSSION AND CONCLUSION
In this paper, we theoretically reveal the inherent concept/label trade-off for CBM with insufficient concept information and develop DCBM to expand the application range for CBM. Furthermore, we propose a phase-two decoupling algorithm via MI-based optimization to implement numan-machin interaction tasks, including forward intervention and backward rectification. This paper also has some limitations: (1) For the theory part, we assume & < min d2). which is not consistent with datasets which have abundant concepts; (2) We do not consider the leakage phenomenon of CBMs in our methodology. Apart from addressing the limitations, we also propose other future works follows: (1) Extend the proposed DCBM to broad scenarios, such graph-structure data and bioinformatics databases: (2) Utilize the concepts information with causal relationship; (3) Constrain the MI between explicit/implicit concepts during the training process.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
The source code to reproduce every experiment will be available to the public, See Appendix C for the proof of the main theory and Appendix for experimental details.
REFERENCES
Abubakar Abid, Mert Yuksekgonul, and James Zou. Meaningfully explaining model mistakes using conceptual counterfactuals. arXiv preprint arXiv:2106. 12723, 2021.
David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. Advances in Neural Information Processing Systems, 31, 2018.
Mohammad Taha Bahadori and David Heckerman. Debiasing concept-based explanations with causal analysis. arXiv preprint arXiv:2007. 110000 2020.
Andrew Bai, Chan Yeh, Pradeep Ravikumar, Neil YC Lin, and Cho-Jui Hsich. Concept gradi- ent: Concept-basse interpretation without linear assumption. arXiv preprint arXiv:2208. 14966, 2022.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition, pp. 6541-6549, 2017.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In Proceedings of the 35th International Conference Machine Learning, 2018.
Irene Y. Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzych Ghas- semi. Ethical machine learning in healthcare. Anmual Review of Biomedical Data Science, 4(1): 123-144, 2021.
Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, ((4):911-919) 1995.
Yin Cui, Yang Song, Chen Sun, Andrew Howard, and Serge Belongie. Large scale fine-grained categorization and domain-specific transfer learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4109-4118, 2018.
Kenneth Davidson and Stanislaw Szarek. Local operator theory, random matrices and banach spaces. Handbook of the geometry of Banach spaces, 1(317-366)131,, 2001.
Yonina C Eldar and Gitta Kutyniok. Compressed sensing: theory and applications. Cambridge
University Press, 2012.
Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart. Causalm: Causal model explanation through counterfactual language models. Computational Linguistics, 47(2):333-386, 2021.
Ruth Fong and Andrea Vedaldi. Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. $730-8738, 2018.
Yash Goyal, Amir Feder, Uri Shalit, and Been Kim. Explaining classifiers with causal concept effect (cace). arXiv preprint arXiv: /907.077633 2019.
Marton Havasi, Sonali Parbhoo, and Finale Doshi-Velez. Addressing leakage in concept bottleneck models. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Ad- vances in Neural Information Processing Systems, 2022. URL httttttts////peeeeeeee
n
forum? Ld=tg IniD_ __9.
Nicholas J Higham. Accuracy and stability of numerical algorithms. SIAM, 2002.
10

---

Under review as a conference paper at ICLR 2023
Jeremy Kawahara, Sara Daneshvar, Giuseppe Argenziano, and Ghassan Hamameh. Seven- checklist and skin lesion classification using multitask multimodal neural nets. IEEE Journal of Biomedical and Health Informatics, 23(2):538-544 2018.
Dmitry Kazhdan, Botty Dimanov, Mateja Jamnik, Pietro Liö, and Adrian Weller. Now you see me (eme): Concept-based model extraction. In CIKM (Workshops), 2020.
Dmitry Kazhdan, Botty Dimanov, Helena Andres Terre, Mateja Jamnik, Pietro Liò, and Adrian Weller. Is disentanglement all you need? comparing concept-based & disentanglement ap- proaches. arXiv preprint arXiv:2104.06917 2021.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International Conference on Machine Learning, pp. 2668-2677. PMLR, 2018.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. ICLR (Poster), 2015.
Harald Kittler, Ashfaq Marghoob, Giuseppe Argenziano, Cristina Carrera, Clara Curiel- Lewandrowski, Rainer Soofmann-Welleheee Josep Malvehy, Scott Menzies, Susana Puig, Harold Rabinovitz, et al. Standardization of terminology in dermocopyderratosooyy: Results of the third consensus conference of the international society of dermoscopy. Journal of the American Academy of Dermatology. 74(6):1093-1106 2016.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference Machine Learning, pp- 5338-5344 5348. PMLR, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yikuan Li, Mohammad Mamouei, Shishir Rao, Abdelaali Hassaine, Dexter Canoy, Thomas Lukasiewicz, Kazem Rahimi, and Gholamreza Salimi-Khorsin Clinical outcome prediction under hypothetical interventions-a representation learning framework for counterfactual reason- ing. arXiv preprint arXiv:2205.072.55 2022.
Max Losch, Mario Fritz, and Bernt Schiele. Semantic bottlenecks: Quantifying and improving inspectability of deep representations. International Journal of Computer Vision, 129 11333333 3153, 2021.
Yulong Lu and Jianfeng Lu. universal approximation theorem of deep neural networks for express- ing probability distributions. Advances in Neural Information Processing $ystems,33:30944 33:3094-3105, 2020.
Adriano Lucieri, Muhammad Naseer Bajwa, Stephan Alexander Braun, Muhammad Imran Malik, Andreas Dengel, and Sheraz Ahmed. On interpretability of deep learning based skin lesion classi- fiers using concept activation vectors. 2020 International Joint Conference on Neural Networks (IJCNN) pp. 1-10. IEEE, 2020.
Anita Mahinpei, Justin Clark, Isaac Lage, Finale Delhi-Velez, and Weiwei Pan. Promises and pitfalls of black-box concept learning models. arXiv preprint arXiv:2106.133: 2021.
Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. Do concept bottleneck models learn intended? arXiv preprint urXiv:2105.04289, 2021.
M Nevitt, D Felson, and Gayle Lester. The osteoarthritis initiative. Protocol for the cohort study, 1, 2006.
Jayneel Parekh, Pavlo Mozharovskyi, and Florence A'ccéé Buc. A framework to learn with inter- pretation. In Advances in Neural Information Processing Systems, 2021.
11

---

Under review as a conference paper at ICLR 2023
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, sigh-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
Yao Rong, Nora Castner, Efe Bozkir, and Enkelejda Kasneci. User trust on an explainable ai-based medical diagnosis support system. arXiv preprint arXiv:2204.12330, 2022.
Veronica Rotemberg, Nicholas Kurtansky, Brigid Betz-Stablein, Liam Caffery, Emmanouil
Chousakos, Noel Codella, Marc Combalia, Stephen Dusza, Pascale Guitera, David Gutman, et al. A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientific Data, 8(1):1-8, 2021.
Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, and Vineeth N Balasubramanian. A frame- work for learning ante-hoc explainable models via concepts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10286-10295, 2022.
Yoshihide Sawada and Keigo Nakamura. Concept bottleneck model with additional unsupervised concepts. JEEE Access, 10:41758 41765, 2022.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2016.
Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data, 5(1):1-9, 2018.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-ucsd birds 200. 2010.
Maleakhi A Wijaya, Dmitry Kazhdan, Botty Dimanov, and Mateja Jamnik. Failing conceptually: Concept-based explanations of dataset shift. arXiv preprint arXiv:2104 08952, 2021.
Han Xuanyuan, Pietro Barbiero, Dobrik Georgiev, Lucie Charlotte Magister, and Pietro Lió. Global concept-based interpretability for graph neural networks via neuron analysis. arXiv preprint arXiv:2208. 10609, 2022.
Chih-Kual Ych, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-aware concept-based explanations in deep neural networks. Advances in Neural Information Processing Systems, 002220554-205666 2020.
Mert Yuksekgonul, Maggic Wang, and James Zou. Post-hoc concept bottleneck models. In ICLR 2022 Workshop on PAIR 2Struct: Privacy, Accountability, Interpretability, Robustness, Reason- ing on Structured Data, 2022.
Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 119-134, 2018
12

---

Under review as a conference paper at ICLR 2023
A RELATED WORKS
Concepts Bottleneck Models. CBMs (Koh et al., 2020; Kazhdan et al., 2020; Losch et al., 2021) extract the high-level expert knowledge from data and utilize them for downstream tasks, which aim to make the AI system more interpretable and interactive. On the one hand, post-hoc CBMs (Kazh- dan et al., 2020; Yuksekgonul et al., 2022; Bai et al., 2022) generate the concept explanations from the pre-trained model, which has no access to concepts during training process. However, such methods cannot guarantee that the information of concepts can be recovered from the hidden layers of the pre-trained networks (Kazhdan et al., 2021). On the other hand, ante-hoc CBMs (Koh et al., 2020) learn the given concepts during training process to obtain informative embedding. However, several works (Kazhdan et al., 2021; Yuksekgonul et al.. 2022) point out that ante-hoc CBMs suffer the catastrophic label distortion with insufficient high-level concepts. In this paper, we focus on ante-hoc CBMs and look more closely at the situations with insufficient concepts. Instead of point- ing out the label distortions empirically (Kawahara al., 2018; Yuksekgonul et al., 2022), we reveal the inherent concept/label trade-off for CBM with insufficient concepts theoretically. Furthermore, we propose DCBM to alleviate this trade-off via decoupling the heterogeneous information.
Some works also target on concept learning tasks with insufficient explicit concepts and utilize implicit information to improve the label accuracy (Li et al., 2022; Sawada Nakamura, 2022). However, these methods do not constrain the power of implicit knowledge, which makes it impos- sible to give a faithful interpretation. Furthermore, several works (Bahadori & Heckerman, 2020; Goyal et al., 2019; Feder et al., 2021) also consider the causal effect of CBM during the intervention phase or model the causal relationship between concepts. However, in this paper, we do not consider the causal effect and regard it an important future work.
Three Training Techniques for CBM. CBM (Koh et al., 2020) devises three constructions to learn the functions y and S, i.e., Independent, Sequential, and Joinl models, which are introduced as follows:
1. Independent CBM learns and g independently: 9 arg min, E Cclg( c.l. f = arg miny En= [f(f(),), yn.. Note that f is trained with the ground-truth -trth concept e, while using ((xn) as input at test time.
2. Sequential CBM learns in the same way as Independent CMB, but uses ((x,) to learn f. i.e., f= arg min, MN [y ((x,), yn].
3. Joint CBM learns g and f jointly via hyperparameter control the weight of two parts, namely g. f arg ming.s E (Cy o g(an), ynl + a&c (Elc)l(())))))))
Leakage in CBMs. Some papers (Mahinpei et al., 2021; Margeloiu et al., 2021) indicate the infor- mation leakage phenomenon in CBMs, which may impact the interpretability of the learned model. Moreover, Havasi et al. (2022) address this issue in CBMs and separate the concept information via a side-channel model improve the leakage. Unlike the leakage of concepts caused via small pertur- bations, our paper focus on that CBMs fail to guarantee the concept/label accuracy simultaneously with insufficient explicit information.
Other Concept Based Models. Several methods also aim to generate high-level synthetic con- cepts automatically during the training process (Bau et al., 2017; Kim et al., 2018; Zhou et al., 2018; Yeh et al., 2020; Parekh et al., 2021; Sarkar et al., 2022). These methods do not require explicit domain knowledge and can give explainable patterns which tions. However, such methods are not interactive due to the synthetic concepts. In this paper, we utilize expert knowledge of the concepts directly and aim to design an interactive AI system.
responsible for the model's predic-
13

---

Under review as a conference paper at ICLR 2023
B DETAILS OF PHASE-TWO DECOUPLING ALGORITHM FOR DCBM
Algorithm 1: Phase-two Decoupling Algorithm
# MI-based min-max optimization for training fDEC;
Input: Explicit (implicit) concepts {2n) ) ((en) w==) learned by DCBM, decoupling
network /DEC and auxiliary network SMI with initialized parameters IEEC and 0 for E1 epochs do
Sample N pairs from the joint distribution {êni fDEO(C,))
for E2 epochs do
Shuffle N residuals: {ej. - (DDE((),))l=
Calculate the lower bound: [(0Ml) =
Ifnrlff Cn (D, Update 8M1: 0uι optim. Adam (0x1, -V
>n exp (fur(Cni fDEC(C),)))
Shuffle N residuals: fčr. fDEC C+., )
Calculate the loss for decoupling network: [(ODEc) = 1 Ln=llfanennd N (Dpe((,)))) logl Σn exp {fMI (Cni Ck. - fDEC (cx.))) + ()]fDEc(@)13:: Update deEc: ADEC optim. SGD(OMI. (((DEE)));
# Forward Intervention:
Input: Explicit (implicit) concepts è (2) learned by DCBM, the ground-truth concepts e* given by expert, and the pre-trained decoupling network SDEC.
Calculate the residual: r = fDE
Update the new predicted label: Vnew = f(c") + f(fDEC(C") + +r);
# Backward Rectification;
Input:
Explicit (implicit) concepts è (2) learned by DCBM, the ground-truth label y" given by experts, and the pre-trained decoupling network fDEC-
Initialize Cnew via é: Cnew è;
for M epochs do Calculate the classification loss:
Cy(Cnew) = Lx [[neww ) + f(é + SDEC (Cne((nnw)) - (CD Update the concept Cnew:
Cuew optim. Adam (CaewCC V FLy (Cruww )):
In this section, we introduce the technical detail for the design of a human-machine interactive system in DCBM. To begin with, we decouple the information of explicit concept values è E Rd from the implicit ones Rd. Specifically, we assume the implicit concept values can divided into the sum of the following two parts: (1) simulated explicit concepts, i.c., the information of explicit concepts, which can be represented as the output of a decouple function fDEC Rd - R i.e., (é) Rd: (2) the residual, i.e., è (é) which has no or limited information of explicit concepts. To achieve the above factorization, we minimize the MI between the explicit concepts and the residual term é fDEC(C), i.e., consider the following optimization problem:
min MI(ê: č fDEE(C)) + (él fDEC
where SDEC is a light neural network and is a hyperparameter to control the energy of the decou- pling parts. Note that once the optimization problem conducted, most information of explicit concepts hidden in implicit ones will be absorbed into (DEC(C), and the residual will have limited information of č. However, the above optimization problem cannot be solved directly due to the challenge of MI estimation in high dimensions. To combat this, we borrow the tool from MI neural estimation (Belghazi al., 2018), and exploit the lower bound of MI to convert the calculation of MI to the following optimization problem:
MT(è; (()) 2 max T.. [MMI (é; (DEC())))) - log Elexp (Sxn(é; ē - [DEC(C))) (5)
fsu
where fMI Rd+d + R is an auxiliary network for the MI estimation. Thus, the optimization problem can be converted to the following min-max optimization:
min max E [far (è; è fDEC(C)) - log Elexp (fxa(é; é - fDEc(e))]] +llDDEc( (6)
JDEC Jsn
14

---

Under review as a conference paper at ICLR 2023
Equipped with the pre-trained decoupling neural network fDEC, it convenient conduct the above two interaction tasks. For forward intervention, we only need to replace the predicted concepts by expert concepts c² in the explicit concepts layer and the input of decoupling network fDEC, respectively. In detail, we update the new predicted label ynew as follows, which is in line with the green line in Fig. 4: ynew f(e") + ((DDE((e")) 2 (DEcc)))
For backward rectification, the main idea is to adjust the explicit concept values to make the model's decision agree with the expert label y". In detail, we fine-tune the concept values via backpropagation, i.e., solve the following light optimization problem and update the concept Cnew which is in line with the red line in Fig. 4:
Cnew arg min Ly [(Cnow ) + f(ē + /DRC(()), y²) Crews
(8)
C THEORETICAL RESULTS
This section presents the inherent dilemma of concept/label distortions in CBM theoretically. The analysis of the main theorem consists of two steps: (1) analyzing the relationship between the number of missing concepts and the label error for Sequential CBM: (2) revealing the trade-off between concept and label accuracy for Joint CBM via the perturbation theory of least square prob- lem (Higham, 2002).
C.I ASSUMPTIONS
To model the above phenomenon mathematically, we firstly introduce the functional families Fa and the non-redundant assumption in Definition and 2, respectively.
Definition 1. Define Fa,k as the families of linear transform from TRdd to R.
Fa a -
E kxd W., N(O. )
Definition 2. Let be a probability distribution over Rd. We say C is non-redundant if and only if: Cg) < H(C,), for all A, c f1,2. ... .d] AnB = Ø, where MI(::)) and H(-) denote the mutual information and entropy, respectively.
(10)
We state our main assumption for the data generating process Assumption Assumption 1. [Two-phase Generation] The data are generated from a two-phase processes G:
G: - -+
(11)
where X C. I. is the distribution of the input images, C Rd and y C. I represent the corre- sponding distribution of concepts and labels, respectively. g" : x C is a Lipschitz continuous concept mapping, which maps the data point its corresponding concept, and f is a linear classi- fier sampled from the families of linear transform Fa,x randomly. Furthermore, we assume that the concept distribution C satisfies the non-redundant assumption in Definition 2.
To generate the dataset, we firstly sample a data generator G from the corresponding faresp G. Then, we sample N triples {æn, Cn, Yn )n) from the data generator G, and only select di d concepts randomly from C, denoted as {æ,. C₂, ). to simulate the case of insufficient concepts.
The non mon-ddanndant assumption reveals that arbitrary concept cannot be represented by the rest ones, which guarantees the uniqueness of the linear probe in the proof of the main theorem. Furthermore, we assume the concept bottleneck model comprises two modules as follows.
Assumption 2. Let and be two modules in CBM, where X - C be powerful neural network function which can represent arbitrary Lipschitz continuous mapping and f E Fa is a linear probe.
The universal approximation assumption of neural network function can be guaranteed by the theo- in (Chen & Chen, 1995; Lu & Lu, 2020).
15

---

Under review as a conference paper at ICLR 2023
C.2 MAIN RESULTS
Now we are ready to state our main theorem in this section as follows.
Theorem 2. Consider a datasel {xn. En Un InJn= generated from G described in Assumption 1, and the CBM model which satisfies the Assumption 2. Let d1 (d2) denote the number of known (unknown) concepts, and assume k < min d1, d2}. Let Ec and Ey denote the lp error of concepts and labels for CBM, respectively. Then, for all e O, the following inequation holds for Joint CBM with probability at least 1 2eepp (-c2 /2):
1
Ey 2 Vd (Vd2 Vk (e)e))ll,,CC)llll
(12)
where IP A (B) denotes the projection of onto the subspace span CT E RNdi E Nxd) denotes the matrices of known (unknown) concepts, and U2 denotes the orthogonal complement space of C1 such that span(C1 + C2) (Ca
) @ span(U2).
R Y P RRNk be the concatenation of N input images, concepts, and labels, re- spectively. Without loss of generality, we assume that the first dj concepts are known, and divide C into two parts, the given concepts C1 E Nxd, and the unknown parts C2 E RRdx,,, respectively. Applying block QR factorization on C, we have:
Proof Let X A II. _NI E RNND, C - [C1, [ci,es.....eN]
(y) y2, ... yN
Bu
B12 B22
C
[C | [1]
(13)
0
where U₁ E Nxdx and U2 E IR Xxda denote two orthogonal basis matrices, and the block Bn E Rdixd, B12 E RRd xd2 and B22 Rd2xd2 denote the coefficient matrices. Similarly, we divide the linear transform matrix W E Rexd in Assumption 1 as W [Wi]] [[ where W1 E R R and W2 E 2x represent the linear transform matrices corresponding to Ci and C2, respectively. It is worth mentioning that W is almost surely unique due to the non-redundant and the Gaussian distribution assumptions in Assumption 1.
Firstly, we analyze the relationship between the number of missing concepts and the label error for Sequential CBM. Due to the universal approximation ability of the neural network function g. we only need to consider the following optimization problem:
min fEFáy.k
Σ (lf(ch) - -3
(14)
n=1
According to the law of data generator G, we have:
Bu
B12
W W2
Y= = CWT = U1
(15)
0 B
and Y A U2 W denote the optimal projection and the residual of Y onto the subspace of span(Ci), respectively. Thus, we can give the lower bound for the & label error of Sequential CBM L3 as follows:
Then, Y' & U W +1,,,
E | 12
= 122 W. 12
(16)
2 2 (W2) IP,,, ([lPu,(C)) (C2)
where Tmin (W2) denotes the least singular value of Ws. Due to W₂ E & is the sub-matrix of W, which sampled from a random matrix family satisfying the Gaussian distribution. According to the random matrix theory (Davidson & Szarek, 2001; Eldar & Kutyniok, 2012), for all 2 with probability at least 1 - exx (-2 we have:
Tmin (2) 2 (V dh Vk- e).
(17)
Vd
16

---

Under review as a conference paper at ICLR 2023
Thus, with high probability, we have:
Cy 1 d (V vk - -
(18)
Next, we consider the equivalent form of the Joint CBM, which can be formulated as the following optimization problem:
min (En of
Cy + aLc,
(19)
Cc
Σ
cn, c..l:: Lr
Σ [ls((n) yn
The equivalence can be guaranteed due to the universal approximation property of the neural net- work function. Let { (e))=1 and f be the optimizer of the optimization problem 19, and concate- nate all {ct ) as Ć Furthermore, let Y denote the projection residual of Y onto the subspace span(C*). Then, according to the perturbation theory of the least square problem (Higham, 2002), we have:
YY l
$I0 Ci Cilll + + - - Y
(20)
<2|l -
Then, we can obtain the lower bound of label error Ey for Joint CBM as follows:
Ey = Y |F
2 Y || | - -
> 2|/ -2 -|I+||C1|||||
(21)
Vd (Vde Vi (E))))Fu, 2| - |1
1
(V - v
C.3 DISCUSSION OF THE MAIN THEORY
Note that we assume that the weight matrix W2 E 2d xk is generated from Gaussian distribution in Assumption which is used when we bound the least singular value of the weight matrix. Note that the assumption can be relaxed as rank(W2) = & and we make such an assumption to obtain a more accurate description. Actually, for any matrix W2, when we add row II* to it, where can be regarded as the weight of new concept, we will have:
W.
1
W₂ w
W. W2 + w W. W2.
(22)
W₂ 1): which indicates that the decrease of concept w
Thus, obtain miin (W2) < Jmin
number will increase the least singular value of the weight matrix, and hence increasing the label error according to Eq. 16.
Moreover, we assume < min(d, d, d2 which not consistent with the CUB dataset. This is one of the limitation of our theoretical result, and we regard this problem as important future work.
17

---

Under review as a conference paper at ICLR 2023
Table 4: Brief summary of the datasets. Here, means 'the number of". Note that the splitting of sett-teet-valiationn is in line with (Koh et al., 2020; Kawahara et al., 2018).
DATASET
#CONCEPT #CLASS #TRAIN #VAL
#TEST
CUB (Wah et al., 2011)
112
CUB%50
56
200
4796
1198
5794
CUB%20
22
CUB% 10
11
Derm7pt (Kawahara et al., 2018)
8
688
322
636
D EXPERIMENTAL PROTOCOLS
D.1 DATASET
We consider two commonly-un datasets, including bird species and dermoscopic images, as shown in Fig. 7. Both of them express high-level concepts such can provide interpretability analysis. A brief' summary of datasets can be seen in Table 4.
CUB (Wah al., 201 Caltech-UCSD Birds-200-2011, which we call CUB in the following text, is an extension of CUB-200 (Welinder et al., 2010), collecting total of 200 bird species with 11,788 bird photographs. CUB comprises 312 bird attributes including wing shape, back pattern, eye color, etc. Since the concepts are noisy and sparse, we leave only 112 binary concepts instead, following the work (Koh et al., 2020), detailed in Table This task aims classify each bird photograph out of 200 different species. Thanks to the splitting standard of train/test/validatos dataset (Koh et al., 2020), we only need to follow this given setting.
Derm7pt (Kawahara et al., 2018) Seven-Point Checklist Dermatology dataset, abbreviated as Derm7pt, dataset comprising 1,011 groups of images. Each group contains a clinic photograph and dermoscopic image, respectively. Due to the lack of Seborrheic Keratosis (SK) samples, we follow the setting of the work (Lucieri et al., 2020), only considering Mclanoma (MEL) and Naevi (NV) ones with total of 823 groups. Derm7pt contains several authoritative concepts (Kittler al., 2016) including Pigmented Networks, Streaks, etc. On account of the concept absence and inter- pretability Melanoma detection, leave Blue Whitish Veil, Pigmented Networks, and Regression Structures, as (Yuksekgonul et al., 2022) did. We also show the concepts in Table 5. This task is a binary classification problem, judging whether an image is a melanoma or naevus. Similarly, the train/test/validation indexes are already given well.
aellignancy-asocciate
binary concepts, including
Note that CUB succeeds to have a large group of concepts, while Derm7pt fails. However, most datasets we have investigated typically have small number of concepts (Nevitt et al., 2006), or have to be extracted (Tschandl et al., 2018; Rotemberg et al., 2021; Krizhevsky et al., 2009; Fong & Vedaldi, 2018) by other trained models (Lucieri al., 2020; Abid al., 2021). This phenomenon is understandable, since high-quality and abundant labeled concepts are labour-intensive and infre- quent. This brings about new question- - What if CUB also has an insufficient group of concepts? And then, how will CBM perform under this condition? To achieve the answer, randomly drop 50%, 80%, and 90% concepts in CUB, respectively, and form three new datasets named CUB 55, CUB % 20, and CUB % 10. We display bricf summary of the referred datasets in Table 4, while the left concepts are randomly chosen by programs and fixed as:
CUB50%: 2, 5, 6, 8, 10, 12, 13, 18, 19, 21, 27, 28, 32, 33, 34, 37, 39, 40, 41, 43, 46, 47, 50, 52, 54, 56, 57, 61, 62, 63, 64, 65, 66, 67, 69, 71, 72, 75, 78, 79, 80, 81, 84, 86, 92, 93, 95, 96, 98, 100, 101, 103, 104, 107, 109.
CUB20%: 6, 13, 18, 28, 33, 34, 37, 39, 46, 50, 52, 54, 62, 63, 65, 66, 75, 80, 96, 98, 101, 109.
CUB10%: 6, 34, 50, 52, 54, 63, 66, 98, 101, 107, 109.
D.2 BASELINES
We adopt seven models as our baselines in this paper, including:
Independent/ Sequential Joint models in CBM, recommended in Sec. 2.1.
18

---

Under review as a conference paper at ICLR 2023
Table 5: Concepts used in CUB and Perm7pt.
Concepis in CUB
Concept
No.
Name
Coucept Name
shage: dagzer
color: black
Jocehoad color. yellow
back patterz stiped
shape: hocked seshird
color: white
forehead color: black
back puttert: colored
shape: all-paposs
31
color:
focthcad color. white
pilter: solid
hill shope: cone
32
tall shupe: nosched
under tall color: brown
pattern: striped
wing color hrown
33
upper biil color: browe
61
under tell color: gray
paltern: mulli-enioeed
wing color gray
24
upper tall coloe: gray
D2
under tall color: Black
90
belly panern soli
wing calor yallow
35
upper tail color: black
63
under tail color white
91
peimary color. boown
wing color: black
56
upper till color: white
64
under tall color bulf
92
primary oolor grey
wing color white
37
upper tail colort bulf
65
nape ooler: brown
93
primary odler yellow
10
wing color: bulf
38
head punem: evebrow
66
nage color: grey
94
peimary color. black
11
upperputs color. beown
lscal pulten plann
67
woler yellow
jeimury color while
upperponts color
60
breast color brown
68
coler black
96
peimary color buff
upperpott color yellow
41
brcast color
69
ooler while
97
coker
upperputt black
42
breust color. yellow
color. butf
color: black
upperpats
lucast black
Irown
upperpants bolf
breast becast bulf
pray
grey
anderrponts brown
underparts color. grey
throat gryy
belly color: black
butt
underponts color: yellow underputs color. underpats color. white
thrcest color: yellow
belly color white
crewn Muee
throst
belly color bulf
crown beaen
21
89
throut color: white
73
wing shape: wounded-wings
1
crown color. grey
22
underpatt color: bulf
50
throut color: bull
78
wing shape: pointed wings
106
crown color. yellow
23
becset pertem: solid
51
eye color: black
70
stre: small (5 9 in)
107
crown color black
24
becsed pattenn striped
52
bill length: about the same as head
80
stze: medium (9 is)
108
crewn color. white
25
berast porsem: mulli-coloeed
53
bill length: shorter than bead
81
sãze: very small (3- Sim
109
wing poltem: selid
20
back coloe, brows
54
Jorchccll color: Dlue
R2
shage: dics like
110
wing puseeni spoited
27
back color
55
forchoal color: brown
83
shapc: perching-like
I1]
wing petten: stipel
28
back color yellow
56
fondicad color: grey
84
back pulten: solid
112
wing malli-colaral
Concepts
Dermippt
Vo.
Concept Name bice whittsh well: present Pigmeent network: absent
No. 3
Concogt Name
No.
Concept Name
No.
Concept Name
Pigment nelworks aispical
regression sinuctures absent
regression structures: combinations regressias structares: white aress
pigment network: typical
repression structures: blue areas
CUB
Derm7pt
Groove BilledAnn
Melanoma (Clinle)
Brewer Blackbird
Melanoma (Dermoscopic)
Gadwall all
Naevus (Clinic)
Conte Sparrow
Naevus (Dermoscopic)
Examples of CUB (left) and Derm7pt (right) datasets. In CUB, each line indicates a bird species, while in Dprm7pt, each line corresponds to the clinic or dermoscopic images of melanoma or nevus. The clinic and dermoscopic images are aligned up and down.
Figure 7:
Standard model with / without bottleneck, introduced in (Koh et al., 2020). The former learns f. with no concept supervised: f. = arg ming, f.g En fy l o ( yn]. while the latter is devised directly predict æ + y: h arg min, Σn (hx)) yo]. with mapping h RR + IR* Multitask, also introduced in (Koh et al., 2020), is devised with an additional auxiliary loss to enable the last layer to be predictive of the concepts, which learns g, l simultaneously: 9, h arg ming.h I [M(2),), ym] + aLc (L
CME (Kazhdan et al., 2020), also named as Concept-based Model Extraction, based on pre- trained deep neural network and searches mapping from deep layer to each concept that performs the best on concept accuracy, and then predicts the final label via traditional machine learning ap- proaches.
CBM-AUC (Sawada & Nakamura, 2022) integrates supervised concepts with unsupervised ones trained with self-explaining neural networks (Alvarez Melis & Jaakkola, 2018) in order to alleviate the restricted dimension of the concept layer.
Note that it is impossible for the Standard and Multitask model to be interpreted with the concepts ignored or uncovered by the final prediction, however, since there is no trade-off between concept and label, their performance on label prediction competitive and worth enough attention.
Each experiment is repeated three times with mean and standard deviation reported in line with (Koh et al., 2020), and is preformed on machine with AMD EPYC 7742 64-Core Processor CPU,
19

---

Under review as a conference paper at ICLR 2023
NVIDIA A100-10000AA Pytorch (Paszke et al., 2019). The source code will be made publicly available for reproduction requirements.
-000-XM4-80GBBB GPU, and IT RAM. All experiments in this paper are implemented by
D.3 METRICS
Top-1 accuracy is exploited in both label and concept tasks. The former is a multi-class learning task, while the latter describes multi-label classification task. In particular, in label prediction task, we use cross entropy loss to describe the difference between the predicted label and the ground-truth one. In concept prediction task, each concept corresponds a binary classification mission and thus we compute the mean accuracy for all labels, where we also use a cross entropy loss to compute the difference of each predicted concept and the ground truth one during training process. This setting is in line with (Koh et al., 2020).
D.4 MODEL SETTING
For each experiment, DCBM fine-tunes a pretrained Inception-v3 network (Szegedy et al., 2016), which maps to explicit and implicit concept values, respectively, and then connects to linear layers to obtain outputs of explicit/implici concepts. In particular, explicit concept values are exploited to predict
c, while the sum of two outputs is used to predict the label. Apart from some general hyperparameters, e.g., learning rate, we mainly search concept weight a and JS Divergence weight 8, to achieve high performance and maintain the interpretability. Detailed setting is shown in Appendix D.4. In short, we choose a = 0.01 and a = 0.1, which respectively perform better on concept and label learning tasks. Analysis about the choice of / is further discussed in Sec. 4.4.
For the pre- rre-processing of the training images, we adjust the size of each image in CUB and Derm7pt to 299 x 299x x via random jittering, horizontal flip, and cropping, which is the same setting in (Koh et al., 2020; Cui et al., 2018), while the images at test time are center-cropped and resized to 299. Similarly, but with a few differences, in DCBM training, we increase the parameter search field. We search hyperparameters on the validation dataset with a in [0.005, 0.01, 0.05, 0.1, 0.5, 1.0], learning rate in [0.001, 0.01]. regularization strengths in [0.0004, 0.00004], reducing learning rates by 10 times after every [10, 15, 20, 30] epochs until the rates come to 0.0001. Once these parameters are fixed, we retrained models with training and validation datasets altogether until convergence with / in [0.0, 0.01, 0.1, 1.0, 10.0]. In Table 1, we display the results of DCBM-0.1 and DCBM-0.01 using = 1.0 on almost all datasets. The mere exception DCBM-0.01 on CUB, where we use 3 = 0.1 because concepts are sufficient, and we don't 't need large 3 to restrict JS Divergence. Such settings can well balance the model's accuracy and interpretabilityy as detailed in Fig. 6. Besides, the dimen- sion of implicit concept values is set to 256 in all cases, we demonstrate in Appendix E. that the dimension of implicit concepts affects little on concept/label accuracy. The best model is chosen by either the best label accuracy or the best concept accuracy on validation dataset. Each experiment is trained with batch size of 64 and Stochastic Gradient Descent (SGD) with momentum of 0.9.
fMI in Sec. 3.2 is a multilayer perception (MLP), which has neuron sizes of [input size, 50, 50, output size], where the inpul size equals to the dimension sum of explicit and implicit concept values, and the output size set as compute MI. In addition, figc in Sec. 3.2 is also an MLP layer with neuron sizes of [input size, 100, 100, 100, output size), where the input size and output size respectively equal the dimensions of explicit and implicit concept values. Among each layer in I'M and /DEC, we add activation functions ReI.U(x:) = max (0,z) to achieve nonlinearity.
For forward intervention, to obtain rather high performance, MI estimation optimization trained by Adam (Kingma & Ba, 2015) with a learning rate in [0.005, 0.01, 0.05, 0. and the decoupling neural network is trained by SGD with a learning rate in [0.005, 0.01, 0.05, 0.1] I] and momentum of O. where MI estimation is optimized for [20. 30, 40, 50] times corresponding to each decoupling process. We use the cross entropy on training datasets to perform early stopping. Besides, the backward rectification is learned by Adam with learning rate in [0.0025, 0.025J and normalization weight in [o, 0.01, 0.1] until convergence. The best models are chosen via the best performance on validation datasets.
20

---

Under review as a conference paper at ICLR 2023
Table 6: Time overheads (seconds) on the training process of concept/label prediction (DCBM), decoupling neural network (DEC), forward intervention (DCBM-INT-DEC), and backward rectifi- cation (DCBM-REC-DEC) which are recorded over an entire training process.
DATASET
APPROACH
CUB
CUB%50
CUB 20
CUB% 10
DERM7PT
DCBM
33260 = 5212
21920 + 9138
26264 + 6824
14654 + 4991
3436 + 788
DEC
362 = 78
312 + 83
210 + 35
147 + 34
491 + 61
DCBM-INT-DEC
0.01
< 0.01
< 0.01
0.01
< 0.01
OCBM-REC-DEE
0.27 1 0.01
0.34 1 0.02
0.47 1 0.02
0.67 - 0.10
0.39 1 0.13
High standard deviations mainly due the early stopping strategy during training. We show the time overhead of each intervenedrrettffied sample.
(a)
(b)
(c)
Values of Divergence
0.8
100
CUB
Concept Accuracy (%)
CUB
Lahol Accuracy (%)
CUB
0.6
CUB%50
100
CUB%50
CUB
CUB%20
CUB%20
90
CUB$20
0.4
CUB%10
CUB%
CUB%
98
0.2
80
96
0
0.0
0.01
0.1
10.0
32
64
128
256
512
32
64
128 256 512
Weight of IS Divergence
Dimension of Implicit Concepts
Dimension of Implicit Concepts
Figure 8: (a) The relationship between B and the values of JS Divergence on CUB and its three variants. (b-c) The impact that different dimensions of implicit concepts [32, 64, 128, 256, 512] have on concept (b) / label (c) accuracy on CUB and its three variants with 0.1 and B = 1.0.
D.5 TIME OVERIEAD FOR HUMAN-MACIING INTERACTIVE SYSTEM
The interaction system is based on a light min-max optimization introduced in Appendix B, which can achieved within minutes. Equipped with the pretrained decoupling networks, DCBM can respectively perform forward intervention and backward rectification within 0.01s and Is for each sample. Detailed time overheads are shown in Table 6, where we choose a = 0.1 and 3 = 1.0 as a typical example in all tasks. Looking from the table it is apparent that the forward intervention and backward rectification process are much quicker than model training.
E EXPERIMENTAL ANALYSIS
E.
HOW DOES THE WEIGHT OF JS DIVERGENCE AFFECT ITS VALUES?
In the methodology section (Sec. 3), we use 3 to force the prediction from explicit concepts not to be far away from the one from both explicit and implicit concepts. An interesting question is whether 3 does achieve such expectations. To evaluate this, in Fig. (a), we detail the relationship between the weight of JS Divergence and its values. When 3 = O, fewer discarded concepts lead to higher values of JS Divergence; however, with rising from 0.0 to 10.0, the values of JS Divergence cor- respondingly decrease to 0 in all cases, which illustrates that by choosing higher 3. the prediction from explicit concepts can be effectively controlled to be almost the same as the prediction from both explicit and implicit concepts.
E.2 How THE DIMENSION OF IMPLICIT CONCEPTS AFFECTS CONCEPT/LABEL ACCURACY?
Hyperparameters like the dimension of implicit concepts d can intuitively affect the model predic- tion. Under this perspective, in Fig. 8 (b-c), we take a = 0.1 and 3 = 0.01 as an example and depict how much the dimension of implicit concepts affects concept/label accuracy on CUB and its three variants; however, both Fig. (b-c) show that the dimension of implicit concepts has little impact on concept/label accuracy, despite some slight fluctuation reasonably exists. This phenomenon shows the robustness of DCBM with respect to d.
21

---

Under review as a conference paper at ICLR 2023
CUB
CUB%
CUB%20
CUB%10
100
100
100
100
Label Accuracy
80
80
80
80
60
60
40
40
40
20
20
20
0.0000
DD25 0.0050 0.0075 0.000
0.005
0.010
0.00
0.02
0.04
0. 0.06
0.0
0.2
0.4
15 Divergence
Divergence
Js Divergence
J5 Divergence
The relationship between IS Divergence and the label accuracy predicted on by explicit concepts on CUB and its three variants. Each point represents a concrete category.
Figure 9:
E.3 WHETHER JS DIVERGENCE RELATES TO INTERPRETABILITY?
It's worth discussing whether the IS Divergence can evaluate the model interpretability. To address this, in Fig. 9, we display the relationship between JS Divergence and the label accuracy predicted by explicit concepts on CUB and its three variants. Each point in the figure represents concrete bird identification. Looking from an overall perspective is apparent that the JS Divergence generally larger when more concepts are discarded, which indicates that the interpretability is weakened when concepts are insufficient. This phenomenon demonstrates that the JS Divergence is an effective metric to evaluate the model interpretabilityy In short, DCBM cannot give explanations for all data points with insufficient concepts and it is also impossible for other models. However, DCBM can give accurate predictions for data via explicit/implict parts and meanwhile judge whether the model decision for this data point can be interpreted by given concepts via JS Divergence. If the answer is yes, DCBM will give the corresponding interpretation.
E,4 WHY DOES THE LABEL ACCURACY OF IMP-DCBM RISE FIRST AND THEN FALL.?
It's intuitive that with the increase of the weight of JS Divergence 3, explicit concepts have a greater impact on label learning, and thus, implicit ones are supposed to be less informative about labels; however, we fail to see such intuition in Fig. 6 (a-c, e) and find the label accuracy of IMP-DCBM rises first and then falls precipitously. To illustrate this counterintuitive phenomenon, we present some detailed information in Fig. 10.
Actually, by increasing B, the distribution of explicit outputs can be forced to be similar to the sum of explicit and implicit outputs, which leads to two consequences: 1) the implicit outputs become similar to the explicit ones; 2) the implicit outputs tend to O. Under this perspective, we use the JS Divergence of explicit and implicit outputs in Fig. 10 (a) to illustrate the former and the Mean Absolute Value (MAV) of implicit outputs in Fig. 10 (b) to illustrate the latter. Looking first of all at CUB in Fig. 10 (a), the IS Divergence keeps stable from 0.0 to 0.1, so it's reasonable for the label accuracy of IMP-DCBM rises together with EXP-DCBM, but it finally drops because the JS Divergence surges. By comparison, there's little rise in the label accuracy of IMP-DCBM when changes from 0.0 to 0.1 on CUB% 10, though an obvious decline of is Divergence exists on CUB% 10. The reason is that due to the low accuracy of EXP-DCBM, the approximation to explicit outputs cannot result in a better label accuracy for IMP-DCBM. These observations are consistent with that shown in Fig. 6. Moreover, in Fig. 10 (b), the MAV of implicit concepts also converges to 0 as expected.
E.5 HOW MUCH CAN THE DECOUPLING NETWORK REDUCE MI?
In this section, we evaluate the decrease of MI between the explicit/iplicit concepts with the help of decoupling neural network. Followed by the setting in the above experiments, we utilize neural MI estimation (Belghazi et al., 2018) to approximate the MI. In detail, we calculate original MI, i.e. MI 2), and the decoupled one, i.e., MI(2; é fuec (c)) on CUB and its three variants for both 3 = 0.01 and B = 1 (see Fig. 11 (a-b)). Except for CUB% 10, can see significant decline in MI on the other three datasets. One possible reason is that the number of concepts is too small, and thus, the decoupling network has limited room for reducing the MI.
22

---

Under review as a conference paper at ICLR 2023
(a)
(b)
Mean Absolute Value (Implicit)
0.6
CUB CUB 10
CUB CUB 10
JS Divergence
4
0.4
0.2
2
8
0
0
0.0
0.01
0.1
1.0
10.0
0.0
0.01
0.1
1.0
10.0
Weight of JS Divergence
Weight of JS Divergence
Figure 10: (a) JS Divergence of explicit and implicit outputs and (b) Mean Absolute Value (MAV) of implicit outputs CUB / CUB% 10 with from 0.0 to 10.0. Note that the JS Divergence in (a) different from the one in the methodology part.
(A)
(
100
100
14
Label Acsuncy (8)
MI Extination
MI Eetimation
Aconaay
10
10
Origital Decorpted
Onginal Deccopled
DCPM
-0-
SEC
10%
20%
30%
100%
10%
2016
50%
100%
10%
20%
50%
100%
10%
20%
50%
100%
Concergt
Concept
Cancepe
Concept
Figure 11: (a-b) MI Estimation of the original DCBM and the decoupled one using 10%/20%%509/1000 concept when (a) 3 0.01 and (b) 1.0. (c-d) Label accuracy of EXP- DCBM and EXP-DCBM assisted with simulated explicit concepts (abbreviated as 'with SEC') using 10%/20%/50%/100%% concept when (c) 3 0.01 and (d) 3 1.0.
E.6 WHETHER THE DECOUPLING NETWORK HELPS THE PREDICTION TASK?
Another interesting question is whether the decoupling network helps the prediction task for the explicit DCBM. Actually, DCBM can naturally combine the explicit concepts č with the simulated explicit information (D) to make a comprehensive decision. As detailed in Fig. 11, simu- lated explicit information can hardly have impact the label prediction with relatively higher 3 = 1.0 or with sufficient concepts (3 = 1.0 on CUB); however, such information can effectively promote the label prediction when concepts that when 3 is low and concepts are insufficient, the explicit concepts are not informative about class predictions, and thus, simulated explicit concepts have larger room for label accuracy promotion.
insufficient with lower B 0.01. The reason is
23


Reviews:
[{"review_id": "lmw2cO_RAX", "reviewer": "Reviewer_k8KK", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors introduce a new version of the Concept Bottleneck Model (Koh et al., 2020) that dims to be more accurate and interpretable w.r.t. other Concept-based models, especially in regimes where concept supervision is limited.\n The proposed method is inspired by a theorem depicting the trade-off between the concept error and the label error in the Joint Concept-Bottleneck model (CBM), due to missing concept information.  Hence, the authors propose a new version (DCBM) of the original model to decouple the information between explicit and implicit concepts. They conduct an extensive experimental comparison with other known CBMs and a standard black-box model, achieving overall better label/concept accuracy performances. \nAlong with a novel strategy to train the DCBM, they also propose a new two-stage human-machine interaction, addressing both the correction of the wrong concepts and of the misclassified labels. This proves effective and improves previous methods, without requiring costly time computations.\n\n", "summary_of_the_review": "This paper presents a novel theoretical result (Theorem 1 for CBMs), which is relevant for future developments of interpretable concept-based models. Theorem 1 indicates an inherent trade-off between label and concept accuracy, up to some assumptions. However, it is not evident why the proposed model should address the limitations of CBMs, in light of the quantities introduced in the theorem. A decoupling phase is introduced in the interactive phase but is not present when giving the prediction, referring to Fig. 3. \nIn the experimental investigation, the proposed model (DCBM) achieves better downstream and interventional label/concept accuracy than CBM, especially in the regime of reduced-concept supervision. \nStill, the claim of being interpretable is not fully supported by the experiments, whereas they indicate the implicit concepts (which are not interpretable) are highly predictive of the label in all regimes tested (exception made for $\\beta=10$ in all datasets, Fig. 7).\nSince the proposed model makes use of implicit concepts for the prediction, their relative importance has to be quantified somehow. Guaranteeing interpretability is necessary for DCBM, and for concept-based models in general, but it was just assumed by the authors. There is still the possibility that implicit concepts are not so relevant for the prediction, but it has to be shown. I am open to revising my score if this point is clarified. \n", "strength_and_weaknesses": "This paper addresses the important problem of the accuracy/interpretability trade-off of Concept-based models. In this respect, the authors prove a theorem, based on known results in Random Matrix Theory, which reveals an inherent trade-off between the concept error and the label error.  While the assumptions are a bit restrictive (the weights are assumed to be sampled independently from a gaussian distribution, which excludes real-world scenarios where correlations among weights are important) the lower bound relates the expected error on the label to that on the concepts, which is a first, relevant theoretical result for Concept-based models. Another limitation is inherited by the dimensionality of the classification space, which is assumed to be $k < \\min(d_1, d_2)$, where $d_1$ and $d_2$ refer to the dimension of the explicit and implicit concepts, respectively. It seems that the CUB dataset violates the assumptions of the Theorem, being not applicable in that scenario (in fact, in the CUB-100% dataset d_1=112, where k=200). \n\nFollowing, motivated by the theorem, the authors suggest that decoupling the explicit concepts and implicit concepts should alleviate the overall label/concept distortion. Upon carefully analyzing the result of the theorem, it is still not clear to me how this claim is effectively supported. In fact, the quantities in Eq. (10) are neither evaluated nor accounted for in the formulation of the model.\nAs it stands, the theorem yields an interesting result but it seems not so related to the model proposed by the authors nor to the experimental evaluation. \n\nThe experimental evaluation of the label/concept accuracy of the model supports the major performances of the model w.r.t. other Concept-based proposals. \nFor what concerns the interpretability of the model, a key requirement of CBM is the prediction to depend mostly on the interpretable (in this case explicit) concepts. \nAs stated by the authors, an important parameter of the DCBM is the weight $\\beta$ of the Jensen-Shannon (JS) divergence, which regulates the influence of the implicit concepts for the prediction. To retain interpretability, the prediction should depend mostly on explicit concepts and, in principle, high values of $\\beta$ should enforce this property. This holds as long as the problem of predicting $Y$ given $\\hat C$ is linearly separable. Does this condition hold with the selected concepts in the CUB experiments? If not, the explicit concepts themselves would not suffice to predict correctly the label; hence optimizing the JS divergence would naturally deteriorate the model accuracy. When unsupervised concepts are relevant for prediction, they should be somehow interpretable, otherwise, the interpretability of the model falls off. For example, in Concept Bottleneck Model With Additional Unsupervised Concepts (in short CBM-AUC) by Sawada and Nakamura, IEEE (2022), additional unsupervised concepts are extracted by a self-explainable neural network (Alvarez-Melis and Jaakkola, NeurIPS (2018)), which guarantees in principle their interpretability. Since CBM-AUC is a natural competitor of DCBM, it should be clear why it is not considered in the empirical evaluation\n\nIn the experiments with CUB-20% and CUB-10%, it seems that any $\\beta \\leq 1 $ favors the intrinsic concepts to be highly informative of the label, refer to Fig. 7. Then, how can the interpretability of the predictions be guaranteed? For the results in Table 2, which $\\beta$ do they refer to? This is not clear from the text, as in Sec. 5.2 the reader is referred to Appendix D, where there is only the study in relative performances upon changing $\\beta$. In Appendix C.4 it is reported that: “the best model is chosen by either the best label accuracy or the best concept accuracy on validation set ”. Are those DCBMs with $\\beta=0.1$ and $\\beta=1$? Conversely, in the intervention experiment, $\\beta=0.01$ and $\\beta=0.1$ were used (I think there is a typo in the caption of Table 4, where there is $\\beta=1.0$ but in the main text, it is said $\\beta=0.1$). In this case, the implicit concepts are highly informative of the label, as clear from Figure 7.\n\nOverall, it must be evaluated the relative importance of $\\tilde f \\circ \\tilde g$ on the prediction w.r.t. $f \\circ g$ to assess how the implicit concepts affect the downstream task. Without such a metric, interpretability cannot be guaranteed.\n\nThe proposal of the two-phase human-machine interaction is novel and interesting: it works effectively w.r.t. other interactive strategies, winning in all scenarios. Also, as a merit, it does not increase sensibly the computational cost. Here, a map $f_{DEC}$ is estimated to decouple the residual concepts from the explicit ones in the implicit layer. While I understood it is only used in the interactive section (correct me if I am wrong), it would be useful to adopt it also in the label prediction, since $\\tilde c - f_{DEC} (\\hat c)$ does not contain redundant information of the explicit concepts $\\hat c$. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bG6rH4zRDK", "reviewer": "Reviewer_FEGm", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, authors analyze the current trade-off between label and concept accuracy in concept bottleneck models when the number of concepts reduces. They prove this trade-off to be inherent to standard CBM  models both theoretically and empirically. Based on their observation, they propose a new approach named DCBM, which allows the model to learn additional \"implicit\" concepts to improve label classification performance when concepts are missing while ensuring good concept classification. To avoid models to bypass explicit concepts in the decision-making of their downstream task, the authors also ensure low JS divergence between explicit concept predictions and label predictions. They show DCBM superiority in a low concept regime over previous CBM models on two benchmark datasets. They also show that their model performs well under forward intervention and their newly introduced backward rectification task. ", "summary_of_the_review": "Overall the authors propose a simple and efficient method for a highly relevant problem in the field. Their work is clearly structured and written. Their experiments show strong results and are well-motivated. However, crucial details surrounding these experiments and discussions are missing to back the authors' claims, in particular around the JSD weighting parameter $\\beta$ and its impact. ", "strength_and_weaknesses": "Strengths: \n- The problem of performance decrease with the number of concepts is highly relevant to the field of interpretability given CBM's popularity.\n- The authors have valuable experiments and theoretical findings supporting the existence of limitations for CBM in low concept regime\n- The method and metrics are well explained making the paper easy to follow.\n- Contrary to other works, the authors consider the possible issue of shortcut learning through implicit concepts and propose a solution for it. \n- While their method is clearly better in all classification aspects compared to CBM, the authors also perform an analysis of the computational overhead.  \n- Authors provide experiments on the impact of both introduced parameters, JSD weight $\\beta$ and concept-label trade-off term $\\alpha$. (but in the appendix, see below) \n\nWeaknesses:\n- The experiments section is sometimes hard to follow, there are some key aspects of the experiment that are unclear to me:\n  -  What is the value of $\\beta$ in Table 2 for DCBM ?\n  - How did the authors select $\\tilde{d}$, the implicit concept dimension ? What is the impact of this parameter on performance?\n  - What do the authors mean by \"computational overhead\" in table 5, is it compared to CBM alone? If yes, what is the relative comparison? Is this time per epoch or over an entire training process?\n- The authors did not provide code yet which forbids any conclusion with respect to reproducibility.\n- The impact of JSD weight $\\beta$ is core to this work, as it's the only way to avoid shortcut learning through implicit concepts. Thus the content of figure 7 belongs in the main paper and should be further discussed. \n- Results from Figure 7 (c,d) and Table 3 seem to be inconsistent. Indeed in Figure 7, for CUB 10% and 20% DCBM($\\beta$ = 0.01) relies mainly on implicit concepts with an accuracy below 50% using explicit concepts only. However, in Table 3, forward intervention (which only affects explicit concepts) has a greater impact than for CUB and CUB 50% where DCBM($\\beta$ = 0.01) does achieve good classification accuracy from explicit concepts. The authors never discuss this matter. What are the underlying reasons?\n\n\nThis is not a weakness given the recentness of the work, but I believe authors should consider citing \"Clinical outcome prediction under hypothetical interventions – a representation learning framework for counterfactual reasoning\" (2022) that propose partial concept bottleneck (PCB). They also propose to learn another discrete representation to alleviate the lack of concepts. However, their work is more limited and does not consider the crucial issue of shortcut learning from implicit concepts making interpretability impossible. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "aav6rjgHdZ", "reviewer": "Reviewer_hv7P", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper examines concept bottleneck models and focuses on the case where these models have unexpected behaviour when there is insufficient concept information. The authors propose a strategy for decoupling CBMs into explicit and implicit concepts while retaining high predictive performance and interpretability. They devise two algorithms based on mutual information to automatically correct labels and trace back to incorrect concepts. Based on this a human may intervene to correct incorrect concept definitions.", "summary_of_the_review": "Overall the paper is well written but the experimental section needs a lot of work in terms of comparing with existing works tackling similar issues and a significantly better job needs to be done in terms of contextualising the contributions of the paper compared to existing works. \n\nThe work seems to lack novelty for the most part.", "strength_and_weaknesses": "Overall the work addresses a very relevant problem and presents a good solution to the proposed issue. The method is well motivated and the authors make several comparisons on CUB and DERM7PT data sets.\n\nThere are definitely several shortcomings though. Perhaps the most critical of these is the fact that the authors are missing several crucial citations that can potentially impact the quality of the analysis and significance of the results. In particular, the authors never mention works such as Mahinpei et al 2021 (https://arxiv.org/pdf/2106.13314.pdf) who describe some of the major pitfalls of CBMs. Mahinpei et al 2021 explicitly talk about this unexpected behaviour of concept bottlenecks in terms of leakage and several followup works from the same group e.g Havasi et al 2022 present some solutions with a side channel using mutual information to separate the concept information into two subsets to overcome this leakage issue.  I would like to see some comparisons here as to how does the issue of having insufficient concept information relate to leakage? And similarly, how does the approach compare against Havasi et al 2022.\n \nMahinpei et al show that while Concept models encourage concept dimensions to be uncorrelated, this does not completely\nprevent information leakage – each concept dimension can still encode for multiple concepts and that the\nconcept dimensions can nonetheless be statistically dependent. They state that CBM training should explicitly\nminimize mutual information between concept dimensions – both aligned and unaligned - if concepts are believed to be independent. Similar views are presented in Klys et al 2018. How does the proposed method compare to these suggestions? \n\nFinally, I would also like to see some discussion and analysis on how having the human-in-the-loop affects the concept definitions. Is it expected that the human intervenes on the concepts akin to interventions in the causal sense?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_Zxc3FDKt5", "reviewer": "Reviewer_ZauB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This is a strong paper that proposes methods that right issues arising from concept bottleneck models. The authors show how to split concepts into explicit and implicit information, and then demonstrate how to debug concept and label errors.", "summary_of_the_review": "This was a very interesting paper that clearly has legs to improve CBMs. However, the lack of clarity in message makes it hard to accept as is. The density of the paper could be reduced to provide the readers clear motivation for the proposed methods.", "strength_and_weaknesses": "Strengths\n- The paper's presentation is strong. Figures are immaculate!\n- The method addresses serious issues with CBMs. I would have preferred a strong motivating (running) example that naturally suggests the need for implicit and explicit concepts.\n- The theory seems correct; however, the authors could do a much better job at motivating its need (perhaps using the aforementioned example). At the moment, it is unclear how the proposed theory impacts the subsequent algorithms/experiments.\n- The notion of forward intervention and backward rectification are sensible and interesting. \n\nWeaknesses\n- While the paper is quite a whirlwind and looks thorough, there are some clarity issues. Instead of easing readers into a clear, convincing narrative, the authors elected to inundate the readers with results without much interpretation. \n- There is a nomenclature issue with Section 6. It's a stretch to call the proposed system as a human-machine system. There is no human subject experiment to validate this. It's unclear if humans will intervene and rectify sensibly. The experiments presume orcale-esque behavior from the human. Instead of walking through the results, the authors make it difficult to follow the results and their implications. Supporting prose can be of much help to all.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "BgMgNiSBAcS", "reviewer": "Reviewer_VESv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Concept Bottleneck Model (CBM) has the advantage of interpretability. One of the issues that limits their use is insufficiency of high-level concepts, that is, available concepts (explicit concepts) do not encode sufficient information to predict class labels. Hence, the paper suggested to use additional concepts (implicit concepts) which are essentially additional neurons in the bottleneck layer. Implicit concepts are introduced to avoid pollution of explicit concepts during training. The paper proposes Decoupling Concept Bottleneck Model (DCBM) which is a concept-based model decoupling heterogeneous information into explicit and implicit concepts. Moreover, they proposed two methods (forward intervention and backward rectification,) which can automatically correct labels and trace back to wrong concepts.\n", "summary_of_the_review": "It seems that there are interesting ideas but not properly described. More importantly, I have major concerns about the experiments and I hope the authors can clarify. ", "strength_and_weaknesses": "+The theoretical arguments about the tradeoff between concept prediction and label prediction accuracy when concepts are not sufficient to predict class labels are interesting and insightful. \n\n+The paper focuses on decoupling implicit and explicit concepts which is an interesting solution to keep the explicit concept prediction accuracy intact and maintain high label accuracy.\n\n+Maintaining high interpretability in CBMs is important and the authors have tried to preserve it by using a JS Divergence term.\n\n+Leveraging mutual information minimization to enable forward and backward intervention on explicit concepts is an interesting approach that the authors have employed.\n\n----------------\n-Backward intervention is poorly described. Figure 4 ---which is about forward and backward intervention--- is not discussed in the text at all. The related equations are not properly described and are deferred to the appendix A but that appendix contain only a sudo-code with no additional explanation about the backward intervention formulation. What are the intuitions behind eq 4, 5 and 6?\n\n-DCBM is claimed to be an interpretable model, but because the final decision is made based on both human-friendly concepts (explicit concepts) and implicit concepts (not interpretable), it seems that the entire model becomes not interpretable. Although JS Divergence loss term is applied to reduce the impact of implicit concepts on the prediction, the final decision still depends on both implicit and explicit concepts. \n\n-It was mentioned that joint concept/label training in CBM suffers from concept distortion and the concept decoupling model is proposed to mitigate this problem. When the weight of JS Divergence goes to infinity, DCBM converges to CBM. When using less number of explicit concepts, it is expected that the label accuracy of CBM (as well as DCBM with large JS weight) decreases. But we don’t see such a pattern in Figure 7. Looking at sub-figures a-d they all have a similar label accuracy of around 80%. This is counterintuitive. According to Figure 7, CUB%10 and CUB have almost similar CBM label accuracy. Same for Exp-CBM with the largest JS weight. \n\n-A similar counter-intuitive observation can be made in Table 3. Label accuracy reported for CUB, CUB%50, CUB%20 and CUB%10 all are in the range of 80%. I expected that all methods perform much worse on CUB%10 dataset. How this similar performance can be justified?\n\n\n-Another counterintuitive behavior in Figure 7 is about the label accuracy for IMP-DCBM. My understanding is that as the JS weight increases, implicit concepts become less informative about class labels (harsher decoupling) and the label accuracy of IMP-DCBM should decrease. But we don’t see such a pattern in any of the sub-figures of Figure 7. In Figure 7, by increasing JS weight, label accuracy of IMP-DCBM increases and then decreases.  How the initial increase can be justified? \n\n-How can we guarantee that the backward intervention captures wrong predicted concepts and not correct concepts?\n\n-What are the exact loss terms used as L_Y and L_C. Are they conventional cross-entropy? \n\n-In Table 4 that the concept accuracy on samples with wrong label prediction is reported, what is the proportion of the misclassified samples that the authors do intervention on? Did you rectify all the samples with a wrong assigned label?\n-It would be interesting to have an experiment to show how well the decoupling network is able to reduce the degree of mutual information when different number of explicit concepts are used.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject