Editable Concept Bottleneck Models

Authors: Lijie Hu, Chenyang Ren, Zhengyu Hu, Hongbin Lin, Cheng-Long Wang, Hui Xiong, Jingfeng Zhang, Di Wang
Primary area: interpretability and explainable AI
Keywords: interpretability, explainability, Concept Bottleneck Models

Abstract:
Concept Bottleneck Models (CBMs) have garnered much attention for their ability to elucidate the prediction process through a human-understandable concept layer. However, most previous studies focused on cases where the data, including concepts, are clean. In many scenarios, we always need to remove/insert some training data or new concepts from trained CBMs due to different reasons, such as privacy concerns, data mislabelling, spurious concepts, and concept annotation errors. Thus, the challenge of deriving efficient editable CBMs without retraining from scratch persists, particularly in large-scale applications. To address these challenges, we propose Editable Concept Bottleneck Models (ECBMs). Specifically, ECBMs support three different levels of data removal: concept-label-level, concept-level, and data-level. ECBMs enjoy mathematically rigorous closed-form approximations derived from influence functions that obviate the need for re-training. Experimental results demonstrate the efficiency and effectiveness of our ECBMs, affirming their adaptability within the realm of CBMs.

Decision: reject
Reviewer scores: 5, 6, 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1730601551328

thread_type:
official_review

<thread_object>
creator: reviewer_ak63

<content>
summary:
This paper delineates 3 levels of editable settings which are concept-label-level, concept-level and data-level. Based on the three settings, the author proposes a concept bottleneck model (CBM) to remove the data or concept influence without retraining based on influence function. Furthermore, the author proposes Editable Concept Bottleneck Model (ECBM) and the streamlined version EK-FAC. Moreover, the author provides the mathematical proof for the three settings and the methods. The extensive experiments shows the effectiveness of the method.

strengths:
- The notation and problem definition is clear.
- The paper is easy to follow.
- The effect of influence function in Section 5.3 is insightful.
- The result visualization is clear.

weaknesses:
- The baseline methods are not complete. For example, there are still some methods for EBM like, probability EBM [1], etc.
- The influence function is only suitable for convex function. I wonder how the author applies it on the non-convex function, like deep neural network.
- The approximate equal sign is mixed with the equal sign, which is not rigorous, like in Theorem 4.7. Please correct me if I have some mis-understanding.


[1] Kim, Eunji, et al. "Probabilistic concept bottleneck models." arXiv preprint arXiv:2306.01574 (2023).

questions:
- I am doubtful with the running time in Table 1. How the running time can be less 1 minute even for retraining method.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730840353544

thread_type:
official_review

<thread_object>
creator: reviewer_squb

<content>
summary:
The paper proposes a framework to acquire adaptively edited concept bottleneck models (CBMs), without actually retraining the model. 
The authors propose using influence functions to approximate the effects of three types of edits: concept-label-level, concept-level, and data-level. These edits address real-world challenges such as correcting erroneous concept annotations, adding/removing concepts, or removing problematic data due to privacy or errors.
Under each setting, they provide the closed-form approximation to measure the influence of each entity--a set of data sample's different concept labels, several concepts, and several samples--based on the influence function. 
The experimental results support the effectiveness and the interpretability of their method--matching the performance of retraining-based correction while reducing the computation overhead.

strengths:
* The paper introduces a novel application of influence functions to CBMs, focusing on the editability of models without full retraining. This is a significant contribution in the context of real-world applications where models need frequent updates due to evolving data or concepts.

* The mathematical formulation is rigorous and well-supported by experiments. The use of influence functions for efficient model editing is a novel approach, and the paper provides detailed derivations and approximations for each editing scenario (concept-label, concept, and data-level).

* The presentation of a handful of real-world interpretability results of their approach is appealing and easy to understand.

weaknesses:
* Scailability: Although the authors include a relatively larger-scaled dataset (e.g., CelebA) to their evaluation, the experiments still focus on relatively small datasets compared to large-scale models like Vision-Language Models (VLMs). It remains unclear how well ECBMs would scale to larger datasets or more complex CBM architectures.
Particularly, it is critical to discuss and run analysis on the scalability of the approach, in terms of the number of parameters in a model, as it could be a key limitation of applying ECBM to real-world large-scale applications; given the need for computationally expensive Hessian computations, or of heavy linear algebra operations for the EK-FAC approximation, or of the complexity of the EK-FAC variants.

questions:
* Could you provide more details on how runtime efficiency scales with larger datasets or models? While your experiments show improvements over retraining on smaller datasets, it would be helpful to understand how ECBM performs with larger models like Vision-Language Models.

* You identify influential samples or concepts using influence scores. How do these rankings compare with simpler methods like analyzing weights in a linear label predictor? What additional insights do influence functions provide?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731091128659

thread_type:
official_review

<thread_object>
creator: reviewer_oeij

<content>
summary:
The submission proposes to apply ideas of influence functions to edit concept bottleneck models (CBMs) at three different levels: concept label level (i.e., changing the ground-truth annotation of a concept), concept level (i.e., removing a concept completely), and data level (i.e., removing a sample from the training set). The proposed edits can be computed efficiently and they can account for the non-convexity of the optimization problem by means of the  Eigenvalue-corrected Kronecker-Factored Approximate Curvature (EK-FAC) method.

Experiments are performed on three datasets to showcase the flexibility and effectiveness of the proposed editing methods. Results show that Editable Concept Bottleneck Models (ECBMs) can retain classification performance while being significantly cheaper than retraining the full CBM from scratch.

strengths:
Efficient and fast editing of CBMs is an interesting problem that is valuable to the research community.

Although influence functions are well-studied, the submission proposes novel methods to apply these ideas within the context of CBMs, which is a substantial contribution.

Experimental evidence in support of the proposed methods is convincing. In particular, I found the experiment on membership attacks compelling and clear.

weaknesses:
The main limitation of the submission, in its current form, is presentation. I believe this limitation to be major, and a significant rewriting of the submission is needed to meet the standards of scientific writing. 

There are several incomplete and unclear sentences, and hand-wavy claims throughout the main body of the submission. This makes it hard for a reader to follow the text and appreciate the contribution, which I believe to be interesting and valuable.

I will expand on these points below, and I am looking forward to discussing with the authors to clarify my questions!

questions:
**General questions and comments**

- lines 145-146: please reword "usually, here the map $f$ is linear" because it is a misleading claim. For CBMs, the predictor $f$ **has** to be linear, otherwise the predictions of the model would not be interpretable by construction. Using a nonlinear $f$ on top of the concepts does not lead to interpretable results.

- notation of loss functions throughout the paper: could the authors clarify why the losses change notation throughout the paper, and with what rationale? For example, in Eq. (1), $L_C$ becomes $L_{C_j}$. In Eq. (2), $L_Y$ becomes $L_{Y_i}$. In appendix D, the loss then becomes $L_{Y_l}$, and later $L_{Y_{ir}}$. Does this mean that each concept is trained with a different loss? Furthermore, it is never stated which losses are used in practice.

- lines 170-172: the derivative should be of $\hat{\theta}_{\epsilon,-z_m}$, not $\hat{\theta}$ as it does not depend on $\epsilon$.

- difference with test-time intervention. I do not think I understand the message of this paragraph. An example is provided where the trained CBM wrongly **predicts** a concept $c$. Then, a user corrects the predictor to observe what the model would have predicted had it correctly classified the concept. But the focus of the submission is to edit an existing CBM with respects to errors/changes in the **training set**. Also, the predictor $f$ is linear, so it suffices to look at the weights of the learned predictor to know how changing the predicted concept would affect the downstream prediction. I do not understand what "secondary editing based on the test data" means, as it seems to boil down to training the model on more samples observed at test time. I also do not understand what "This process extends the rectification from the data level to the model level" means.

- lines 246-249: "This is because the ..." this sentence is very confusing. It is intuitive that gradients cannot change the number of parameters of a network.

- line 254: "by inserting a zero-row vector into the $r$-th row of the matrix in the final layer of". This sentence makes several implicit assumptions about the structure of the encoder $g$. They are reasonable, but they should be stated clearly in order not to confuse readers.

- line 264: I do not understand why "the value of 00" is written several times instead of simply 0. Also, what is "MM"?

- theorem 4.3: I do not understand where the mapping phase plays a role in this statement. The gradient is taken with respects to the original predictor $\hat{g}$. The loss is computed only on the concepts not in $M$. And we are assuming each concept is predicted independently, so why would the weights in the last matrix of $g$ that correspond to the deleted concepts change? Wouldn't the gradients be 0 by definition? Then, the final predictor $\hat{g}^*_{-p_M}$ is obtained by simply removing those weights. How is this different from removing the weights to begin with from $\hat{g}$, and the editing the remaining network?

- lemma 4.4: the sentence "if the label ... . For $r \in M$ ..." is not grammatical. It is unclear whether the statement of the Lemma applies **to one concept $r \in M$** or **for all concepts in $M$**. Also, I could not find the proof of this Lemma in the Appendix. Does the statement of the Lemma hold for any loss $L_Y$? Or are there any assumptions being made here. The way I interpret the Lemma is that padding with 0 does not change the solution of the problem in the remaining entries, and I am wondering whether there might be losses where this would not be true.

- general question about two-phase approach. If I understand correctly, the need to use a two-step editing approach is due to the fact that $f$ depends on the output of $g$, and that the optimization process is over two losses: $L_C$ that only involves $g$, and $L_Y$ that involves both. I agree that in the general setting, writing out the full influence function might be hard. But $f$ will always be a linear function of the output of $g$, so wouldn't it be possible to write out the influence function for the end-to-end task, at least in the data level setting? Could the authors expand on what limitations they encountered that informed the use of a two-step approach?

- evaluation metric: could the authors expand on what f1 score they are computing? Is this the average 1-vs-all classification f1 score computed on each class in the dataset? Could the authors expand on the choice of using this metric instead of classification accuracy?

- implementation details: "at the concept-label level ...". Am I understanding correctly that, in this experiment, ground-truth labels are randomly flipped to noisy labels? This process introduces more noise rather than "correcting" any existing noise, right? I was somewhat surprised to see that this process does not seem to affect the overall F1 score significantly, neither from Table 1 not Figure 2, where the drop in performance is less than 1%. Maybe a more compelling experiment could be to noise the ground-truth annotations to a level where performance degrades significantly, and see what percentage of labels need to be restored for ECBM to obtain a predictor that is close to the one trained on the full dataset without noisy labels. This would reflect the motivating example mentioned in the main text.

- table 1: bolding the ECBM line by default is misleading because the "Retrain" method always provides a higher F1 score. The runtime for the "Retrain" method is in the order of seconds, which is quite small. Am I understanding correctly that the "Retrain" method also retrains the CNN backbone as in Koh et al. (2020)? Or is this just retraining the linear layer $f$? Could the authors expand on the data splits/optimizers/hyperparameters/number of epochs used in the experiments to guarantee reproducibility of the results?

- line 451: "We first select 1-10 most influential and 1-10 least influential" is not grammatical, please rephrase.

- Visualization: it is stated that "ECBM can accurately recognize 9 with their correct concept labels", but Fig. 5 does not seem to report ground-truth annotations, only predictions? Could the authors clearly state in Fig. 5 what the 1/0 column represents? I would rephrase claims that state "correctly identify" because there is no ground-truth notion of what should or should not be used to classify a bird. Figure 5 does support the claim that ECBM retrieves the same concepts as the Retrain method, which may or may not be correct.

- Limitations: conclusions should include a discussion of the limitations of the proposed method, and future research directions.

**Writing comments**

- all equations should be numbered.
- proofs should be explicitly referenced after theorems.
- lines 35-36: ".. for placing human-understandable concepts. In the prediction process ...". This sentence is not grammatical.
- line 80: it is unclear what erroneous or poisoned data mean.
- line 82: it is not clear what "the learning models" means.
- line 94: the sentence "and the concept for all data on model parameters" is incomplete and should be rephrased.
- lines 96-97: it is unclear what "due to their composite structure, i.e. the intermediate representation layer" means. All neural networks have intermediate representation layers, why is this a specific limitation about CBMs?
- line 128: the sentence "CBMs work on the image field also includes" is incomplete and should be rephrased.
- line 131-132: "thereby deciphering the interaction of concept models and providing an adaptive solution to concept editing" is a confusing sentence. What does it mean to decipher an interaction?
- line 161: "thus capturing the essence of the original CBMs" should be rephrased. What does it mean to capture the essence of a model? Eqs. (1) and (2) are the standard definition of a CBM, what is the "original CBM" being referred to here?
- line 189: "for certain concepts, we may opt to" is incomplete. These two sentences should be linked by a connector.
- line 193: "estimating the changes in the parameters of the retraining model holds significance" is not grammatical and should be rephrased.
- line 197-198: "the retrained concept predictor label predictor" is not grammatical and should be rephrased.
- line 208-209: "a two-stage edition approach", typo in "edition" -> "editing".
- line 256: typo in "is the subset of is the subset of $T$".
- line 311: the sentence "... due to different reasons, such as the training data involving poisoned or erroneous issues" is not grammatical.
- line 312: $G$ is overloaded, it was already used to define the Fisher information matrix.
- line 328: I do not understand the sentence "with the original loss before unlearning in equation 2".
- line 424: "we conducted experiments ... removed." is unclear and should be rephrased.
- line 455-456: typos in "11-1010", "0.0250.025".
- line 493: There is no need to report results with 6 decimals.
- line 517: "1010" is a typo.
- line 1248: "We can see that there is a *huge gap* between $\hat{f}_{-z_G}$ and $\hat{f}$". Could the authors clarify in what sense they are characterizing the gap between the two functions as *huge*?

**Minor comments**

- abstract: "elucidate the prediction process". It is unclear what prediction process this sentence is referring to.
- abstract: "affirming their adaptability within the realm of CBMs" this sentence sounds very unnatural and should be rephrased.
- line 27: "large multimodal models" (include "models").
- line 323: typo in "loss function **with** respect to".
</content>
</thread_object>

</thread>
<thread>
creation date:
1731180812969

thread_type:
official_review

<thread_object>
creator: reviewer_kddj

<content>
summary:
This paper introduces a method to edit a trained concept bottleneck model without retraining. They use the influence function to estimate the parameters of the edited model in three scenarios: (1) removing concept labels for some samples; (2) removing some concepts for all samples; and (3) removing some data points.

strengths:
The paper addresses an interesting problem by proposing a post-hoc approach to edit concept bottleneck models, providing a effective solution. Additionally, the authors extensively examine three different editing scenarios concerning different kinds of noises in training data.

weaknesses:
- The related work section could be expanded. Other existing methods for model editing without retraining (if available) should be discussed.
- The experiments do not fully support the claims made in earlier sections. They claimed that the motivation for editing a model is to remove the impacts of erroneous labels, spurious concepts, and incorrect concept annotations. However, the experiment design demonstrates none of these situations, but only randomly removing concepts, concept labels, or data points. The authors might consider intentionally introducing some noise (at different levels) to their training data, and then use their model editing strategy to remove their impacts and see if the model’s performance/interpretability benefits from their methods.
- The presentation needs further improvement. There are several inconsistent notations in the paper. Some terms are used before they are defined.  Some equations are numbered but some are not. See more details in the Questions section. Some important proof/deviations are in the appendix, but not referenced appropriately in the main text (e.g., Appendix A).

questions:
- I am curious if the edited model should have similar model parameters to the retrained model, as in principle both remove the impact of certain concepts/labels/samples in different ways.  
- Why does ECBM outperform CBM-IF in terms of F1 scores? Although the authors explained that Hessian matrices might be ill-defined, ill-conditioned, or singular, how is the inversion conducted exactly in these cases?
- In Figure 5, it looks like neither the ECBM nor the Retrain model generates reasonable explanations. For example, I can not see orange upperparts or legs in either figure. Moreover,  the explanations are exactly the same for the two very different figures. Is it a bug or feature?
- As shown in equations (1) and (2), the authors seem to focus on CBM models where the concept predictor and the label predator are trained separately. What about CBM models where these two predictors are trained jointly?
- In equation (1) and (2), what is L_{C_j} and L_{Y_i}?
- In Lines 65-67, what is the difference between l and capital L?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731292664128

thread_type:
official_review

<thread_object>
creator: reviewer_nbld

<content>
summary:
The paper proposes editable concept bottleneck models (ECBMs), that allow a CBM model to be updated efficiently without a full retraining. They consider three different types of editing: concept-level, data-level, and concept-label level, which cover practical scenarios such as data mis-labeling, spurious concepts, and concept annotation errors. 

Using the idea of influence functions, they propose closed-form approximations to update the CBM parameters (both concept predictor and label predictor) under the three types of editing scenarios. They utilize the EK-FAC method to accelerate the computation of Hessian inverse and non-convex loss functions.

strengths:
Being able to edit or adapt a CBM model under the scenarios described in the paper is an important practical problem, with applications in domains such as medical imaging. I like how the paper formally addresses the three scenarios of concept-level, data-level, and concept label-level editing in a theoretical way. 

Addressing data removal issues in a CBM, due to mis-labeling, poisoning, or privacy issues and the adoption of influence functions for this is a novel contribution. Using ECBMs to effectively edit the model’s knowledge about specific samples, and the connection to membership inference attacks is interesting.

weaknesses:
1. Lots of issues with inconsistent and non-intuitive notation which makes it hard to follow the theory. I think it needs to be proof-read carefully and if possible use simpler notations. The loss functions should be defined so that the method can be understood better (are the loss functions convex?). In multiple places, it is not clear how the gradient and Hessian are defined with respect to a function, e.g., is $H_{\hat{g}}$ (last line of page 4) the Hessian of the function w.r.t the parameters of $\hat{g}$ ?

1. The theorems need some explanations to convey the implications. The paper is currently like theorem after theorem and it may be hard for readers to see the connection. In general, the presentation of ideas could be improved. 

1. In the experiments, it is not clear what is the CBM architecture and the number of model parameters? How well does the proposed method scale to a large number of parameters, given that Hessian matrices or their approximations are involved?

1. Limitations of the proposed work are not addressed.

questions:
1. How frequently are such editing updates expected to be made in a practical setting? After training the CBM, one may find some concept labeling errors and/or the need to remove some training samples or concepts. So applying the ECBM method once after training makes sense, but are there scenarios where the editing is done periodically? In situations where the distribution of the test data can change over time, it makes sense to adapt the CBM efficiently, instead of retraining. 

1. Referring to Figure 2, the F1 score decreases as more edits are made to the CBM. This could be because the experiments consider a contrived scenario where a random subset of concepts or data points are removed. In reality, if there are concept/data labeling errors or some spurious concepts, then I think the performance of the (edited) CBM is expected to improve?

1. In Table 1, all the values corresponding to the ECBM method are bold-faced. It is more useful to highlight the best (and perhaps second-best) performing methods.

1. Lines 455–456: note the errors or typos in the values. On line 517, it says the top 1010 most influential concepts. I think it should be top 10.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732446932227

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (Revision Updates)

comment:
Dear Reviewers and ACs,

Thank you for providing your detailed, insightful, and helpful review comments. We addressed your concerns in our responses to each reviewer and revised the paper based on your suggestions. We greatly appreciate how the reviewers' comments have strengthened our paper. Below, we summarize the key revisions made.

1. We add harmful removal experiments at the concept, data, and concept label levels in 'H.2 Improvement via Harmful Data Removal' in the appendix of the revision. In this experiment, retraining and ECBM both demonstrate improved model performance, further highlighting the effectiveness of ECBM.
2. We add experiment to evaluate the multiple editing ability of ECBM in the appendix 'H.3 Periodic Editing Performance'.
3. We expand the discussion on related work concerning machine unlearning in 'I. More Related Work' in the appendix.
4. We include a discussion of the limitations and broader impacts in 'J. Limitations and Broader Impacts' in the appendix.
5. We add explanation for Figure 5 in the appendix 'H 4.1. Explanation for Visualization Results'.
6. We revise the description of retraining, changing it from 'baseline' to 'ground truth,' in the Baselines and Evaluation Metric section.
7. We add notation table in the appendix A.

We noticed that reviewers may have some misunderstandings when comparing retrain and ECBM. For our model editing task, retrain serves as a ground truth rather than a baseline. Our proposed ECBM method is an efficient and accurate approximation of this ground truth. As shown in Table 1, the estimation results of ECBM are very similar to those of the retrain method, while the runtime is significantly reduced. Therefore, to avoid misinterpretation, we update the description of retraining from 'baseline' to 'ground truth' in the Baselines and Evaluation Metric part.

We sincerely hope that the responses and the revised paper assist in your re-evaluation of our paper.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733226745501

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
Dear Reviewers and Area Chair,

Thank you for your valuable feedback and thoughtful comments. We addressed all the concerns raised in the reviews during the rebuttal process. 

For issues related to grammar and clarity, we have ensured that these concerns have been fully resolved through thorough revisions. The only remaining point pertains to differences in writing style. We have followed established conventions in scientific writing and have incorporated feedback from reviewers to improve the overall quality and precision of the revision.

Thank you again for your time and effort in reviewing our work.

Best regards,\
Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
This work studies an approach, based on approximate influence functions, to edit Concept Bottleneck Models (CBM) to accommodate for three levels of interventions: modifying a (potentially erroneous) concept annotated for a given sample, removing a concept from the model altogether, or removing the influence of a training sample. The approach is empirically effective while circumventing the need for retraining the CMB from scratch. 

**Strengths**
* Innovative approach, and nice trick, of influence functions to edit CMBs.
* Experiments support the claims in the paper.
* The three levels of edits are interesting.

**Weaknesses**
* The annotation often is unclear, making it more difficult to understand the methodology fully.
* Lack of clarity in writing and typos.
* Despite the stress on the rigor of the (approximate) mathematical formulations, no guarantees on the level of approximations are given, nor evaluated empirically. Additional assumptions - like the independence of the network layers - are unrealistic and only mentioned in passing.

**Summary**

This paper received profuse discussions among reviewers and authors, resulting in a more comprehensive picture of the paper from the former and an improvement of the quality of the manuscript by the latter. All reviewers agree on the relevant nature of the problem, and on the innovative solution. At the same time, several reviewers noted that the quality of the presentation is far from where they would like it to be. After reviewing all of the comments, questions, and responses from the authors, I turned to review the paper myself. Unfortunately, I concur with the reviewers nBLD, KdDJ, and oeij, that the presentation of the content is still confusing and unclear. I won't list all of my concerns here, but a non-exhaustive list includes:
* line 158: ".. different columns as a vector $g^j(x_i)$", whereas notation indicates that this should be a scalar.
* line 167: loss L is undefined;
* The notation $m$ is used to denote the index of a sample as well as the dimension of the data,
* line 207: The sentence beginning "Observing.." doesn't have a subject nor predicate.

Moreover, while verifying the modifications done by the authors, I noted that they added a Limitations and Broader Impacts section, as suggested by nBLD. However, this section only has 2 simple sentences on the limitations of the approximation used; the rest of it is simply a repeated verbatim copy of the abstract. This is, naturally, not acceptable.

As a result, given the strengths and weaknesses above, as well as revising the discussions and revision, I cannot recommend acceptance. I encourage the authors to continue improving on this work to provide the best presentation for their innovative and nice ideas.