Original: P OST - HOC C ONCEPT B OTTLENECK M ODELS
Revised: P OST - HOC C ONCEPT B OTTLENECK M ODELS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (“the bottleneck”) and use the concepts to make predictions.
Revised: Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (“the bottleneck”) and use the concepts to make predictions.

Original: A concept bottleneck enhances interpretability since it can be investigated to understand what concepts the model ”sees” in an input and which of these concepts are deemed important.
Revised: A concept bottleneck enhances interpretability since it can be investigated to understand what concepts the model ”sees” in an input and which of these concepts are deemed important.

Original: However, CBMs are restrictive in practice as they require dense concept annotations in the training data to learn the bottleneck.
Revised: However, CBMs are restrictive in practice as they require dense concept annotations in the training data to learn the bottleneck.

Original: Moreover, CBMs often do not match the accuracy of an unrestricted neural network, reducing the incentive to deploy them in practice.
Revised: Moreover, CBMs often do not match the accuracy of an unrestricted neural network, reducing the incentive to deploy them in practice.

Original: In this work, we address these limitations of CBMs by introducing Post-hoc Concept Bottleneck models (PCBMs).
Revised: In this work, we address these limitations of CBMs by introducing Post-hoc Concept Bottleneck models (PCBMs).

Original: We show that we can turn any neural network into a PCBM without sacrificing model performance while still retaining the interpretability benefits.
Revised: We show that we can turn any neural network into a PCBM without sacrificing model performance while still retaining the interpretability benefits.

Original: When concept annotations are not available on the training data, we show that PCBM can transfer concepts from other datasets or from natural language descriptions of concepts via multimodal models.
Revised: When concept annotations are not available on the training data, we show that PCBM can transfer concepts from other datasets or from natural language descriptions of concepts via multimodal models.

Original: A key benefit of PCBM is that it enables users to quickly debug and update the model to reduce spurious correlations and improve generalization to new distributions.
Revised: A key benefit of PCBM is that it enables users to quickly debug and update the model to reduce spurious correlations and improve generalization to new distributions.

Original: PCBM allows for global model edits, which can be more efficient than previous works on local interventions that fix a specific prediction.
Revised: PCBM allows for global model edits, which can be more efficient than previous works on local interventions that fix a specific prediction.

Original: Through a model-editing user study, we show that editing PCBMs via concept-level feedback can provide significant performance gains without usingany data from the target domain or model retraining.
Revised: Through a model-editing user study, we show that editing PCBMs via concept-level feedback can provide significant performance gains without using data from the target domain or model retraining.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: There is growing interest in developing deep learning models that are interpretable and yet still flexible.
Revised: There is growing interest in developing deep learning models that are interpretable and yet still flexible.

Original: One approach is concept analysis (Kim et al., 2018), where the goal is to understand if and how high-level human-understandable features are “engineered” and used by neural networks.
Revised: One approach is concept analysis (Kim et al., 2018), where the goal is to understand if and how high-level human-understandable features are “engineered” and used by neural networks.

Original: For instance, we may like to probe a skin lesion classifier to understand if the Irregular Streaks concept is encoded in the embedding space of the classifier and used later to make the prediction.
Revised: For instance, we may like to probe a skin lesion classifier to understand if the Irregular Streaks concept is encoded in the embedding space of the classifier and used later to make the prediction.

Original: Our work builds on the earlier idea of concept bottlenecks, specifically Concept Bottleneck Models (CBMs) (Koh et al., 2020).
Revised: Our work builds on the earlier idea of concept bottlenecks, specifically Concept Bottleneck Models (CBMs) (Koh et al., 2020).

Original: Concept bottlenecks are inspired by the idea that we can solve the task of interest by applying a function to an underlying set of human-interpretable concepts.
Revised: Concept bottlenecks are inspired by the idea that we can solve the task of interest by applying a function to an underlying set of human-interpretable concepts.

Original: For instance, when trying to classify whether a skin tumor is malignant, dermatologists look for different visual patterns, e.g. existence of Blue-Whitish Veils can be a useful indicator of melanoma (Menzies et al., 1996; Lucieri et al., 2020).
Revised: For instance, when trying to classify whether a skin tumor is malignant, dermatologists look for different visual patterns, e.g. existence of Blue-Whitish Veils can be a useful indicator of melanoma (Menzies et al., 1996; Lucieri et al., 2020).

Original: CBMs train an entire model in an end-to-end fashion by first predicting concepts (e.g. the presence of Blue-Whitish Veils ), then using these concepts to predict the label.
Revised: CBMs train an entire model in an end-to-end fashion by first predicting concepts (e.g. the presence of Blue-Whitish Veils ), then using these concepts to predict the label.

Original: By constraining the model to only rely on a set of concepts and an interpretable predictor, we can: explain what information the model is using when classifying an input by looking at the weights/rules in the interpretable predictor and understand when the model made a particular mistake due to incorrect concept predictions.
Revised: By constraining the model to only rely on a set of concepts and an interpretable predictor, we can: explain what information the model is using when classifying an input by looking at the weights/rules in the interpretable predictor and understand when the model made a particular mistake due to incorrect concept predictions.

Original: While CBMs provide several of the benefits mentioned above, they have several key limitations:
Revised: While CBMs provide several of the benefits mentioned above, they have several key limitations:

Original: Data: CBMs require access to concept labels during model training,
Revised: Data: CBMs require access to concept labels during model training,

Original: i
Revised: i

Original: e. training data should be annotated with which concepts are present.
Revised: e. training data should be annotated with which concepts are present.

Original: Even though there are a number of densely annotated datasets such as CUB (Wah et al., 2011), this is particularly restrictive for real-world use cases, where training datasets rarely have concept annotations.
Revised: Even though there are a number of densely annotated datasets such as CUB (Wah et al., 2011), this is particularly restrictive for real-world use cases, where training datasets rarely have concept annotations.

Original: Performance: CBMs often do not match the accuracy of an unrestricted model, potentially reducing the incentive to use them in practice.
Revised: Performance: CBMs often do not match the accuracy of an unrestricted model, potentially reducing the incentive to use them in practice.

Original: When the concepts are not enough to solve the desired task, it is not clear how to improve the CBM and match the original model performance, while retaining the interpretability benefits.
Revised: When the concepts are not enough to solve the desired task, it is not clear how to improve the CBM and match the original model performance, while retaining the interpretability benefits.

Original: Model editing : Koh et al.
Revised: Model editing : Koh et al.

Original: (2020) discuss intervening on the model to fix the prediction for a single input, yet it is not shown how to holistically edit and improve the model itself.
Revised: (2020) discuss intervening on the model to fix the prediction for a single input, yet it is not shown how to holistically edit and improve the model itself.

Original: Intervening only changes the model behavior for a single sample, but global editing changes the model behavior completely.
Revised: Intervening only changes the model behavior for a single sample, but global editing changes the model behavior completely.

Original: When the model picks up an unintended cue, or learns spurious associations, using the latter approach and editing the concept bottleneck can improve the model performance more generally than an intervention tailored toward one specific input.
Revised: When the model picks up an unintended cue, or learns spurious associations, using the latter approach and editing the concept bottleneck can improve the model performance more generally than an intervention tailored toward one specific input.

Original: Prior work on CBMs does not discuss how to globally edit a model’s behavior.
Revised: Prior work on CBMs does not discuss how to globally edit a model’s behavior.

Original: Ideally, we would like to edit models with the help of human input in order to lower computational costs and remove assumptions about data access.
Revised: Ideally, we would like to edit models with the help of human input in order to lower computational costs and remove assumptions about data access.

Original: Our contributions.
Revised: Our contributions.

Original: In this work, we propose the Post-hoc Concept Bottleneck Model (PCBM) to address these important challenges.
Revised: In this work, we propose the Post-hoc Concept Bottleneck Model (PCBM) to address these important challenges.

Original: PCBMs can convert any pre-trained model into a concept bottleneck model in a data-efficient manner, and enhance the model with the desired interpretability benefits.
Revised: PCBMs can convert any pre-trained model into a concept bottleneck model in a data-efficient manner, and enhance the model with the desired interpretability benefits.

Original: When the training data does not have concept annotations, which is often the case, PCBM can flexibly leverage concepts annotated in other datasets and natural language descriptions of concepts.
Revised: When the training data does not have concept annotations, which is often the case, PCBM can flexibly leverage concepts annotated in other datasets and natural language descriptions of concepts.

Original: When applicable, PCBMs can remove the laborious concept annotation process by leveraging multimodal models to obtain concept representations; this results in richer and more expressive bottlenecks using natural language descriptions of a concept, making PCBMs more accessible in various settings.
Revised: When applicable, PCBMs can remove the laborious concept annotation process by leveraging multimodal models to obtain concept representations; this results in richer and more expressive bottlenecks using natural language descriptions of a concept, making PCBMs more accessible in various settings.

Original: Furthermore, when the available concepts are not sufficiently rich, we introduce a residual modeling step to the PCBM to recover the original blackbox model’s performance.
Revised: Furthermore, when the available concepts are not sufficiently rich, we introduce a residual modeling step to the PCBM to recover the original blackbox model’s performance.

Original: In experiments across several tasks, we show that PCBMs can be used with comparable performance compared to black-box models.
Revised: In experiments across several tasks, we show that PCBMs can be used with comparable performance compared to black-box models.

Original: While prior work (Koh et al., 2020) demonstrated the possibility of performing local model interventions to change individual predictions, here we propose interventions for changing global model behavior.
Revised: While prior work (Koh et al., 2020) demonstrated the possibility of performing local model interventions to change individual predictions, here we propose interventions for changing global model behavior.

Original: Through user studies, we show that PCBM enables efficient global model edits without retraining or access to data from the target domain and that users can improve PCBM performance by using concept-level feedback to drive editing decisions.
Revised: Through user studies, we show that PCBM enables efficient global model edits without retraining or access to data from the target domain and that users can improve PCBM performance by using concept-level feedback to drive editing decisions.

Original:  P OST - HOC C ONCEPT B OTTLENECK M ODELS
Revised:  P OST - HOC C ONCEPT B OTTLENECK M ODELS

Original: [Figure]
Revised: [Figure]

Original: With the CAV approach, for each concept, e.g. stripes, we train a linear SVM to distinguish the embeddings of examples that contain the concept and use the vector normal to the boundary (CAV).
Revised: With the CAV approach, for each concept, e.g. stripes, we train a linear SVM to distinguish the embeddings of examples that contain the concept and use the vector normal to the boundary (

Original: When annotations are hard to obtain, we can leverage multimodal models and use the text encoder to map each concept to a vector.
Revised: When annotations are hard to obtain, we can leverage multimodal models and use the text encoder to map each concept to a vector.

Original: Next, we project the embeddings produced by the backbone onto the concept subspace defined by the set of vectors.
Revised: Next, we project the embeddings produced by the backbone onto the concept subspace defined by the set of vectors.

Original: We then train an interpretable predictor to classify the examples from their projections.
Revised: We then train an interpretable predictor to classify the examples from their projections.

Original: When the concept library is incomplete, we can construct a PCBM-h by sequentially introducing a residual predictor that maps the embeddings to the target space.
Revised: When the concept library is incomplete, we can construct a PCBM-h by sequentially introducing a residual predictor that maps the embeddings to the target space.

Original: There are two main steps to building a PCBM.
Revised: There are two main steps to building a PCBM.

Original: We let f : X → R d be any pretrained backbone model, where d is the size of the corresponding embedding space and X is the input space.
Revised: We let f : X → R d be any pretrained backbone model, where d is the size of the corresponding embedding space and X is the input space.

Original: For instance, f can be the image encoder of CLIP (Radford et al., 2021) or the model layers up to the penultimate layer of a ResNet (He et al., 2016).
Revised: For instance, f can be the image encoder of CLIP (Radford et al., 2021) or the model layers up to the penultimate layer of a ResNet (He et al., 2016).

Original: An overview of PCBMs can be found in 1, and we describe the steps in detail below.
Revised: An overview of PCBMs can be found in 1, and we describe the steps in detail below.

Original: Learning the Concept Subspace ( C ): To learn concept representations, we make use of CAVs (Concept Activation Vectors) (Kim et al., 2018).
Revised: Learning the Concept Subspace ( C ): To learn concept representations, we make use of CAVs (Concept Activation Vectors) (Kim et al., 2018).

Original: In particular, we first define a concept library
Revised: In particular, we first define a concept library

Original: I =
Revised: I =

Original: { i 1 , i 2 , ..., i N c } , where N c denotes the number of concepts.
Revised: { i 1 , i 2 , ..., i N c } , where N c denotes the number of concepts.

Original: The concepts in the library can be selected by a domain expert or learned automatically from the data (Ghorbani et al., 2019; Yeh et al., 2020).
Revised: The concepts in the library can be selected by a domain expert or learned automatically from the data (Ghorbani et al., 2019; Yeh et al., 2020).

Original: For each concept i , we collect embeddings for the positive examples, denoted by the set P i
Revised: For each concept i , we collect embeddings for the positive examples, denoted by the set P i

Original: =
Revised: =

Original: { f ( x p 1 ) , ..., f ( x p Np ) } , that exhibit the concept, and negative examples N i =
Revised: { f ( x p 1 ) , ..., f ( x p Np ) } , that exhibit the concept, and negative examples N i =

Original: { f ( x n 1 ) , ..., f ( x p Nn ) } that do not contain the concept.
Revised: { f ( x n 1 ) , ..., f ( x p Nn ) } that do not contain the concept.

Original: In practice, densely annotated datasets (Caesar et al., 2018;
Revised: In practice, densely annotated datasets (Caesar et al., 2018;

Original: Fong &
Revised: Fong &

Original: Vedaldi, 2018) are used to collect points that are positive/negative for the concept.
Revised: Vedaldi, 2018) are used to collect points that are positive/negative for the concept.

Original: Importantly, unlike
Revised: Importantly, unlike

Original: CBMs, these samples can be different from the data used to train the backbone model.
Revised: CBMs, these samples can be different from the data used to train the backbone model.

Original: Following (Kim et al., 2018), we train a linear SVM using P i and N i to learn the corresponding CAV, that is, the vector normal to the linear classification boundary.
Revised: Following (Kim et al., 2018), we train a linear SVM using P i and N i to learn the corresponding CAV, that is, the vector normal to the linear classification boundary.

Original: We denote the CAV for concept i as c i .
Revised: We denote the CAV for concept i as c i .

Original: Let
Revised: Let

Original: C ∈ R N c
Revised: C ∈ R N c

Original: ×
Revised: ×

Original: d denote the matrix of concept vectors, where each row c i represents the corresponding concept i .
Revised: d denote the matrix of concept vectors, where each row c i represents the corresponding concept i .

Original: Given an input, we project the embedding of the input onto the subspace spanned by concept vectors (the concept subspace).
Revised: Given an input, we project the embedding of the input onto the subspace spanned by concept vectors (the concept subspace).

Original: Particularly, we let f C ( x ) =
Revised: Particularly, we let f C ( x ) =

Original: proj
Revised: proj

Original: C f ( x ) ∈ R N c , where the ith entry is f ( i ) C ( x ) = ⟨ f ( x ) , c i ⟩ || c i ||∈ R .
Revised: C f ( x ) ∈ R N c , where the ith entry is f ( i ) C ( x ) = ⟨ f ( x ) , c i ⟩ || c i ||∈ R .

Original: It is important to observe that we do not need to annotate the training data with concepts.
Revised: It is important to observe that we do not need to annotate the training data with concepts.

Original: Namely, the dataset used to learn concepts can be different from the original training data.
Revised: Namely, the dataset used to learn concepts can be different from the original training data.

Original: In several of our experiments, we use held-out datasets to learn the concept subspace.
Revised: In several of our experiments, we use held-out datasets to learn the concept subspace.

Original: Leveraging multimodal models to learn concepts: Annotating images with concepts is a laborious process.
Revised: Leveraging multimodal models to learn concepts: Annotating images with concepts is a laborious process.

Original: In practice, this can be an important roadblock before using CBMs.
Revised: In practice, this can be an important roadblock before using CBMs.

Original: To address this, here we show that we can also leverage natural language concept descriptions and multimodal models to implement concept bottlenecks.
Revised: To address this, here we show that we can also leverage natural language concept descriptions and multimodal models to implement concept bottlenecks.

Original: This approach alleviates the need of collecting concept-labeled data to construct the concept subspace.
Revised: This approach alleviates the need of collecting labeled data to construct the concept subspace.

Original: Multimodal models such as CLIP (Radford et al., 2021) have a text encoder f text along with an image encoder, which maps a description to the shared embedding space.
Revised: Multimodal models such as CLIP (Radford et al., 2021) have a text encoder f text along with an image encoder, which maps a description to the shared embedding space.

Original: We leverage the text encoder to augment the process of learning concept vectors.
Revised: We leverage the text encoder to augment the process of learning concept vectors.

Original: For instance, we can have c textstripes =
Revised: For instance, we can have c textstripes =

Original: f text (“ stripes ”) where the concept vector for stripes is obtained by mapping the prompt using the text encoder.
Revised: f text (“ stripes ”) where the concept vector for stripes is obtained by mapping the prompt using the text encoder.

Original: For each concept description, we can collect the text embeddings and construct our multimodal concept bank C text as our subspace.
Revised: For each concept description, we can collect the text embeddings and construct our multimodal concept bank C text as our subspace.

Original: For a given classification task, we can use ConceptNet (Speer et al., 2017) to obtain concepts that are relevant to these classes.
Revised: For a given classification task, we can use ConceptNet (Speer et al., 2017) to obtain concepts that are relevant to these classes.

Original: ConceptNet is an open knowledge graph, where we can find concepts that have particular relations to a query concept.
Revised: ConceptNet is an open knowledge graph, where we can find concepts that have particular relations to a query concept.

Original: For instance, we can find relations of the form “A
Revised: For instance, we can find relations of the form “A

Original: Cat has { whiskers, four legs, sharp claws, .. } ”.
Revised: Cat has { whiskers, four legs, sharp claws, .. } ”.

Original: Similarly, we can find ”parts” of a given class (e.g.
Revised: Similarly, we can find ”parts” of a given class (e.g.

Original: ”bumper”, ”roof” for ”truck”), or the superclass of a given class (e.g. ”animal”, ”canine” for ”dog”).
Revised: ”bumper”, ”roof” for ”truck”), or the superclass of a given class (e.g. ”animal”, ”canine” for ”dog”).

Original: We restrict ourselves to five sets of relations for each class: the hasA , isA , partOf , HasProperty ,
Revised: We restrict ourselves to five sets of relations for each class: the hasA , isA , partOf , HasProperty ,

Original: MadeOf relations in ConceptNet.
Revised: MadeOf relations in ConceptNet.

Original: We collect all concepts that have these relations with classes in each classification task to build the concept subspace.
Revised: We collect all concepts that have these relations with classes in each classification task to build the concept subspace.

Original: Learning the Interpretable Predictor : Next, we define an interpretable predictor that maps the concept subspace to the model prediction.
Revised: Learning the Interpretable Predictor : Next, we define an interpretable predictor that maps the concept subspace to the model prediction.

Original: Concretely, let g : R N c
Revised: Concretely, let g : R N c

Original: → Y be an interpretable predictor, such as a sparse linear model or a decision tree, where Y = { 1 , 2 , ..., K } denotes the label space.
Revised: → Y be an interpretable predictor, such as a sparse linear model or a decision tree, where Y = { 1 , 2 , ..., K } denotes the label space.

Original: An interpretable predictor is desirable because it provides insight into which concepts the model is relying on when making a decision.
Revised: An interpretable predictor is desirable because it provides insight into which concepts the model is relying on when making a decision.

Original: If a domain expert observes a counter-intuitive phenomenon in the predictor, they can edit the predictor to improve the model.
Revised: If a domain expert observes a counter-intuitive phenomenon in the predictor, they can edit the predictor to improve the model.

Original: To learn the PCBM, we solve the following problem:
Revised: To learn the PCBM, we solve the following problem:

Original: [Equation]
Revised: [Equation]

Original: where f C = proj
Revised: where f C = proj

Original: C f
Revised: C f

Original: ( x ) is the projection onto the concept subspace, L (ˆ y, y ) is a loss function such as cross-entropy loss, Ω( g ) is a complexity measure to regularize the model, and λ is the regularization strength.
Revised: ( x ) is the projection onto the concept subspace, L (ˆ y, y ) is a loss function such as cross-entropy loss, Ω( g ) is a complexity measure to regularize the model, and λ is the regularization strength.

Original: Note that the concept subspace is fixed during PCBM training.
Revised: Note that the concept subspace is fixed during PCBM training.

Original: In this work, we use sparse linear models to learn the interpretable predictor, where g ( f C ( x ))
Revised: In this work, we use sparse linear models to learn the interpretable predictor, where g ( f C ( x ))

Original: =
Revised: =

Original: w T f C ( x ) + b .
Revised: w T f C ( x ) + b .

Original: We apply the softmax function to g if the problem is a classification problem.
Revised: We apply the softmax function to g if the problem is a classification problem.

Original: Similarly, we define Ω( g ) =
Revised: Similarly, we define Ω( g ) =

Original: α || w ||
Revised: α || w ||

Original: + (1 − α )
Revised: + (1 − α )

Original: ||
Revised: ||

Original: w || 22 to be the elastic-net penalty parameterized by α , then normalize by the number of classes and concepts.
Revised: w || 22 to be the elastic-net penalty parameterized by α , then normalize by the number of classes and concepts.

Original: The expressiveness of the concept subspace is crucial to PCBM performance.
Revised: The expressiveness of the concept subspace is crucial to PCBM performance.

Original: However, even when we have a rich concept subspace, concepts may not be enough to solve a task of interest.
Revised: However, even when we have a rich concept subspace, concepts may not be enough to solve a task of interest.

Original: When the performance of the PCBM does not match the performance of the original model, users have less incentive to use interpretable models like concept bottlenecks.
Revised: When the performance of the PCBM does not match the performance of the original model, users have less incentive to use interpretable models like concept bottlenecks.

Original: Next, we aim to address this limitation.
Revised: Next, we aim to address this limitation.

Original: Recovering the original model performance with residual modeling: What happens when the concept bank is not sufficiently expressive, and PCBM performs worse than the original model?
Revised: Recovering the original model performance with residual modeling: What happens when the concept bank is not sufficiently expressive, and PCBM performs worse than the original model?

Original: For instance, there may be skin lesion descriptors that are not available in the concept library.
Revised: For instance, there may be skin lesion descriptors that are not available in the concept library.

Original: Ideally, we would like to preserve the original model’s accuracy while retaining the interpretability benefits.
Revised: Ideally, we would like to preserve the original model’s accuracy while retaining the interpretability benefits.

Original: Drawing inspiration from the semiparametric model literature on fitting residuals (H¨ardle et al., 2004), we introduce Hybrid Post-hoc CBMs (PCBM-h) .
Revised: Drawing inspiration from the semiparametric model literature on fitting residuals (H¨ardle et al., 2004), we introduce Hybrid Post-hoc CBMs (PCBM-h) .

Original: After fixing the concept bottleneck and the interpretable predictor, we re-introduce the embeddings to ‘fit the residuals’.
Revised: After fixing the concept bottleneck and the interpretable predictor, we re-introduce the embeddings to ‘fit the residuals’.

Original: In particular, we solve the following:
Revised: In particular, we solve the following:

Original: [Equation]
Revised: [Equation]

Original: where r : R d → Y is the residual predictor.
Revised: where r : R d → Y is the residual predictor.

Original: Note that in Equation 2, while training the residual predictor, the trained concept bottleneck (the concept subspace ( f C ) and the interpretable predictor( g )) is kept fixed, i.e. fitting PCBM-hs is a sequential procedure.
Revised: Note that in Equation 2, while training the residual predictor, the trained concept bottleneck (the concept subspace ( f C ) and the interpretable predictor( g )) is kept fixed, i.e. fitting PCBM-hs is a sequential procedure.

Original: We hypothesize that the residual predictor will compensate for what is missing from the concept bank and recover the original model’s accuracy.
Revised: We hypothesize that the residual predictor will compensate for what is missing from the concept bank and recover the original model’s accuracy.

Original: We implement the residual predictor as a linear model, i.e. r ( f ( x )) =
Revised: We implement the residual predictor as a linear model, i.e. r ( f ( x )) =

Original: w Tr f ( x ) + b r .
Revised: w Tr f ( x ) + b r .

Original: Given a trained
Revised: Given a trained

Original: PCBM-h, if we would like to observe model performance in the absence of the residual predictor, we can simply drop r (in other words, concept contributions do not depend on the residual step).
Revised: PCBM-h, if we would like to observe model performance in the absence of the residual predictor, we can simply drop r (in other words, concept contributions do not depend on the residual step).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate the PCBM and PCBM-h in challenging image classification and medical settings, demonstrating several use cases for PCBMs.
Revised: We evaluate the PCBM and PCBM-h in challenging image classification and medical settings, demonstrating several use cases for PCBMs.

Original: We further address practical concerns and show that
Revised: We further address practical concerns and show that

Original: PCBMs can be used without a loss in the original model performance.
Revised: PCBMs can be used without a loss in the original model performance.

Original: We used the following datasets to systematically evaluate the PCBM and PCBM-h:
Revised: We used the following datasets to systematically evaluate the PCBM and PCBM-h:

Original: CIFAR10, CIFAR100 (Krizhevsky et al., 2009)
Revised: CIFAR10, CIFAR100 (Krizhevsky et al., 2009)

Original: We use CLIP-ResNet50 (Radford et al., 2021) as the backbone model.
Revised: We use CLIP-ResNet50 (Radford et al., 2021) as the backbone model.

Original: For the concept bottleneck, we use 170 concepts introduced in (Abid et al., 2022) which are extracted from the BRODEN visual concepts dataset (Fong & Vedaldi, 2018).
Revised: For the concept bottleneck, we use 170 concepts introduced in (Abid et al., 2022) which are extracted from the BRODEN visual concepts dataset (Fong & Vedaldi, 2018).

Original: These include objects (e.g. dog ), settings, (e.g. snow ) textures (e.g. stripes ), and image qualities (e.g. blurriness ).
Revised: These include objects (e.g. dog ), settings, (e.g. snow ) textures (e.g. stripes ), and image qualities (e.g. blurriness ).

Original: The full list of concepts can be found in (Abid et al., 2022).
Revised: The full list of concepts can be found in (Abid et al., 2022).

Original: COCO-Stuff (Caesar et al., 2018) is a dataset derived from MS-COCO (Lin et al., 2014), consisting of scenes with various object annotations.
Revised: COCO-Stuff (Caesar et al., 2018) is a dataset derived from MS-COCO (Lin et al., 2014), consisting of scenes with various object annotations.

Original: Previous work (Singh et al., 2020) has shown that there are severe co-occurrence biases in this dataset.
Revised: Previous work (Singh et al., 2020) has shown that there are severe co-occurrence biases in this dataset.

Original: For instance, images from the wine glass often also have a dining table in the image.
Revised: For instance, images from the wine glass often also have a dining table in the image.

Original: Singh et al.
Revised: Singh et al.

Original: (2020) identifies these 20 classes with heavy co-occurrence biases and we train PCBMs to recognize these 20 objects, where we minimize the binary cross-entropy loss individually for each class.
Revised: (2020) identifies these 20 classes with heavy co-occurrence biases and we train PCBMs to recognize these 20 objects, where we minimize the binary cross-entropy loss individually for each class.

Original: In this scenario, we again use CLIP-ResNet50 (Radford et al., 2021) as the backbone and BRODEN visual concepts as the set of concepts.
Revised: In this scenario, we again use CLIP-ResNet50 (Radford et al., 2021) as the backbone and BRODEN visual concepts as the set of concepts.

Original: CUB (Wah et al., 2011)
Revised: CUB (Wah et al., 2011)

Original: In the 200-way bird identification task, we use a ResNet18 (He et al., 2016) trained on the CUB dataset 1 .
Revised: In the 200-way bird identification task, we use a ResNet18 (He et al., 2016) trained on the CUB dataset 1 .

Original: We use the same training/validation splits and 112 concepts as in (Koh et al., 2020).
Revised: We use the same training/validation splits and 112 concepts as in (Koh et al., 2020).

Original: These concepts include wing shape , back pattern , and eye color .
Revised: These concepts include wing shape , back pattern , and eye color .

Original: HAM10000 (Tschandl et al., 2018) is a dataset of dermoscopic images, which contain skin lesions from a representative set of diagnostic categories.
Revised: HAM10000 (Tschandl et al., 2018) is a dataset of dermoscopic images, which contain skin lesions from a representative set of diagnostic categories.

Original: The task is detecting whether a skin lesion is benign or malignant.
Revised: The task is detecting whether a skin lesion is benign or malignant.

Original: We use the Inception (Szegedy et al., 2015) model trained on this dataset, which is available from (Daneshjou et al., 2021).
Revised: We use the Inception (Szegedy et al., 2015) model trained on this dataset, which is available from (Daneshjou et al., 2021).

Original: Following the setting in (Lucieri et al., 2020), we collect concepts from the Derm7pt
Revised: Following the setting in (Lucieri et al., 2020), we collect concepts from the Derm7pt

Original: (Kawahara et al., 2018) dataset.
Revised: (Kawahara et al., 2018) dataset.

Original: The 8 concepts obtained from this dataset include Blue Whitish Veil , Pigmented Networks , Regression Structures , which are reportedly associated with the malignancy of a lesion.
Revised: The 8 concepts obtained from this dataset include Blue Whitish Veil , Pigmented Networks , Regression Structures , which are reportedly associated with the malignancy of a lesion.

Original: SIIM-ISIC (Rotemberg et al., 2021)
Revised: SIIM-ISIC (Rotemberg et al., 2021)

Original: To test a real-world transfer learning use case, we evaluate the model trained on HAM10000 on a subset of the SIIM-ISIC Melanoma Classification dataset.
Revised: To test a real-world transfer learning use case, we evaluate the model trained on HAM10000 on a subset of the SIIM-ISIC Melanoma Classification dataset.

Original: We use the same concepts described in the HAM10000 dataset.
Revised: We use the same concepts described in the HAM10000 dataset.

Original: We use linear probing on top of the backbones for CIFAR, COCO, and ISIC to obtain the baseline model performance.
Revised: We use linear probing on top of the backbones for CIFAR, COCO, and ISIC to obtain the baseline model performance.

Original: For CUB and HAM10000, we use the out-of-the-box models trained on the respective datasets without any additional training.
Revised: For CUB and HAM10000, we use the out-of-the-box models trained on the respective datasets without any additional training.

Original: We emphasize that in all of our experiments except CUB, we learn concepts using a held-out dataset.
Revised: We emphasize that in all of our experiments except CUB, we learn concepts using a held-out dataset.

Original: Namely, while the original CBMs need to have concept annotations for training images, we remove this limitation by post-hoc learning of concepts with any dataset.
Revised: Namely, while the original CBMs need to have concept annotations for training images, we remove this limitation by post-hoc learning of concepts with any dataset.

Original: To learn the concept subspace, we use 50 pairs of images for each concept to train a Linear SVM.
Revised: To learn the concept subspace, we use 50 pairs of images for each concept to train a Linear SVM.

Original: We refer the reader to the Appendix for further details on datasets and training.
Revised: We refer the reader to the Appendix for further details on datasets and training.

Original: PCBMs achieve comparable performance to the original model:
Revised: PCBMs achieve comparable performance to the original model:

Original: In Table 1, we report results over these five datasets.
Revised: In Table 1, we report results over these five datasets.

Original: PCBM achieves comparable performance to the original model in all datasets except CIFAR100, and PCBM-h matches the performance in all scenarios.
Revised: PCBM achieves comparable performance to the original model in all datasets except CIFAR100, and PCBM-h matches the performance in all scenarios.

Original: Strikingly, PCBMs match the performance of the original model in HAM10000 and ISIC, with as few as 8 human-interpretable concepts .
Revised: Strikingly, PCBMs match the performance of the original model in HAM10000 and ISIC, with as few as 8 human-interpretable concepts .

Original: In CIFAR100, we hypothesize that the concept bank available is not sufficient to classify finer-grained classes, and hence there is a performance gap between the PCBM and the original model.
Revised: In CIFAR100, we hypothesize that the concept bank available is not sufficient to classify finer-grained classes, and hence there is a performance gap between the PCBM and the original model.

Original: When the concept bank is not sufficient to solve the task, PCBM-h can be introduced to recover the original model performance while retaining the benefits of PCBM (see next sections).
Revised: When the concept bank is not sufficient to solve the task, PCBM-h can be introduced to recover the original model performance while retaining the benefits of PCBM (see next sections).

Original: Comparing to CBM: A comparison to CBM was not possible in most datasets, as CBMs cannot be trained due to the lack of dense annotations.
Revised: Comparing to CBM: A comparison to CBM was not possible in most datasets, as CBMs cannot be trained due to the lack of dense annotations.

Original: To compare the data efficiency of both methods, we report a comparison of CBM to PCBM when trained on CUB in Appendix C, finding that CBM can only reach a similar performance to PCBM with 20x more annotations.
Revised: To compare the data efficiency of both methods, we report a comparison of CBM to PCBM when trained on CUB in Appendix C, finding that CBM can only reach a similar performance to PCBM with 20x more annotations.

Original: Analyzing the residual predictor: To understand whether PCBM-h overrides PCBM predictions, in Appendix B, we look at the consistency between PCBM and PCBM-h predictions.
Revised: Analyzing the residual predictor: To understand whether PCBM-h overrides PCBM predictions, in Appendix B, we look at the consistency between PCBM and PCBM-h predictions.

Original: We show that the residual component in
Revised: We show that the residual component in

Original: PCBM-h intervenes only when the prediction is wrong, and fixes mistakes.
Revised: PCBM-h intervenes only when the prediction is wrong, and fixes mistakes.

Original: When PCBM is confident,
Revised: When PCBM is confident,

Original: PCBM-h does not modify the prediction or significantly increase the confidence.
Revised: PCBM-h does not modify the prediction or significantly increase the confidence.

Original: In general, the residual component may dominate when the bottleneck is insufficient and future work can aim to explicitly limit the residual component, e.g. PIE (Wang et al., 2021) regularizer.
Revised: In general, the residual component may dominate when the bottleneck is insufficient and future work can aim to explicitly limit the residual component, e g. PIE (Wang et al., 2021) regularizer.

Revised: Original CBMs cannot be trained on CIFAR/HAM10000/ISIC/COCO-Stuff, as they do not have concept labels in the training dataset.

Revised: The mean and standard errors are reported over 10 random seeds.

Revised: We report AUROC for HAM10000 and ISIC,

Original:  [Table]
Revised: COCO CIFAR [Table] [Table]

Revised: of concepts with CLIP

Revised: to implement CBMs.

Original: [Table]
Revised: [Table]

Original: PCBM using CLIP concepts:
Revised: PCBM using CLIP concepts:

Original: Here, we show that when labeled examples to learn concepts are not available, we can use multimodal representations such as CLIP to generate concepts without the laborious annotation cost.
Revised: Here, we show that when labeled examples to learn concepts are not available, we can use multimodal representations such as CLIP to generate concepts without the laborious annotation cost.

Original: Using ConceptNet, we obtain concept descriptions for CIFAR10, CIFAR100, and COCOStuff tasks (206, 527, andconcepts, respectively), using the relations described in Section 2.
Revised: Using ConceptNet, we obtain concept descriptions for CIFAR10, CIFAR100, and COCOStuff tasks (206, 527, andconcepts, respectively), using the relations described in Section 2.

Original: The concept subspace is obtained by using text embeddings for the concept descriptions generated by the ResNet50 variant of CLIP.
Revised: The concept subspace is obtained by using text embeddings for the concept descriptions generated by the ResNet50 variant of CLIP.

Original: In Table 2, we give an overview of the results.
Revised: In Table 2, we give an overview of the results.

Original: We observe that with a more expressive multimodal bottleneck, we can almost recover the original model accuracy in CIFAR10, and we reduce the performance gap in CIFAR100.
Revised: We observe that with a more expressive multimodal bottleneck, we can almost recover the original model accuracy in CIFAR10, and we reduce the performance gap in CIFAR100.

Original: CLIP contains extensive knowledge of natural images, and thus captures natural image concepts (e.g. red , thorns ).
Revised: CLIP contains extensive knowledge of natural images, and thus captures natural image concepts (e.g. red , thorns ).

Original: However, CLIP is less reliable at representing more granular or domain-specific concepts (e.g. blue whitish veils ) and hence this methodology is less applicable to more domain-specific applications.
Revised: However, CLIP is less reliable at representing more granular or domain-specific concepts (e.g. blue whitish veils ) and hence this methodology is less applicable to more domain-specific applications.

Original: While there is a recent trend in training multimodal models for biomedical applications (Zhang et al., 2020; EleutherAI), these models are trained with smaller datasets and are less competitive.
Revised: While there is a recent trend in training multimodal models for biomedical applications (Zhang et al., 2020; EleutherAI), these models are trained with smaller datasets and are less competitive.

Original: We hope that as these models improve, PCBMs will make it easier to deploy concept bottlenecks in more specialized domains.
Revised: We hope that as these models improve, PCBMs will make it easier to deploy concept bottlenecks in more specialized domains.

Original: Explanations in PCBMs: In Figure 2, we provide sample concept weights in the corresponding
Revised: Explanations in PCBMs: In Figure 2, we provide sample concept weights in the corresponding

Original: PCBMs.
Revised: PCBMs.

Original: For instance, in HAM10000, PCBMs use Blue Whitish Veils , Atypical Pigment Networks , and Irregular Streaks to identify malignant lesions, whereas Typical Pigment Networks , Regular
Revised: For instance, in HAM10000, PCBMs use Blue Whitish Veils , Atypical Pigment Networks , and Irregular Streaks to identify malignant lesions, whereas Typical Pigment Networks , Regular

Original: Streaks , and Regular Dots and Globules are used to identify benign lesions.
Revised: Streaks , and Regular Dots and Globules are used to identify benign lesions.

Original: These associations are consistent with medical knowledge (Menzies et al., 1996; Lucieri et al., 2020).
Revised: These associations are consistent with medical knowledge (Menzies et al., 1996; Lucieri et al., 2020).

Original: We observe similar cases in CIFAR-10 and CIFAR-100, where the class Rose is associated with the concepts
Revised: We observe similar cases in CIFAR-10 and CIFAR-100, where the class Rose is associated with the concepts

Original: Flower , Thorns and Red .
Revised: Flower , Thorns and Red .

Original: In Appendix Section F, we further analyze the global explanations of
Revised: In Appendix Section F, we further analyze the global explanations of

Original: COCO-Stuff and show that PCBM reveals co-occurence biases in the training data.
Revised: COCO-Stuff and show that PCBM reveals co-occurrence biases in the training data.

Original: Note that the concept weights for PCBM-h are exactly the same as for PCBM since the residual model is fit after the concepts weights are fixed.
Revised: Note that the concept weights for PCBM-h are exactly the same as for PCBM since the residual model is fit after the concept weights are fixed.

Original: However, given the addition of the residual component, the net effect of concepts may be different.
Revised: However, given the addition of the residual component, the net effect of concepts may be different.

Original: We also note that the concept weights discussed may not reflect the causal effect.
Revised: We also note that the concept weights discussed may not reflect the causal effect.

Original: Estimating the causal effect of a concept requires careful treatment (Goyal et al., 2019), e.g. accounting for the interactions between concepts, and we leave this to future work.
Revised: Estimating the causal effect of a concept requires careful treatment (Goyal et al., 2019), e.g. accounting for the interactions between concepts, and we leave this to future work.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original:  M ODEL EDITING WITH P OST - HOC C ONCEPT B OTTLENECKS
Revised:  M ODEL EDITING WITH P OST - HOC C ONCEPT B OTTLENECKS

Original: When we observe a counter-intuitive phenomenon or a spurious correlation in the concept bottleneck, can we make the model perform better by simple edit operations?
Revised: When we observe a counter-intuitive phenomenon or a spurious correlation in the concept bottleneck, can we make the model perform better by simple edit operations?

Original: In this section, we show that
Revised: In this section, we show that

Original: PCBMs come with the benefit of easy and global concept-level model feedback.
Revised: PCBMs come with the benefit of easy and global concept-level model feedback.

Original: Koh et al.
Revised: Koh et al.

Original: (2020) demonstrate the ability of CBMs to incorporate local interventions.
Revised: (2020) demonstrate the ability of CBMs to incorporate local interventions.

Original: Namely, if a user identifies a misprediction in the concept bottleneck, they can intervene on the concept prediction, and change the prediction for the particular instance, we call this a local intervention or edit.
Revised: Namely, if a user identifies a misprediction in the concept bottleneck, they can intervene on the concept prediction, and change the prediction for the particular instance, we call this a local intervention or edit.

Original: Here, we are interested in editing models to perform ‘global’ edits.
Revised: Here, we are interested in editing models to perform ‘global’ edits.

Original: Namely, with simple concept-level feedback, we would like to edit a PCBM and change the entire model behavior.
Revised: Namely, with simple concept-level feedback, we would like to edit a PCBM and change the entire model behavior.

Original: For instance, in our experiments, we investigate the use case of adapting a PCBM to a new distribution.
Revised: For instance, in our experiments, we investigate the use case of adapting a PCBM to a new distribution.

Original: Unlike most existing model editing approaches, we do not need any data from either the training or target domains to perform the model edit , which can be a significant advantage in practice when data is inaccessible.
Revised: Unlike most existing model editing approaches, we do not need any data from either the training or target domains to perform the model edit , which can be a significant advantage in practice when data is inaccessible.

Original: Given a trained PCBM, we edit our concept bottleneck by manipulating the concept weights.
Revised: Given a trained PCBM, we edit our concept bottleneck by manipulating the concept weights.

Original: For simplicity, in the experiments below we only focus on positive weights in the linear model.
Revised: For simplicity, in the experiments below we only focus on positive weights in the linear model.

Original: 4. C ONTROLLED EXPERIMENT : EDITING PCBM WHEN THE SPURIOUS CONCEPT IS KNOWN
Revised: 4. C ONTROLLED EXPERIMENT : EDITING PCBM WHEN THE SPURIOUS CONCEPT IS KNOWN

Original: For our editing experiments, we use the Metashift (Liang & Zou, 2022) dataset to simulate distribution shifts.
Revised: For our editing experiments, we use the Metashift (Liang & Zou, 2022) dataset to simulate distribution shifts.

Original: We use 10 different scenarios where there is a distribution shift for a particular class between the training and test datasets.
Revised: We use 10 different scenarios where there is a distribution shift for a particular class between the training and test datasets.

Original: For instance, during training, we only use table images that also contain a dog in the image and test the model with table images where there is not a dog in the image.
Revised: For instance, during training, we only use table images that also contain a dog in the image and test the model with table images where there is not a dog in the image.

Original: We denote the training domain as table(dog) .
Revised: We denote the training domain as table(dog) .

Original: We give more details and the results of all domains in the
Revised: We give more details and the results of all domains in the

Original: Appendix.
Revised: Appendix.

Original: We use an ImageNet pretrained ResNet50 variant as our backbone and the visual concept bank described in the Experiments section.
Revised: We use an ImageNet pretrained ResNet50 variant as a backbone and the visual concept bank described in the Experiments section.

Original: Given a PCBM, we evaluate three editing strategies :
Revised: Given a PCBM, we evaluate three editing strategies :

Original: Prune : We set the weight of a particular concept on a particular class prediction to 0 ,
Revised: Prune : We set the weight of a particular concept on a particular class prediction to 0 ,

Original: i
Revised: i

Original: e. for a concept indexed by i , we let ˜ w i = 0 .
Revised: e. for a concept indexed by i , we let ˜ w i = 0 .

Original: Prune+Normalize :
Revised: Prune+Normalize :

Original: After applying pruning, we rescale the concept weights.
Revised: After applying pruning, we rescale the concept weights.

Original: Let P denote the indices of positive weights that are pruned, ˜ P denote the indices of weights that remain, and w P , w ˜ P be corresponding weight vectors for the particular class.
Revised: Let P denote the indices of positive weights that are pruned, ˜ P denote the indices of weights that remain, and w P , w ˜ P be corresponding weight vectors for the particular class.

Original: We rescale each weight to match the original norm of the vector by letting ∀ i ∈ ˜ P, ˜ w i =
Revised: We rescale each weight to match the original norm of the vector by letting ∀ i ∈ ˜ P, ˜ w i =

Original: w i (1+ || w P
Revised: w i (1+ || w P

Original: || 

Revised: ||

Original: || w ˜ P
Revised: || w ˜ P

Original: || 1 )
Revised: || 1 )

Original: , leading to || ˜
Revised: , leading to || ˜

Original: w
Revised: w

Original: ||
Revised: ||

Original: =
Revised: =

Original: ||
Revised: ||

Original: w
Revised: w

Original: || 1 .
Revised: || 1 .

Original: The normalization step alleviates the imbalance between class weights upon pruning concepts with large weights for a particular class.
Revised: The normalization step alleviates the imbalance between class weights upon pruning concepts with large weights for a particular class.

Original: Fine-tune (Oracle) : We compare our simple editing strategies to fine-tuning on the test domain, which can be considered an oracle.
Revised: Fine-tune (Oracle) : We compare our simple editing strategies to fine-tuning on the test domain, which can be considered an oracle.

Original: Particularly, we fine-tune the PCBM using samples from the test domain and then test the fine-tuned model with a set of held-out samples.
Revised: Particularly, we fine-tune the PCBM using samples from the test domain and then test the fine-tuned model with a set of held-out samples.

Original: In the context of Metashift experiments, we simply edit the concept spuriously correlated with a particular class.
Revised: In the context of Metashift experiments, we simply edit the concept spuriously correlated with a particular class.

Original: Concretely, for the domain table(dog) , we prune the weight of the dog concept for the class table .
Revised: Concretely, for the domain table(dog) , we prune the weight of the dog concept for the class table .

Original: In Table 3, we report the result of our editing experiments over 10 scenarios.
Revised: In Table 3, we report the result of our editing experiments over 10 scenarios.

Original: We observe that for PCBMs, the Prune+Normalize strategy can recover almost half of the accuracy gains achieved by fine-tuning on the original test domain.
Revised: We observe that for PCBMs, the Prune+Normalize strategy can recover almost half of the accuracy gains achieved by fine-tuning on the original test domain.

Original: Further, we observe that PCBM-h obtains lower accuracy gains with model editing.
Revised: Further, we observe that PCBM-h obtains lower accuracy gains with model editing.

Original: In PCBM, we can remove the concept from the model, but in
Revised: In PCBM, we can remove the concept from the model, but in

Original: PCBM-h, there may still be leftover information about the spurious concept in the residual part of the model.
Revised: PCBM-h, there may still be leftover information about the spurious concept in the residual part of the model.

Original: This is in line with our expectations, i.e. PCBM-h is a ‘less’ interpretable but more powerful variant of PCBM.
Revised: This is in line with our expectations, i.e. PCBM-h is a ‘less’ interpretable but more powerful variant of PCBM.

Original: While it still offers partial editing improvements that would not be possible with the vanilla model, it does not bring as much improvement as the PCBM.
Revised: While it still offers partial editing improvements that would not be possible with the vanilla model, it does not bring as much improvement as the PCBM.

Original: [Table]
Revised: [Table]

Original: Even though our edit strategy is extremely simple, we can recover almost half of the gains made by fine-tuning the model.
Revised: Even though our edit strategy is extremely simple, we can recover almost half of the gains made by fine-tuning the model.

Original: It is particularly easy to use since it can be applied without fine-tuning or using any knowledge or data from the target domain.
Revised: It is particularly easy to use since it can be applied without fine-tuning or using any knowledge or data from the target domain.

Original: However, this methodology requires knowledge of the spurious concept in the training domain.
Revised: However, this methodology requires knowledge of the spurious concept in the training domain.

Original: This may not be realistic in practice, and in the next section, we turn to human users for guidance.
Revised: This may not be realistic in practice, and in the next section, we turn to human users for guidance.

Original: 4. U SER STUDY : EDITING PCBM WITH HUMAN GUIDANCE
Revised: 4. U SER STUDY : EDITING PCBM WITH HUMAN GUIDANCE

Original: One of the advantages of pruning is that it is naturally amenable to bringing humans into the loop.
Revised: One of the advantages of pruning is that it is naturally amenable to bringing humans into the loop.

Original: Rather than attempting to automate the edits, we can rely on a human to pick concepts that make logical sense to prune.
Revised: Rather than attempting to automate the edits, we can rely on a human to pick concepts that make logical sense to prune.

Original: We show that users make fast and insightful pruning decisions that improve model performance on test data when there is a distribution shift.
Revised: We show that users make fast and insightful pruning decisions that improve model performance on test data when there is a distribution shift.

Original: Notably, these decisions can be made even when the user knows little about the model architecture and has no knowledge of the true underlying spurious correlation in the training data.
Revised: Notably, these decisions can be made even when the user knows little about the model architecture and has no knowledge of the true underlying spurious correlation in the training data.

Original: We conduct a user study where we explore the benefits of human-guided pruning on PCBM and PCBM-h performance.
Revised: We conduct a user study where we explore the benefits of human-guided pruning on PCBM and PCBM-h performance.

Original: [Figure]
Revised: [Figure]

Original: We train PCBMs on MetaShift scenarios, each with a distribution shift between the training and test datasets.
Revised: We train PCBMs on MetaShift scenarios, each with a distribution shift between the training and test datasets.

Original: The user selects concepts to prune from the model.
Revised: The user selects concepts to prune from the model.

Original: Study Design:
Revised: Study Design:

Original: We construct 9 MetaShift scenarios with a distribution shift for one of the classes between the training and testing datasets, which we refer to as the ”shifted class”.
Revised: We construct 9 MetaShift scenarios with a distribution shift for one of the classes between the training and testing datasets, which we refer to as the ”shifted class”.

Original: For instance, all of the training images for the class Keyboard also contains a Cat in the image, whereas test images do not have a cat in the image.
Revised: For instance, all of the training images for the class Keyboard also contain a Cat in the image, whereas test images do not have a cat in the image.

Original: A complete list of the 9 scenarios can be found in the Appendix.
Revised: The list of the 9 scenarios can be found in the Appendix.

Original: For each scenario, we train a PCBM and a PCBM-h with CLIP concepts on the training dataset and task the users with making edits to the trained models by selecting concepts to prune.
Revised: For each scenario, we train a PCBM and a PCBM-h with CLIP concepts on the training dataset and task the users with making edits to the trained models by selecting concepts to prune.

Original: We use CLIP’s ResNet50 variant as our backbone, and leverage ConceptNet to construct the concept subspace.
Revised: We use CLIP’s ResNet50 variant as our backbone, and leverage ConceptNet to construct the concept subspace.

Original: Figure 3 shows the concept-selection interface.
Revised: Figure 3 shows the concept-selection interface.

Original: We refer to the
Revised: We refer to the

Original: Appendix for the full set of instructions given to the user.
Revised: Appendix for the full set of instructions given to the user.

Original: The display consists of the classification task the model was trained on and the concepts with the ten most positive weights for the shifted class.
Revised: The display consists of the classification task the model was trained on and the concepts with the ten most positive weights for the shifted class.

Original: The user selects a subset of the concepts to prune.
Revised: The user selects a subset of the concepts to prune.

Original: Neither the original model accuracy nor the accuracy after pruning is revealed to the user until all scenarios are completed.
Revised: Neither the original model accuracy nor the accuracy after pruning is revealed to the user until all scenarios are completed.

Original: The target audience for the study was machine learning practitioners and researchers;volunteers participated for a total of 30 × 9 = 270 experiments.
Revised: The target audience for the study was machine learning practitioners and researchers;volunteers participated for a total of 30 × 9 = 270 experiments.

Original: The
Revised: The

Original: IRB has determined that this does not require review.
Revised: IRB has determined that this does not require review.

Original: Human-guided editing is fast and improves model accuracy:
Revised: Human-guided editing is fast and improves model accuracy:

Original: Even though we do not impose any time limits, participants make fast pruning decisions.

Original: On average, users prune 3 .
Revised: On average, users prune 3 .

Original: 65 ± 0 .
Revised: 65 ± 0 .

Original: 39 concepts in 34 .
Revised: 39 concepts in 34 .

Original: 3 ± 6 .
Revised: 3 ± 6 .

Original: 4 s per scenario.
Revised: 4 s per scenario.

Original: We evaluate the effectiveness of user pruning by comparing the model accuracy after user pruning to the accuracy of the unedited model on a held-out dataset from the test domain.
Revised: We evaluate the effectiveness of user pruning by comparing the model accuracy after user pruning to the accuracy of the unedited model on a held-out dataset from the test domain.

Original: All 30 / 30 users improve the model accuracy when averaged over scenarios.
Revised: All 30 / 30 users improve the model accuracy when averaged over scenarios.

Original: 8 / 30 users improve model accuracy in all scenarios and 26 / 30 users improve model accuracy in at least 6 scenarios.
Revised: 8 / 30 users improve model accuracy in all scenarios and 26 / 30 users improve model accuracy in at least 6 scenarios.

Original: We compareour human-guided editingstrategy to three baselines:
Revised: We compare human-guided editing to three baselines:

Original: Random Pruning : We uniformly select without replacement a subset of the top ten concepts to prune, matching the number of concepts pruned by the user for a fair comparison.
Revised: Random Pruning : We uniformly select without replacement a subset of the top ten concepts to prune, matching the number of concepts pruned by the user for a fair comparison.

Original: Greedy Pruning (Oracle) :
Revised: Greedy Pruning (Oracle) :

Original: We iteratively select one of the top ten concepts that, when pruned, improves  model accuracy the most samples from the test domain; again, we match the number of concepts pruned by the user.
Revised: We greedily select one of the top 10 concepts that, when pruned, improves the model accuracy the most. We match the number of concepts pruned by the user.

Original: Fine-tune (Oracle) : The model is fine-tuned on samples from the test domain.
Revised: Fine-tune (Oracle) : The model is fine-tuned on samples from the test domain.

Original: [Table]
Revised: [Table]

Original: We consider greedy pruning and retraining to be oracles since they require ”leaked” knowledge of the test domain.
Revised: We consider greedy pruning and retraining to be oracles since they require ”leaked” knowledge of the test domain.

Original: Table 4 reports the model performance improvements for PCBM and PCBM-h, averaged over all 270 experiments in the user study.
Revised: Table 4 reports the model performance improvements for PCBM and PCBM-h, averaged over all 270 experiments in the user study.

Original: Improvements within each separate scenario can be found in the Appendix.
Revised: Improvements within each separate scenario can be found in the Appendix.

Original: We note that even with the residual component in PCBM-h, we can still improve model performance by editing the concept bottleneck.
Revised: We note that even with the residual component in PCBM-h, we can still improve model performance by editing the concept bottleneck.

Original: We observe that user pruning achieves marked improvement in model performance compared to the unedited model, attaining around 50% of the accuracy gains from fine-tuning and 80% of the gains from greedy pruning.
Revised: We observe that user pruning achieves marked improvement in model performance compared to the unedited model, attaining around 50% of the accuracy gains from fine-tuning and 80% of the gains from greedy pruning.

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: Concepts Using human concepts to interpret model behavior has been drawing increasing interest (Kim et al., 2018;
Revised: Concepts Using human concepts to interpret model behavior has been drawing increasing interest (Kim et al., 2018;

Original: Bau et al., 2017; 2020).
Revised: Bau et al., 2017; 2020).

Original: Related work focuses on understanding if neural networks encode and use concepts (Lucieri et al., 2020;
Revised: Related work focuses on understanding if neural networks encode and use concepts (Lucieri et al., 2020;

Original: Kim et al., 2018; McGrath et al., 2021), or generate counterfactual explanations to understand model behavior (Ghandeharioun et al., 2021; Abid et al., 2022; Akula et al., 2020).
Revised: Kim et al., 2018; McGrath et al., 2021), or generate counterfactual explanations to understand model behavior (Ghandeharioun et al., 2021; Abid et al., 2022; Akula et al., 2020).

Original: Recent works evaluated the causal validity of explanations (Feder et al., 2021; Elazar et al., 2021;
Revised: Recent works evaluated the causal validity of explanations (Feder et al., 2021; Elazar et al., 2021;

Original: Goyal et al., 2019), e.g. to eliminate potential confounding effects.
Revised: Goyal et al., 2019), e.g. to eliminate potential confounding effects.

Original: There is further increasing interest in automatically discovering the concepts that are used by a model (Yeh et al., 2020; Ghorbani et al., 2019; Lang et al., 2021).
Revised: There is further increasing interest in automatically discovering the concepts that are used by a model (Yeh et al., 2020; Ghorbani et al., 2019; Lang et al., 2021).

Original: Concept-based Models: Concept bottleneck models (CBMs)
Revised: Concept-based Models: Concept bottleneck models (CBMs)

Original: (Koh et al., 2020) extend the earlier idea (Lampert et al., 2009; Kumar et al., 2009) of first predicting the concepts, then using concepts to predict the target.
Revised: (Koh et al., 2020) extend the earlier idea (Lampert et al., 2009; Kumar et al., 2009) of first predicting the concepts, then using concepts to predict the target.

Original: CBMs bring about interpretability benefits but require training the model using concept labels for the entire training dataset, which is a key limitation.
Revised: CBMs bring about interpretability benefits but require training the model using concept labels for the entire training dataset, which is a key limitation.

Original: CBMs have not been analyzed in terms of model edits.
Revised: CBMs have not been analyzed in terms of model edits.

Original: Recent work reveals that end-to-end learned CBMs encode information beyond the desired concept (Mahinpei et al., 2021; Margeloiu et al., 2021).
Revised: Recent work reveals that end-to-end learned CBMs encode information beyond the desired concept (Mahinpei et al., 2021; Margeloiu et al., 2021).

Original: Concept Whitening (Chen et al., 2020) aims to align each concept with an individual dimension in a layer.
Revised: Concept Whitening (Chen et al., 2020) aims to align each concept with an individual dimension in a layer.

Original: While a subset of dimensions are aligned, this approach lacks a bottleneck since there are dimensions that are potentially not aligned with a concept (comparable to PCBM-h).
Revised: While a subset of dimensions are aligned, this approach lacks a bottleneck since there are dimensions that are potentially not aligned with a concept (comparable to PCBM-h).

Original: Barnett et al.
Revised: Barnett et al.

Original: (2021) learns prototypes and uses them as concepts, where the distance to prototypes determine the model prediction.
Revised: (2021) learns prototypes and uses them as concepts, where the distance to prototypes determine the model prediction.

Original: PCBM-h is inspired by semiparametric models on fitting residuals (H ¨ ardle et al., 2004).
Revised: PCBM-h is inspired by semiparametric models on fitting residuals (H ¨ ardle et al., 2004).

Original: PIE (Wang et al., 2021) has a similar approach, where they combine individual features withmore complicated interaction terms for tabular data.
Revised: PIE (Wang et al., 2021) has a similar approach, where they combine individual features with

Original: Model Editing Model editing aims to achieve the removal or modification of information in a given neural network.
Revised: Model Editing Model editing aims to achieve the removal or modification of information in a given neural network.

Original: Several models investigated editing factual knowledge in language models: Zhu et al.
Revised: Several models investigated editing factual knowledge in language models: Zhu et al.

Original: (2020) use variants of fine-tuning to achieve this objective while retaining performance on unmodified factual knowledge and Mitchell et al.
Revised: (2020) use variants of fine-tuning to achieve this objective while retaining performance on unmodified factual knowledge and Mitchell et al.

Original: De Cao et al.
Revised: De Cao et al.

Original: ; Hase et al.
Revised: ; Hase et al.

Original: (2021) update the model by training a separate network to modify model parameters to achieve the desired edit.
Revised: (2021) update the model by training a separate network to modify model parameters to achieve the desired edit.

Original: Similarly, Sotoudeh & Thakur (2021) proposes ”repairing” models by finding minimal parameter updates that satisfy a given specification.
Revised: Similarly, Sotoudeh & Thakur (2021) proposes ”repairing” models by finding minimal parameter updates that satisfy a given specification.

Original: One thread of work focuses on intervening on the latent space of neural networks to alter the generated output towards the desired state, e.g. removal of artifacts or manipulation of object positions (Sinitsin et al., 2020;
Revised: One thread of work focuses on intervening on the latent space of neural networks to alter the generated output towards the desired state, e.g. removal of artifacts or manipulation of object positions (Sinitsin et al., 2020;

Original: Bau et al., 2020; Santurkar et al., 2021).
Revised: Bau et al., 2020; Santurkar et al., 2021).

Original: For instance, Bau et al.
Revised: For instance, Bau et al.

Original: (2020) edits generative models.
Revised: (2020) edits generative models.

Original: Santurkar et al.
Revised: Santurkar et al.

Original: (2021) edits classifiers by modifying ‘rules’, such as making a model perceive the concept of a ‘snowy road‘ as the concept of a ‘road’, and they achieve this by modifying minimal updates to specific feature extractors
Revised: (2021) edits classifiers by modifying ‘rules’, such as making a model perceive the concept of a ‘snowy road‘ as the concept of a ‘road’, and they achieve this by modifying minimal updates to specific feature extractors.

Revised: FIND (Lertvittayakumjorn et al., 2020) prunes individual neurons chosen by users and later fine-tunes the model.

Revised: Right for the Right Concepts (Stammer et al., 2021) similarly fine-tune a neuro-symbolic model with user-provided feedback maps.

Revised: Bontempelli et al.

Original:  (2021) give an overview of concept-based models and a discussion on  strategies while concurrent work ProtoPDebug (Bontempelli ) proposes an efficient debugging strategy for prototype-based concept models.
Revised: Bontempelli et al. (2021) give an overview of concept-based models and a discussion on debugging strategies while concurrent work ProtoPDebug (Bontempelli et al., 2022) proposes an efficient debugging strategy.

Original:  L IMITATIONS AND C ONCLUSION
Revised:  L IMITATIONS AND C ONCLUSION

Original: In this work, we presented Post-hoc CBMs as a way of converting any model into a CBM, retaining the original model performance without losing the interpretability benefits.
Revised: In this work, we presented Post-hoc CBMs as a way of converting any model into a CBM, retaining the original model performance without losing the interpretability benefits.

Original: We leveraged multimodal models as an interface to use concepts, without the laborious concept annotation steps.
Revised: We leveraged multimodal models as an interface to use concepts, without the laborious concept annotation steps.

Original: Further, in addition to the local intervention benefits of CBM, we demonstrated that PCBMs can be leveraged to perform global model interventions.
Revised: Further, in addition to the local intervention benefits of CBM, we demonstrated that PCBMs can be leveraged to perform global model interventions.

Original: Many benefits of CBMs depend on the quality of the concept library.
Revised: Many benefits of CBMs depend on the quality of the concept library.

Original: The concept set should be expressive enough to solve the task of interest.
Revised: The concept set should be expressive enough to solve the task of interest.

Original: Users should be careful about the concept dataset used to learn concepts, which can reflect various biases.
Revised: Users should be careful about the concept dataset used to learn concepts, which can reflect various biases.

Original: While there are several such real-world tasks, it is an open question if human-constructed concept bottlenecks can solve larger-scale tasks(e.g.
Revised: While there are several such real-world tasks, it is an open question if human-constructed concept bottlenecks can solve larger-scale tasks(e.g.

Original: ImageNet level).
Revised: ImageNet level).

Original: Hence, finding concept subspaces for models in an unsupervised fashion is an active area of research that will help with the usability and expressivity of concept bottlenecks.
Revised: Hence, finding concept subspaces for models in an unsupervised fashion is an active area of research that will help with the usability and expressivity of concept bottlenecks.

Original: Multimodal models provide an effective interface for concept-level reasoning, yet it is unclear what is the optimal way to have humans in the loop.
Revised: Multimodal models provide an effective interface for concept-level reasoning, yet it is unclear what is the optimal way to have humans in the loop.

Original: Here, we presented a simple way of taking human input via concept pruning; how to incorporate richer feedback is an interesting direction for future work. 
Revised: Here, we presented a simple way of taking human input via concept pruning; how to incorporate richer feedback is an interesting direction for future work. A CKNOWLEDGMENTS

Original: E THICAL S TATEMENT
Revised: We would like to thank Adarsh Jeewajee, Carlos Guestrin, Edward Chen, Kyle Swanson, Omer Faruk Akgun and Roxana Daneshjou for their support and comments on the manuscript, and all members of the Zou Lab and Guestrin Lab for helpful discussions. We thank the anonymous reviewers for their suggestions to improve this manuscript. J.Z. is supported by NSF CAREER 1942926 and the

Revised: Chan-Zuckerberg Biohub.

Revised: E THICAL S TATEMENT

Original: In our user study, we did not collect any user information.
Revised: In our user study we did not collect any user information.

Original: We applied to the IRB within our institution and “IRB has determined that this research does not involve human subjects as defined in 45 CFR 46.102(f) and therefore does not require review by the IRB”.
Revised: We applied to the IRB within our institution and “IRB has determined that this research does not involve human subjects as defined in 45 CFR 46.102(f) and therefore does not require review by the IRB”.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: The code to reproduce all experiments will be released at redacted for anonymity .
Revised: The code to reproduce all experiments will be released at https://github.com/mertyg/ post-hoc-cbm .

Original: All of the backbones we used to construct concept bottlenecks are released public checkpoints, and we stated all of these in the manuscript
Revised: All of the backbones we used to construct concept bottlenecks are released public checkpoints, and we stated all of these in the manuscript

Original: A T RAINING D ETAILS
Revised: A T RAINING D ETAILS

Original: Hyperparameters:
Revised: Hyperparameters:

Original: In all of our experiments ElasticNet sparsity ratio parameter was α = 0 . 99 .
Revised: In all of our experiments ElasticNet sparsity ratio parameter was α = 0 . 99 .

Original: We trained all our models on a single NVIDIA-Titan Xp gpu.
Revised: We trained all our models on a single NVIDIA-Titan Xp gpu.

Original: All of the models were trained for a total number of 10 epochs.
Revised: All of the models were trained for a total number of 10 epochs.

Original: We tune the regularization strength on a subset of the training set, that is kept as a validation set.
Revised: We tune the regularization strength on a subset of the training set, that is kept as a validation set.

Original: PCBMs are fitted using scikit-learn’s SGDClassifier class, withmaximum steps.
Revised: PCBMs are fitted using scikit-learn’s SGDClassifier class, withmaximum steps.

Original: Hybrid parts are trained with PyTorch, where we used Adam as the optimizer with 0 .
Revised: Hybrid parts are trained with PyTorch, where we used Adam as the optimizer with 0 .

Original: 01 learning rate, with 0 .
Revised: 01 learning rate, with 0 .

Original: L 2 regularization on the residual classifier weights, and trained forepochs.
Revised: L 2 regularization on the residual classifier weights, and trained forepochs.

Original: Dataset Details:
Revised: Dataset Details:

Original: Metashift: For all metashift experiments, we have 5 class classification problems.
Revised: Metashift: For all metashift experiments, we have 5 class classification problems.

Original: We have 50 images for each class as the training set, and 50 images for each class in the test dataset, which is different from the training images.
Revised: We have 50 images for each class as the training set, and 50 images for each class in the test dataset, which is different from the training images.

Original: The regularization strength for Metashift experiments is 0 .
Revised: The regularization strength for Metashift experiments is 0 .

Original: CIFAR:
Revised: CIFAR:

Original: We use the original training and test splits of CIFAR datasets.
Revised: We use the original training and test splits of CIFAR datasets.

Original: The regularization strength for CIFAR10 and CIFAR100 is 2 .
Revised: The regularization strength for CIFAR10 and CIFAR100 is 2 .

Original: KN c .
Revised: KN c .

Original: We use linear probing to evaluate the original model 3.
Revised: We use linear probing to evaluate the original model 3.

Original: COCO-Stuff: We use the original training and test splits of COCO dataset.
Revised: COCO-Stuff: We use the original training and test splits of COCO dataset.

Original: We sample 500 training and 250 test images from the dataset for each class, and we upsample the images whenever there are not enough images.
Revised: We sample 500 training and 250 test images from the dataset for each class, and we upsample the images whenever there are not enough images.

Original: Recognizing each of the 20 biased classes itepsingh2020don is treated as a binary classification task where we minimize binary cross entropy loss separately for 20 classes, and compute the mean average precision metric.
Revised: Recognizing each of the 20 biased classes itepsingh2020don is treated as a binary classification task where we minimize binary cross entropy loss separately for 20 classes, and compute the mean average precision metric.

Original: The regularization strength for COCO-Stuff is 0 .
Revised: The regularization strength for COCO-Stuff is 0 .

Original: We evaluate the original model performance using linear probes with CLIP.
Revised: We evaluate the original model performance using linear probes with CLIP.

Original: Ham10k:
Revised: Ham10k:

Original: We split 80% of the HAM10k dataset and use it as the training set, and use the remaining 20% as the test data.
Revised: We split 80% of the HAM10k dataset and use it as the training set, and use the remaining 20% as the test data.

Original: The regularization strength for Ham10k is 2 .
Revised: The regularization strength for Ham10k is 2 .

Original: KN c . 5.
Revised: KN c . 5.

Original: ISIC:
Revised: ISIC:

Original: We use 2000 images for training (400 malignant, 1600 benign) and evaluate the model on a held-out set of 500 images (100 malignant, 400 benign).
Revised: We use 2000 images for training (400 malignant, 1600 benign) and evaluate the model on a held-out set of 500 images (100 malignant, 400 benign).

Original: The regularization strength for ISIC is 0 .
Revised: The regularization strength for ISIC is 0 .

Original: 001 KN c .
Revised: 001 KN c .

Original: We evaluate the original model performance using a linear probe.
Revised: We evaluate the original model performance using a linear probe.

Original: CUB:
Revised: CUB:

Original: We use the training and test splits provided in Koh et al.
Revised: We use the training and test splits provided in Koh et al.

Original: (Koh et al., 2020).
Revised: (Koh et al., 2020).

Original: The regularization strength for CIFAR10 and CUB is 0 .
Revised: The regularization strength for CIFAR10 and CUB is 0 .

Original: 01 KN c.
Revised: 01 KN c

Original: Visual Concepts : We used Broden Visual concept bank for CIFAR and controlled editing experiments,
Revised: Visual Concepts : We used Broden Visual concept bank for CIFAR and controlled editing experiments,

Original: CUB’s training data for CUB concepts, and derm7pt dataset for dermatology concepts.
Revised: CUB’s training data for CUB concepts, and derm7pt dataset for dermatology concepts.

Original: For each of these, we use 50 pairs of positive and negative images, and learn a linear SVM.
Revised: For each of these, we use 50 pairs of positive and negative images, and learn a linear SVM.

Original: We use the vector normal to the decision boundary as the concept vector.
Revised: We use the vector normal to the decision boundary as the concept vector.

Original: Multimodal concepts: For concept learning, we leveraged the ConceptNet hierarchy.
Revised: Multimodal concepts: For concept learning, we leveraged the ConceptNet hierarchy.

Original: For each classification task, we searched concept net for the class name and obtained concepts that have the following relation with the query concept: hasA , isA , partOf , HasProperty , MadeOf .
Revised: For each classification task, we searched concept net for the class name and obtained concepts that have the following relation with the query concept: hasA , isA , partOf , HasProperty , MadeOf .

Original: We share the code for obtaining natural language concepts using ConceptNet.
Revised: We share the code for obtaining natural language concepts using ConceptNet.

Original: For each of these concepts, we obtain the text embedding using CLIP and use those as the concept vector.
Revised: For each of these concepts, we obtain the text embedding using CLIP and use those as the concept vector.

Original: B R ESIDUAL COMPONENT INTERVENES ONLY WHEN NECESSARY
Revised: B R ESIDUAL COMPONENT INTERVENES ONLY WHEN NECESSARY

Original: Does PCBM-h alter predictions made by the PCBM?
Revised: Does PCBM-h alter predictions made by the PCBM?

Original: We hypothesized that the residual component would intervene only when the concept bottleneck is not sufficient.
Revised: We hypothesized that the residual component would intervene only when the concept bottleneck is not sufficient.

Original: To better understand the effect of residual predictor, we analyzed the prediction consistency between the two models for CIFAR10 and
Revised: To better understand the effect of residual predictor, we analyzed the prediction consistency between the two models for CIFAR10 and

Original: CIFAR100, where we looked at the models with labeled concepts (i.e. 170 concepts).
Revised: CIFAR100, where we looked at the models with labeled concepts (i.e. 170 concepts).

Original: In Figure 4 and 5, x-axis denotes the confidence of PCBM, the orange line gives the accuracy for samples with the given confidence, and the blue line gives the consistency between PCBM and PCBM-h for the same samples, i.e. whether they make the same prediction.
Revised: In Figure 4 and 5, x-axis denotes the confidence of PCBM, the orange line gives the accuracy for samples with the given confidence, and the blue line gives the consistency between PCBM and PCBM-h for the same samples, i.e. whether they make the same prediction.

Original: In CIFAR10 and CIFAR100, we show that
Revised: In CIFAR10 and CIFAR100, we show that

Original: PCBM and PCBM-h consistency is high when the model confidence is high, and the model accuracy is high.
Revised: PCBM and PCBM-h consistency is high when the model confidence is high, and the model accuracy is high.

Original: Consequently, PCBM-h changes the model prediction mostly when the PCBM prediction is likely a mistake, and the confidence is low.
Revised: Consequently, PCBM-h changes the model prediction mostly when the PCBM prediction is likely a mistake, and the confidence is low.

Original: In Figure 5, we show that all of the predictions changed by PCBM-h are to fix model mistakes.
Revised: In Figure 5, we show that all of the predictions changed by PCBM-h are to fix model mistakes.

Original: Further, in Figure 6 we show the PCBM confidence and the mean absolute deviation between the PCBM confidence and the PCBM-h confidence.
Revised: Further, in Figure 6 we show the PCBM confidence and the mean absolute deviation between the PCBM confidence and the PCBM-h confidence.

Original: We observe that PCBM-h has less effect on model confidence as the model gets more confident. 
Revised: We observe that PCBM-h has less effect on model confidence as the model gets more confident. PCBM Confidence 0.

Original: [Figure]
Revised: M e t r i c

Original: [Equation]
Revised: CIFAR PCBM-h Consistency Accuracy 0 . 0 - 0 . PCBM Confidence 0.

Original: [Figure] [Table]
Revised: M e t r i c CIFAR

Original: [Equation]
Revised: PCBM-h Consistency

Original: [Figure]
Revised: Accuracy

Original: C C OMPARISON TO CBM
Revised: C C OMPARISON TO CBM

Original: In most of our experiments, there is no clear way to run CBM as a benchmark, as none of the datasets have dense concept annotations - except CUB.
Revised: In most of our experiments, there is no clear way to run CBM as a benchmark, as none of the datasets have dense concept annotations - except CUB.

Original: In CUB, we run CBM as a baseline.
Revised: In CUB, we run CBM as a baseline.

Original: Particularly, we try the joint training strategy from Koh et al.
Revised: Particularly, we try the joint training strategy from Koh et al.

Original: (Koh et al., 2020), where it was reported to  give the best results in the CUB dataset.
Revised: (Koh et al., 2020), where it was reported to [Figure] [Figure] give the best results in the CUB dataset.

Original: To make the comparison equal,we use the same set of concepts in both models, and use a linear predictor layer.
Revised: To make the comparison equal, use the same set of concepts in both models [Figure] and use a linear predictor layer.

Original: Further, in both cases, we use the same frozen ResNet18 backbone.
Revised: Further, in both cases, we use the same frozen ResNet18 backbone.

Original: We searched over a grid of learning rates from { 0 . 01 , 0 . 01 , 0 . 1 , 1 . 0 } and λ ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } where λ is the coefficient of the concept predictors in the joint training objective (see the CBM paper, Section 3), and the ElasticNet regularization strength in { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } .
Revised: We searched over a grid of learning rates from { 0 . 01 , 0 . 01 , 0 . 1 , 1 . 0 } and λ ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } where λ is the coefficient of the concept predictors in the joint training objective (see the CBM paper, Section 3), and the ElasticNet regularization strength in { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } .

Original: In Table 5, we report the model performance.
Revised: In Table 5, we report the model performance.

Original: Overall, we observe some benefit from using dense concept annotations over the entire training dataset, where CBMs achieve a [Figure]
Revised: Overall, we observe some benefit from using dense concept annotations over the entire training dataset, where CBMs achieve a slightly better performance than PCBMs, and the original backbone.

Original: [Figure]
Revised: We note that this was at the cost of 112 concept annotations for each of the training samples.

Original: We further analyze the behavior of CBM under different number of annotations.
Revised: We further analyze the behavior of CBM under different number of annotations.

Original: In Figure 7 we train the CBM with varying number of annotations.
Revised: In Figure 7 we train the CBM with varying number of annotations.

Original: CBMs require dense annotations, e.g.annotations would mean 11200 / 112 = 100 images.
Revised: CBMs require dense annotations, e.g.annotations would mean 11200 / 112 = 100 images.

Original: On the x-axis, we give the number of annotations used to train the CBM.
Revised: On the x-axis, we give the number of annotations used to train the CBM.

Original: On the y-axis, we give the accuracy on the test set.
Revised: On the y-axis, we give the accuracy on the test set.

Original: We observe that CBMs require a much larger amount of annotations to achieve the same accuracy as PCBMs.
Revised: We observe that CBMs require a much larger amount of annotations to achieve the same accuracy as PCBMs.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure] 6 # of Annotations
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure] [Figure] [Figure]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: D C ONTROLLED M ETASHIFT E XPERIMENTS FOR M ODEL E DITINGFor
Revised: D C ONTROLLED M ETASHIFT E XPERIMENTS FOR M ODEL E DITING

Original:  Metashift, we have 2 tasks.
Revised: For Metashift, we have 2 tasks.

Original: Both tasks are 5-class object recognition tasks, where in the first one classes are airplane, bed, car, cow, keyboard , and for the second one we have beach, computer, motorcycle, stove, table .
Revised: Both tasks are 5-class object recognition tasks, where in the first one classes are airplane, bed, car, cow, keyboard , and for the second one we have beach, computer, motorcycle, stove, table .

Original: For each of these, we use a ResNet18 pretrained on ImageNet as the backbone of the P-CBM, and then use 100 images per class to train the concept bottleneck.
Revised: For each of these, we use a ResNet18 pretrained on ImageNet as the backbone of the P-CBM, and then use 100 images per class to train the concept bottleneck.

Original: For all experiments, we use the Adam Optimizer with a learning rate of 0 .
Revised: For all experiments, we use the Adam Optimizer with a learning rate of 0 .

Original: 05 , the regularization parameters λ = 0 .
Revised: 05 , the regularization parameters λ = 0 .

Original: 05 , α = 0 . 99 .
Revised: 05 , α = 0 . 99 .

Original: Similar to CIFAR experiments, we use the Broden Concept dataset.
Revised: Similar to CIFAR experiments, we use the Broden Concept dataset.

Original: Below we give the entire set of results.
Revised: Below we give the entire set of results.

Original: [Table]
Revised: [Table]

Original: E U SER S TUDY E. U SER I NTERFACE
Revised: E U SER S TUDY E. U SER I NTERFACE

Original: Below, we provide screenshots of the initial launch page and the concluding summary page from the user study.
Revised: Below, we provide screenshots of the initial launch page and the concluding summary page from the user study.

Original: For the concept-selection interface, refer to the main paper.
Revised: For the concept-selection interface, refer to the main paper.

Original: E. H UMAN - GUIDED PCBM EDITING : D ETAILED PERFORMANCE RESULTS
Revised: E. H UMAN - GUIDED PCBM EDITING : D ETAILED PERFORMANCE RESULTS

Original: In each of the 9 scenarios in the user study, the underlying classification task is a 5-class object recognition task and one of the classes has a spurious correlation in the training set.
Revised: In each of the 9 scenarios in the user study, the underlying classification task is a 5-class object recognition task and one of the classes has a spurious correlation in the training set.

Original: For instance, all training images for the class keyboard also contain a cat in the image.
Revised: For instance, all training images for the class keyboard also contain a cat in the image.

Original: In Table 7, we list the classes in the underlying classification task and the spurious correlation for each scenario.
Revised: In Table 7, we list the classes in the underlying classification task and the spurious correlation for each scenario.

Original: [Table]
Revised: [Table]

Original: Below
Revised: Below

Original: [Table]
Revised: [Table]

Original: we report the test accuracies for each of the 9 scenarios in our user study, both for the shifted class and for the overall classification task.
Revised: we report the test accuracies for each of the 9 scenarios in our user study, both for the shifted class and for the overall classification task.

Original: [Table]
Revised: [Table]

Original: painting(lamp) 0.
Revised: painting(lamp) 0.

Original: 0.617 ± 0.
Revised: 0.617 ± 0.

Original: 0.640 ± 0.
Revised: 0.640 ± 0.

Original: 0.659 ± 0.
Revised: 0.659 ± 0.

Original: pillow(clock) 0.
Revised: pillow(clock) 0.

Original: 0.776 ± 0.
Revised: 0.776 ± 0.

Original: 0.815 ± 0.
Revised: 0.815 ± 0.

Original: 0.815 ± 0. television(fireplace) 0.
Revised: 0.815 ± 0. television(fireplace) 0.

Original: 0.485 ± 0.
Revised: 0.485 ± 0.

Original: 0.535 ± 0.
Revised: 0.535 ± 0.

Original: 0.564 ± 0. fork(tomato) 0.
Revised: 0.564 ± 0. fork(tomato) 0.

Original: 0.583 ± 0.
Revised: 0.583 ± 0.

Original: 0.655 ± 0.
Revised: 0.655 ± 0.

Original: 0.707 ± 0. car(snow) 0.
Revised: 0.707 ± 0. car(snow) 0.

Original: 0.676 ± 0.
Revised: 0.676 ± 0.

Original: 0.807 ± 0.
Revised: 0.807 ± 0.

Original: 0.817 ± 0.
Revised: 0.817 ± 0.

Original: Overall Test Accuracy bed(dog) 0.
Revised: Overall Test Accuracy bed(dog) 0.

Original: 0.850 ± 0.
Revised: 0.850 ± 0.

Original: 0.882 ± 0. 0.882 ± 0.
Revised: 0.882 ± 0. 0.882 ± 0.

Original: keyboard(cat) 0.
Revised: keyboard(cat) 0.

Original: 0.741 ± 0.
Revised: 0.741 ± 0.

Original: 0.811 ± 0.
Revised: 0.811 ± 0.

Original: 0.814 ± 0.
Revised: 0.814 ± 0.

Original: bed(cat) 0.
Revised: bed(cat) 0.

Original: 0.808 ± 0.
Revised: 0.808 ± 0.

Original: 0.834 ± 0.
Revised: 0.834 ± 0.

Original: 0.841 ± 0. couch(cat) 0.
Revised: 0.841 ± 0. couch(cat) 0.

Original: 0.839 ± 0.
Revised: 0.839 ± 0.

Original: 0.851 ± 0.
Revised: 0.851 ± 0.

Original: 0.858 ± 0.
Revised: 0.858 ± 0.

Original: painting(lamp) 0.
Revised: painting(lamp) 0.

Original: 0.841 ± 0.
Revised: 0.841 ± 0.

Original: 0.845 ± 0.
Revised: 0.845 ± 0.

Original: 0.854 ± 0.
Revised: 0.854 ± 0.

Original: pillow(clock) 0.
Revised: pillow(clock) 0.

Original: 0.882 ± 0.
Revised: 0.882 ± 0.

Original: 0.891 ± 0.
Revised: 0.891 ± 0.

Original: 0.891 ± 0. television(fireplace) 0.
Revised: 0.891 ± 0. television(fireplace) 0.

Original: 0.817 ± 0.
Revised: 0.817 ± 0.

Original: 0.827 ± 0.
Revised: 0.827 ± 0.

Original: [Table]
Revised: [Table]

Original: F A NALYSIS ON THE COCO-S TUFF B IASES
Revised: F A NALYSIS ON THE COCO-S TUFF B IASES

Original: Singh et al (Singh et al., 2020) identifies co-occurence biases in the COCO-Stuff dataset, where 20 categories frequently co-occur with other identified categories.
Revised: Singh et al (Singh et al., 2020) identifies co-occurence biases in the COCO-Stuff dataset, where 20 categories frequently co-occur with other identified categories.

Original: In Table 10, reader can find the concepts, taken from Singh et al.
Revised: In Table 10, reader can find the concepts, taken from Singh et al.

Original: In Table 10, we report the Top-5 concepts for the PCBM trained with CLIP concepts.
Revised: In Table 10, we report the Top-5 concepts for the PCBM trained with CLIP concepts.

Original: In the Biased Context column, we give the category that co-occurs frequently with the given context.
Revised: In the Biased Context column, we give the category that co-occurs frequently with the given context.

Original: In PCBM Top-5 Concepts column, we give the concepts that have the highest
Revised: In PCBM Top-5 Concepts column, we give the concepts that have the highest

Original: [Table]
Revised: [Table]

Original: weight for the given category.
Revised: weight for the given category.

Original: Looking at PCBM categories, we can see that we can identify the biased contexts.
Revised: Looking at PCBM categories, we can see that we can identify the biased contexts.

Original: For instance, cup category has table, dining table , apple has fruit, fresh vegetables , car has traffic circle, intersection in the concepts identified as important, which are parallel to the biased context.
Revised: For instance, cup category has table, dining table , apple has fruit, fresh vegetables , car has traffic circle, intersection in the concepts identified as important, which are parallel to the biased context.

Original: In other cases, our model surfaced different potential co-occurence biases, such as chimney for clock , and toilet for hair drier .
Revised: In other cases, our model surfaced different potential co-occurence biases, such as chimney for clock , and toilet for hair drier .

Original: One important limitation for this analysis is that the desired concepts should exist in the concept bank.
Revised: One important limitation for this analysis is that the desired concepts should exist in the concept bank.

Original: To further improve the pipeline, automatic concept discovery approaches shall be a fruitful research direction (Ghorbani et al., 2019;
Revised: To further improve the pipeline, automatic concept discovery approaches shall be a fruitful research direction (Ghorbani et al., 2019;

Original: Yeh et al., 2020).
Revised: Yeh et al., 2020).