Original: P OST - HOC C ONCEPT B OTTLENECK M ODELS
Revised: P OST - HOC C ONCEPT B OTTLENECK M ODELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (“the bottleneck”) and use the concepts to make predictions.
Revised: Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts (“the bottleneck”) and use the concepts to make predictions.

Original: A concept bottleneck enhances interpretability since it can be investigated to understand what concepts the model ”sees” in an input and which of these concepts are deemed important.
Revised: A concept bottleneck enhances interpretability since it can be investigated to understand what concepts the model ”sees” in an input and which of these concepts are deemed important.

Original: However, CBMs are restrictive in practice as they require dense concept annotations in the training data to learn the bottleneck.
Revised: However, CBMs are restrictive in practice as they require dense concept annotations in the training data to learn the bottleneck.

Original: Moreover, CBMs often do not match the accuracy of an unrestricted neural network, reducing the incentive to deploy them in practice.
Revised: Moreover, CBMs often do not match the accuracy of an unrestricted neural network, reducing the incentive to deploy them in practice.

Original: In this work, we address these limitations of CBMs by introducing Post-hoc Concept Bottleneck models (PCBMs).
Revised: In this work, we address these limitations of CBMs by introducing Post-hoc Concept Bottleneck models (PCBMs).

Original: We show that we can turn any neural network into a PCBM without sacrificing model performance while still retaining the interpretability benefits.
Revised: We show that we can turn any neural network into a PCBM without sacrificing model performance while still retaining the interpretability benefits.

Original: When concept annotations are not available on the training data, we show that PCBM can transfer concepts from other datasets or from natural language descriptions of concepts via multimodal models.
Revised: When concept annotations are not available on the training data, we show that PCBM can transfer concepts from other datasets or from natural language descriptions of concepts via multimodal models.

Original: A key benefit of PCBM is that it enable users to quickly debug and update the model to reduce spurious correlations and improve generalization to new distributions.
Revised: A key benefit of PCBM is that it enables users to quickly debug and update the model to reduce spurious correlations and improve generalization to new distributions.

Original: PCBM allows for global model edits, which can be more efficient than previous works on local interventinos that fixes a specific prediction.
Revised: PCBM allows for global model edits, which can be more efficient than previous works on local interventions that fix a specific prediction.

Original: Through a model-editing user study, we show that editing PCBMs via concept-level feedback can provide significant performance gains without using any data from the target domain or model retraining.
Revised: Through a model-editing user study, we show that editing PCBMs via concept-level feedback can provide significant performance gains without using any data from the target domain or model retraining.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: There is growing interest in developing deep learning models that are interpretable and yet still flexible.
Revised: There is growing interest in developing deep learning models that are interpretable and yet still flexible.

Original: One approach is concept analysis (Kim et al., 2018), where the goal is to understand if and how high-level human-understandable features are “engineered” and used by neural networks.
Revised: One approach is concept analysis (Kim et al., 2018), where the goal is to understand if and how high-level human-understandable features are “engineered” and used by neural networks.

Original: For instance, we may like to probe a skin lesion classifier to understand if the Irregular Streaks concept is encoded in the embedding space of a skin lesion classifier and used later to make the prediction.
Revised: For instance, we may like to probe a skin lesion classifier to understand if the Irregular Streaks concept is encoded in the embedding space of a skin lesion classifier and used later to make the prediction.

Original: Our work builds on the earlier idea of concept bottlenecks, specifically Concept Bottleneck Models (CBMs) (Koh et al., 2020).
Revised: Our work builds on the earlier idea of concept bottlenecks, specifically Concept Bottleneck Models (CBMs) (Koh et al., 2020).

Original: Concept bottlenecks are inspired by the idea that we can solve the task of interest by applying a function to an underlying set of human-interpretable concepts.
Revised: Concept bottlenecks are inspired by the idea that we can solve the task of interest by applying a function to an underlying set of human-interpretable concepts.

Original: For instance, when trying to classify whether a skin tumor is malignant, dermatologists look for different visual patterns, e.g. existence of Blue-Whitish Veils can be a useful indicator of melanoma (Menzies et al., 1996; Lucieri et al., 2020).
Revised: For instance, when trying to classify whether a skin tumor is malignant, dermatologists look for different visual patterns, e.g. existence of Blue-Whitish Veils can be a useful indicator of melanoma (Menzies et al., 1996; Lucieri et al., 2020).

Original: CBMs train an entire model in an end-to-end fashion by first predicting concepts (e.g. the presence of Blue-Whitish Veils ), then using these concepts to predict the label.
Revised: CBMs train an entire model in an end-to-end fashion by first predicting concepts (e.g. the presence of Blue-Whitish Veils ), then using these concepts to predict the label.

Original: By constraining the model to only rely on a set of concepts and an interpretable predictor, we can: explain what information the model is using when classifying an input by looking at the weights/rules in the interpretable predictor and understand when the model made a particular mistake due to incorrect concept predictions.
Revised: By constraining the model to only rely on a set of concepts and an interpretable predictor, we can: explain what information the model is using when classifying an input by looking at the weights/rules in the interpretable predictor and understand when the model made a particular mistake due to incorrect concept predictions.

Original: While CBMs provide several of the benefits mentioned above, they have several key limitations:
Revised: While CBMs provide several of the benefits mentioned above, they have several key limitations:

Original: Data: CBMs require access to concept labels during model training,
Revised: Data: CBMs require access to concept labels during model training,

Original: i
Revised: i

Original: e. training data should be annotated with which concepts are present.
Revised: e. training data should be annotated with which concepts are present.

Original: Even though there are a number of densely annotated datasets such as CUB (Wah et al., 2011), this is particularly restrictive for real-world use cases, where training datasets rarely have concept annotations.
Revised: Even though there are a number of densely annotated datasets such as CUB (Wah et al., 2011), this is particularly restrictive for real-world use cases, where training datasets rarely have concept annotations.

Original: Performance: CBMs often do not match the accuracy of an unrestricted model, potentially reducing the incentive to use them in practice.
Revised: Performance: CBMs often do not match the accuracy of an unrestricted model, potentially reducing the incentive to use them in practice.

Original: When the concepts are not enough to solve the
Revised: When the concepts are not enough to solve the

Original: desired task, it is not clear how to improve the CBM and match the original model performance, while retaining the interpretability benefits.
Revised: desired task, it is not clear how to improve the CBM and match the original model performance, while retaining the interpretability benefits.

Original: Model editing : Koh et al.
Revised: Model editing : Koh et al.

Original: (2020) discuss intervening on the model to fix the prediction for a single input yet it is not shown how to holistically edit and improve the model itself.
Revised: (2020) discuss intervening on the model to fix the prediction for a single input yet it is not shown how to holistically edit and improve the model itself.

Original: Intervening only changes the model behavior for a single sample, but global editing changes the model behavior completely.
Revised: Intervening only changes the model behavior for a single sample, but global editing changes the model behavior completely.

Original: When the model picks up an unintended cue, or learns spurious associations, using the latter approach and editing the concept bottleneck can improve the model performance more generally than an intervention tailored toward one specific input.
Revised: When the model picks up an unintended cue, or learns spurious associations, using the latter approach and editing the concept bottleneck can improve the model performance more generally than an intervention tailored toward one specific input.

Original: Prior work on CBMs does not discuss how to globally edit a model’s behavior.
Revised: Prior work on CBMs does not discuss how to globally edit a model’s behavior.

Original: Ideally, we would like to edit models with the help of human input in order to lower computational costs and remove assumptions about data access.
Revised: Ideally, we would like to edit models with the help of human input in order to lower computational costs and remove assumptions about data access.

Original: Our contributions.
Revised: Our contributions.

Original: In this work, we propose the Post-hoc Concept Bottleneck Model (PCBM) to address these important challenges.
Revised: In this work, we propose the Post-hoc Concept Bottleneck Model (PCBM) to address these important challenges.

Original: PCBMs can convert any pre-trained model into a concept bottleneck model in a data-efficient manner, and enhance the model with the desired interpretability benefits.
Revised: PCBMs can convert any pre-trained model into a concept bottleneck model in a data-efficient manner, and enhance the model with the desired interpretability benefits.

Original: When the training data does not have concept annotations, which is often the case, PCBM can flexibly leverage concepts annotated in other datasets and natural language descriptions of concepts.
Revised: When the training data does not have concept annotations, which is often the case, PCBM can flexibly leverage concepts annotated in other datasets and natural language descriptions of concepts.

Original: When applicable, PCBMs can remove the laborious concept annotation process by leveraging multimodal models to obtain concept representations; this results in richer and more expressive bottlenecks using natural language descriptions of a concept, making PCBMs more accessible  various settings.
Revised: When applicable, PCBMs can remove the laborious concept annotation process by leveraging multimodal models to obtain concept representations; this results in richer and more expressive bottlenecks using natural language descriptions of a concept, making PCBMs more accessible in various settings.

Original: Furthermore, when the available concepts are not sufficiently rich, we introduce a residual modeling step to the PCBM to recover the original blackbox model’s performance.
Revised: Furthermore, when the available concepts are not sufficiently rich, we introduce a residual modeling step to the PCBM to recover the original blackbox model’s performance.

Original: In experiments across several tasks, we show that PCBMs can be used without a loss compared to black-box model performance.
Revised: In experiments across several tasks, we show that PCBMs can be used without a loss compared to black-box model performance.

Original: While prior work (Koh et al., 2020) demonstrated the possibility of performing local model interventions to change individual predictions, here we propose interventions for changing global model behavior.
Revised: While prior work (Koh et al., 2020) demonstrated the possibility of performing local model interventions to change individual predictions, here we propose interventions for changing global model behavior.

Original: Through user studies, we show that PCBM enables efficient global model edits without retraining or access to data from the target domain and that users can improve PCBM performance by using concept-level feedback to drive editing decisions.
Revised: Through user studies, we show that PCBM enables efficient global model edits without retraining or access to data from the target domain and that users can improve PCBM performance by using concept-level feedback to drive editing decisions.

Original:  P OST - HOC C ONCEPT B OTTLENECK M ODELS
Revised:  P OST - HOC C ONCEPT B OTTLENECK M ODELS

Original: [Figure]
Revised: [Figure]

Original: L e a r n i n g t
Revised: L e a r n i n g t

Original: h e C o n c e p t B a n k
Revised: h e C o n c e p t B a n k

Original: [Figure]
Revised: [Figure]

Original:  i tt i n g t
Revised: F i tt i n g t

Original: h e C o n c e p t B o tt l e n e c k
Revised: h e C o n c e p t B o tt l e n e c k

Original: [Figure]
Revised: [Figure]

Original: There are two main steps to building a PCBM.
Revised: There are two main steps to building a PCBM.

Original: We let f : X → R d be any pretrained backbone model, where d is the size of the corresponding embedding space and X is the input space.
Revised: We let f : X → R d be any pretrained backbone model, where d is the size of the corresponding embedding space and X is the input space.

Original: For instance, f can be the image encoder of CLIP (Radford et al., 2021) or the model layers up to the penultimate layer of a ResNet (He et al., 2016).
Revised: For instance, f can be the image encoder of CLIP (Radford et al., 2021) or the model layers up to the penultimate layer of a ResNet (He et al., 2016).

Original: An overview of PCBMs can be found in 1, and we describe the steps in detail below.
Revised: An overview of PCBMs can be found in 1, and we describe the steps in detail below.

Original: Learning the Concept Subspace ( C ): To learn concept representations, we make use of CAVs (Concept Activation Vectors) (Kim et al., 2018).
Revised: Learning the Concept Subspace ( C ): To learn concept representations, we make use of CAVs (Concept Activation Vectors) (Kim et al., 2018).

Original: In particular, we first define a concept library
Revised: In particular, we first define a concept library

Original: I =
Revised: I =

Original: { i 1 , i 2 , ..., i N c } , where N c denotes the number of concepts.
Revised: { i 1 , i 2 , ..., i N c } , where N c denotes the number of concepts.

Original: The concepts in the library can be selected by a domain expert, or learned automatically from the data (Ghorbani et al., 2019; Yeh et al., 2020).
Revised: The concepts in the library can be selected by a domain expert or learned automatically from the data (Ghorbani et al., 2019; Yeh et al., 2020).

Original: For each concept i , we collect embeddings for the positive examples, denoted by the set P i
Revised: For each concept i , we collect embeddings for the positive examples, denoted by the set P i

Original: =
Revised: =

Original: { f ( x p 1 ) , ..., f ( x p Np ) } , that exhibit the concept, and negative examples N i =
Revised: { f ( x p 1 ) , ..., f ( x p Np ) } , that exhibit the concept, and negative examples N i =

Original: { f ( x n 1 ) , ..., f ( x p Nn ) } that do not contain the concept.
Revised: { f ( x n 1 ) , ..., f ( x p Nn ) } that do not contain the concept.

Revised: In practice, densely annotated datasets (Caesar et al., 2018;

Revised: Fong &

Revised: Vedaldi, 2018) are used to collect points that are positive/negative for the concept.

Original: Importantly,note that unlike CBMs, these samples can be different from the data used to train the backbone model.
Revised: Importantly, unlike CBMs, these samples can be different from the data used to train the backbone model.

Original: Following (Kim et al., 2018), we train a linear SVM using P i and N i to learn the corresponding CAV, that is, the vector normal to the linear classification boundary.
Revised: Following (Kim et al., 2018), we train a linear SVM using P i and N i to learn the corresponding CAV, that is, the vector normal to the linear classification boundary.

Original: We denote the CAV for concept i as c i .
Revised: We denote the CAV for concept i as c i .

Original: Let C ∈ R N c
Revised: Let C ∈ R N c

Original: ×
Revised: ×

Original: d denote the matrix of concept vectors, where each row c i represents the corresponding concept i .
Revised: d denote the matrix of concept vectors, where each row c i represents the corresponding concept i .

Original: Given an input, we project the embedding of the input onto the subspace spanned by concept vectors (the concept subspace).
Revised: Given an input, we project the embedding of the input onto the subspace spanned by concept vectors (the concept subspace).

Original: Particularly, we let f C ( x ) =
Revised: Particularly, we let f C ( x ) =

Original: proj
Revised: proj

Original: C f ( x ) ∈ R N c , where the ith entry is f ( i ) C ( x ) = ⟨ f ( x ) , c i ⟩ || c i ||∈ R .
Revised: C f ( x ) ∈ R N c , where the ith entry is f ( i ) C ( x ) = ⟨ f ( x ) , c i ⟩ || c i ||∈ R .

Original: It is important to observe that we do not need to annotate the training data with concepts.
Revised: It is important to observe that we do not need to annotate the training data with concepts.

Original: Namely, the dataset used to learn concepts can be different from the original training data.
Revised: Namely, the dataset used to learn concepts can be different from the original training data.

Original: In several of our experiments, we use held-out datasets to learn the concept subspace.
Revised: In several of our experiments, we use held-out datasets to learn the concept subspace.

Original: Leveraging multimodal models to learn concepts: Annotating images with concepts is a laborious process.
Revised: Leveraging multimodal models to learn concepts: Annotating images with concepts is a laborious process.

Original: In practice, this can be an important roadblock before using CBMs, and here we aim to tackle this problem.
Revised: In practice, this can be an important roadblock before using CBMs, and here we aim to tackle this problem.

Original: To address this, here we show that we can also leverage natural language concept descriptions and multimodal models to implement concept bottlenecks.
Revised: To address this, here we show that we can also leverage natural language concept descriptions and multimodal models to implement concept bottlenecks.

Original: This approach alleviates the need of collecting concept-labeled data to construct the concept subspace.
Revised: This approach alleviates the need of collecting concept-labeled data to construct the concept subspace.

Original: Multimodal models such as CLIP (Radford et al., 2021) have a text encoder f text along with an image encoder, which maps a description to the shared embedding space.
Revised: Multimodal models such as CLIP (Radford et al., 2021) have a text encoder f text along with an image encoder, which maps a description to the shared embedding space.

Original: We leverage the text encoder to augment the process of learning concept vectors.
Revised: We leverage the text encoder to augment the process of learning concept vectors.

Original: For instance, we can have c textstripes =
Revised: For instance, we can have c textstripes =

Original: f text (“ stripes ”) where the concept vector for stripes is obtained by mapping the prompt using the text encoder.
Revised: f text (“ stripes ”) where the concept vector for stripes is obtained by mapping the prompt using the text encoder.

Original: For each concept description, we can collect the text embeddings and construct our multimodal concept bank C text as our subspace.
Revised: For each concept description, we can collect the text embeddings and construct our multimodal concept bank C text as our subspace.

Original: For a given classification task, we can use ConceptNet (Speer et al., 2017) to obtain concepts that are relevant to these classes.
Revised: For a given classification task, we can use ConceptNet (Speer et al., 2017) to obtain concepts that are relevant to these classes.

Original: ConceptNet is an open knowledge graph, where we can find concepts that have particular relations to a query concept.
Revised: ConceptNet is an open knowledge graph, where we can find concepts that have particular relations to a query concept.

Original: For instance, we can find relations of the form “A
Revised: For instance, we can find relations of the form “A

Original: Cat has { whiskers, four legs, sharp claws, .. } ”.
Revised: Cat has { whiskers, four legs, sharp claws, .. } ”.

Original: Similarly, we can find ”parts” of a given class (e.g.
Revised: Similarly, we can find ”parts” of a given class (e.g.

Original: ”bumper”, ”roof” for ”truck”), or the superclass of a given class (e.g. ”animal”, ”canine” for ”dog”).
Revised: ”bumper”, ”roof” for ”truck”), or the superclass of a given class (e.g. ”animal”, ”canine” for ”dog”).

Original: We restrict ourselves to five sets of relations for each class: the hasA , isA , partOf , HasProperty ,
Revised: We restrict ourselves to five sets of relations for each class: the hasA , isA , partOf , HasProperty ,

Original: MadeOf relations in ConceptNet.
Revised: MadeOf relations in ConceptNet.

Original: We collect all concepts that have these relations with classes in each classification task to build the concept subspace.
Revised: We collect all concepts that have these relations with classes in each classification task to build the concept subspace.

Original: Learning the Interpretable Predictor : Next, we define an interpretable predictor that maps the concept subspace to the model prediction.
Revised: Learning the Interpretable Predictor : Next, we define an interpretable predictor that maps the concept subspace to the model prediction.

Original: Concretely, let g : R N c
Revised: Concretely, let g : R N c

Original: → Y be an interpretable predictor, such as a sparse linear model or a decision tree, where Y = { 1 , 2 , ..., K } denotes the label space.
Revised: → Y be an interpretable predictor, such as a sparse linear model or a decision tree, where Y = { 1 , 2 , ..., K } denotes the label space.

Original: An interpretable predictor is desirable because it provides insight into which concepts the model is relying on when making a decision.
Revised: An interpretable predictor is desirable because it provides insight into which concepts the model is relying on when making a decision.

Original: If a domain expert observes a counter-intuitive phenomenon in the predictor, they can edit the predictor to improve the model.
Revised: If a domain expert observes a counter-intuitive phenomenon in the predictor, they can edit the predictor to improve the model.

Original: To learn the PCBM, we solve the following problem:
Revised: To learn the PCBM, we solve the following problem:

Original: [Equation]
Revised: [Equation]

Original: where f C = proj
Revised: where f C = proj

Original: C f
Revised: C f

Original: ( x ) is the projection onto the concept subspace, L (ˆ y, y ) is a loss function such as cross-entropy loss, Ω( g ) is a complexity measure to regularize the model, and λ is the regularization strength.
Revised: ( x ) is the projection onto the concept subspace, L (ˆ y, y ) is a loss function such as cross-entropy loss, Ω( g ) is a complexity measure to regularize the model, and λ is the regularization strength.

Original: Note that the concept subspace is fixed during PCBM training.
Revised: Note that the concept subspace is fixed during PCBM training.

Original: In this work, we use sparse linear models to learn the interpretable predictor, where g ( f C ( x ))
Revised: In this work, we use sparse linear models to learn the interpretable predictor, where g ( f C ( x ))

Original: =
Revised: =

Original: w T f C ( x ) + b .
Revised: w T f C ( x ) + b .

Original: We apply the softmax function to g if the problem is a classification problem.
Revised: We apply the softmax function to g if the problem is a classification problem.

Original: Similarly, we define Ω( g ) =
Revised: Similarly, we define Ω( g ) =

Original: α || w ||
Revised: α || w ||

Original: + (1 − α )
Revised: + (1 − α )

Original: ||
Revised: ||

Original: w || 22 to be the elastic-net penalty parameterized by α , then normalize by the number of classes and concepts.
Revised: w || 22 to be the elastic-net penalty parameterized by α , then normalize by the number of classes and concepts.

Original: The expressiveness of the concept subspace is crucial to PCBM performance.
Revised: The expressiveness of the concept subspace is crucial to PCBM performance.

Original: However, even when we have a rich concept subspace, concepts may not be enough to solve a task of interest.
Revised: However, even when we have a rich concept subspace, concepts may not be enough to solve a task of interest.

Original: When the performance of the PCBM does not match the performance of the original model, users have less incentive to use interpretable models like concept bottlenecks.
Revised: When the performance of the PCBM does not match the performance of the original model, users have less incentive to use interpretable models like concept bottlenecks.

Original: Next, we aim to address this limitation.
Revised: Next, we aim to address this limitation.

Original: Recovering the original model performance with residual modeling: What happens when the concept bank is not sufficiently expressive, and PCBM performs worse than the original model?
Revised: Recovering the original model performance with residual modeling: What happens when the concept bank is not sufficiently expressive, and PCBM performs worse than the original model?

Original: For instance, there may be skin lesion descriptors that are not available in the concept library.
Revised: For instance, there may be skin lesion descriptors that are not available in the concept library.

Original: Ideally, we would like to preserve the original model accuracy while retaining the interpretability benefits.
Revised: Ideally, we would like to preserve the original model’s accuracy while retaining the interpretability benefits.

Original: Drawing inspiration from the semiparametric models on fitting residuals (H¨ardle et al., 2004), we introduce Hybrid Post-hoc CBMs (PCBM-h) .
Revised: Drawing inspiration from the semiparametric models on fitting residuals (H¨ardle et al., 2004), we introduce Hybrid Post-hoc CBMs (PCBM-h) .

Original: Particularly, after fixing the concept bottleneck and the interpretable predictor, we re-introduce the embeddings to ‘fit the residuals’.
Revised: Particularly, after fixing the concept bottleneck and the interpretable predictor, we re-introduce the embeddings to ‘fit the residuals’.

Original: In particular, we solve the following:
Revised: In particular, we solve the following:

Original: [Equation]
Revised: [Equation]

Original: where r : R d → Y is the residual predictor.
Revised: where r : R d → Y is the residual predictor.

Original: Note that in Equation 2, while training the residual predictor, the trained concept bottleneck (the concept subspace ( f C ) and the interpretable predictor( g )) is kept fixed, i.e. fitting PCBM-hs is a sequential procedure.
Revised: Note that in Equation 2, while training the residual predictor, the trained concept bottleneck (the concept subspace ( f C ) and the interpretable predictor( g )) is kept fixed, i.e. fitting PCBM-hs is a sequential procedure.

Original: We hypothesize that the residual predictor will compensate for what is missing from the concept bank and recover the original model accuracy.
Revised: We hypothesize that the residual predictor will compensate for what is missing from the concept bank and recover the original model’s accuracy.

Original: We implement the residual predictor as a linear model, i.e. r ( f ( x )) =
Revised: We implement the residual predictor as a linear model, i.e. r ( f ( x )) =

Original: w Tr f ( x ) + b r .
Revised: w Tr f ( x ) + b r .

Original: Given a trained
Revised: Given a trained

Original: PCBM-h, if we would like to observe model performance in the absence of the residual predictor, we can simply drop r (in other words, concept contributions do not depend on the residual step).
Revised: PCBM-h, if we would like to observe model performance in the absence of the residual predictor, we can simply drop r (in other words, concept contributions do not depend on the residual step).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate the PCBM and PCBM-h in challenging image classification and medical settings, demonstrating several use cases for PCBMs.
Revised: We evaluate the PCBM and PCBM-h in challenging image classification and medical settings, demonstrating several use cases for PCBMs.

Original: We further address practical concerns and show that
Revised: We further address practical concerns and show that

Original: PCBMs can be used without a loss in the original model performance.
Revised: PCBMs can be used without a loss in the original model performance.

Original: We used the following datasets to systematically evaluate the PCBM and PCBM-h:
Revised: We used the following datasets to systematically evaluate the PCBM and PCBM-h:

Original: CIFAR10, CIFAR100 (Krizhevsky et al., 2009)
Revised: CIFAR10, CIFAR100 (Krizhevsky et al., 2009)

Original: We use CLIP-ResNet50 (Radford et al., 2021) as the backbone model.
Revised: We use CLIP-ResNet50 (Radford et al., 2021) as the backbone model.

Original: For the concept bottleneck, we use 170 concepts introduced in (Abid et al., 2022) which are extracted from the BRODEN visual concepts dataset (Fong & Vedaldi, 2018).
Revised: For the concept bottleneck, we use 170 concepts introduced in (Abid et al., 2022) which are extracted from the BRODEN visual concepts dataset (Fong & Vedaldi, 2018).

Original: These include objects (e.g. dog ), settings (e.g. snow ) textures (e.g. stripes ), and image qualities (e.g. blurriness ).
Revised: These include objects (e.g. dog ), settings (e.g. snow ) textures (e.g. stripes ), and image qualities (e.g. blurriness ).

Original: The full list of concepts can be found in (Abid et al., 2022).
Revised: The full list of concepts can be found in (Abid et al., 2022).

Original: COCO-Stuff (Caesar et al., 2018) is a dataset derived from MS-COCO (Lin et al., 2014), consisting of scenes with various object annotations.
Revised: COCO-Stuff (Caesar et al., 2018) is a dataset derived from MS-COCO (Lin et al., 2014), consisting of scenes with various object annotations.

Original: Previous work (Singh et al., 2020) has shown that there are severe co-occurrence biases in this dataset.
Revised: Previous work (Singh et al., 2020) has shown that there are severe co-occurrence biases in this dataset.

Original: For instance, images from the wine glass often also have a dining table in the image.
Revised: For instance, images from the wine glass often also have a dining table in the image.

Original: Singh et al.
Revised: Singh et al.

Original: (2020) identifies these 20 classes with heavy co-occurence biases and we train PCBMs to recognize these 20 objects, where we minimize the binary cross-entropy loss individually for each class.
Revised: (2020) identifies these 20 classes with heavy co-occurrence biases and we train PCBMs to recognize these 20 objects, where we minimize the binary cross-entropy loss individually for each class.

Original: In this scenario, we again use CLIP-ResNet50 (Radford et al., 2021) as the backbone and BRODEN visual concepts as the set of concepts.
Revised: In this scenario, we again use CLIP-ResNet50 (Radford et al., 2021) as the backbone and BRODEN visual concepts as the set of concepts.

Original: CUB (Wah et al., 2011)
Revised: CUB (Wah et al., 2011)

Original: In the 200-way bird identification task, we use a ResNet18 (He et al., 2016) trained on the CUB dataset 1 .
Revised: In the 200-way bird identification task, we use a ResNet18 (He et al., 2016) trained on the CUB dataset 1 .

Original: We use the same training/validation splits and 112 concepts as in (Koh et al., 2020).
Revised: We use the same training/validation splits and 112 concepts as in (Koh et al., 2020).

Original: These concepts include wing shape , back pattern , and eye color .
Revised: These concepts include wing shape , back pattern , and eye color .

Original: HAM10000 (Tschandl et al., 2018) is a dataset of dermoscopic images, which contain skin lesions from a representative set of diagnostic categories.
Revised: HAM10000 (Tschandl et al., 2018) is a dataset of dermoscopic images, which contain skin lesions from a representative set of diagnostic categories.

Original: The task is detecting whether a skin lesion is benign or malignant.
Revised: The task is detecting whether a skin lesion is benign or malignant.

Original: We use the Inception (Szegedy et al., 2015) model trained on this dataset, which is available from (Daneshjou et al., 2021).
Revised: We use the Inception (Szegedy et al., 2015) model trained on this dataset, which is available from (Daneshjou et al., 2021).

Original: Following the setting in (Lucieri et al., 2020), we collect concepts from the Derm7pt
Revised: Following the setting in (Lucieri et al., 2020), we collect concepts from the Derm7pt

Original: (Kawahara et al., 2018) dataset.
Revised: (Kawahara et al., 2018) dataset.

Original: The 8 concepts obtained from this dataset include Blue Whitish Veil , Pigmented Networks , Regression Structures , which are reportedly associated with the malignancy of a lesion.
Revised: The 8 concepts obtained from this dataset include Blue Whitish Veil , Pigmented Networks , Regression Structures , which are reportedly associated with the malignancy of a lesion.

Original: SIIM-ISIC (Rotemberg et al., 2021)
Revised: SIIM-ISIC (Rotemberg et al., 2021)

Original: To test a real-world transfer learning use case, we evaluate the model trained on HAM10000 on a subset of the SIIM-ISIC Melanoma Classification dataset.
Revised: To test a real-world transfer learning use case, we evaluate the model trained on HAM10000 on a subset of the SIIM-ISIC Melanoma Classification dataset.

Original: We use the same concepts described in the HAM10000 dataset.
Revised: We use the same concepts described in the HAM10000 dataset.

Original: We use linear probing on top of the backbones for CIFAR, COCO, and ISIC to obtain the baseline model performance.
Revised: We use linear probing on top of the backbones for CIFAR, COCO, and ISIC to obtain the baseline model performance.

Original: For CUB and HAM10000, we use the out-of-the-box models trained on the respective datasets without any additional training.
Revised: For CUB and HAM10000, we use the out-of-the-box models trained on the respective datasets without any additional training.

Original: We emphasize that in all of our experiments except CUB, we learn concepts using a held-out dataset.
Revised: We emphasize that in all of our experiments except CUB, we learn concepts using a held-out dataset.

Original: Namely, while the original CBMs need to have concept annotations for training images, we remove this limitation by post-hoc learning of concepts with any dataset.
Revised: Namely, while the original CBMs need to have concept annotations for training images, we remove this limitation by post-hoc learning of concepts with any dataset.

Original: To learn the concept subspace, we use 50 pairs ofpositive and negative images for each concept to train a Linear SVM.
Revised: To learn the concept subspace, we use 50 pairs of images for each concept to train a Linear SVM.

Original: We refer the reader to the Appendix for further details on datasets, hyperparameters and training.
Revised: We refer the reader to the Appendix for further details on datasets and training.

Original: Post-hoc concept bottlenecks do not hurt the full model performance:
Revised: PCBMs achieve comparable performance to the original model:

Original: In Table 1, we report results over these five datasets.
Revised: In Table 1, we report results over these five datasets.

Original: PCBM achieves comparable performance to the original model in all datasets except CIFAR100, and PCBM-h matches the performance in all scenarios.
Revised: PCBM achieves comparable performance to the original model in all datasets except CIFAR100, and PCBM-h matches the performance in all scenarios.

Original: Strikingly, PCBMs match the performance of the original model in HAM10000 and ISIC, with as few ashuman-interpretable concepts .
Revised: Strikingly, PCBMs match the performance of the original model in HAM10000 and ISIC, with as few as 8 human-interpretable concepts .

Original: In CIFAR100, we hypothesize that the concept bank available is not sufficient to classify finer-grained classes, and hence there is a performance gap between the PCBM and the original model.
Revised: In CIFAR100, we hypothesize that the concept bank available is not sufficient to classify finer-grained classes, and hence there is a performance gap between the PCBM and the original model.

Original: When the concept bank is not sufficient to solve the task, PCBM-h can be introduced to recover the original model performance while retaining the benefits of PCBM (see next sections).
Revised: When the concept bank is not sufficient to solve the task, PCBM-h can be introduced to recover the original model performance while retaining the benefits of PCBM (see next sections).

Original: Comparing to CBM: A comparison to CBM was not possible in most datasets, as CBMs cannot be trained due to the lack of dense annotations.
Revised: Comparing to CBM: A comparison to CBM was not possible in most datasets, as CBMs cannot be trained due to the lack of dense annotations.

Original: To compare the data efficiency of both methods, we report a comparison of CBM to PCBM when trained on CUB in Appendix C, finding that CBM can only reach a similar performance to PCBM with 20x more annotations.
Revised: To compare the data efficiency of both methods, we report a comparison of CBM to PCBM when trained on CUB in Appendix C, finding that CBM can only reach a similar performance to PCBM with 20x more annotations.

Original: Analyzing the residual predictor: To understand whether PCBM-h overrides PCBM predictions, in Appendix B, we look at the consistency between PCBM and PCBM-h predictions.
Revised: Analyzing the residual predictor: To understand whether PCBM-h overrides PCBM predictions, in Appendix B, we look at the consistency between PCBM and PCBM-h predictions.

Original: We show that the residual component in PCBM-h intervenes only when the prediction is wrong, and fixes mistakes.
Revised: We show that the residual component in PCBM-h intervenes only when the prediction is wrong, and fixes mistakes.

Original: When 
Revised: When PCBM is confident,

Original: PCBM is confident, PCBM-h does not modify the prediction or significantly increase the confidence.
Revised: PCBM-h does not modify the prediction or significantly increase the confidence.

Original: In general, the residual component may dominate when the bottleneck is insufficient and future work can aim to explicitly limit thecontribution of the residual component, e g. as in the regularizer from PIE (Wang et al., 2021).
Revised: In general, the residual component may dominate when the bottleneck is insufficient and future work can aim to explicitly limit the residual component, e.g. PIE (Wang et al., 2021) regularizer.

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  Here, we show that when labeled examples to learn concepts are not available, we can use multimodal representations such as CLIP to generate concepts without the laborious annotation cost.
Revised: PCBM using CLIP concepts: Here, we show that when labeled examples to learn concepts are not available, we can use multimodal representations such as CLIP to generate concepts without the laborious annotation cost.

Original: Using ConceptNet, we obtain concept descriptions for CIFAR10, CIFAR100, and COCOStuff tasks (206, 527, andconcepts, respectively), using the relations described in Section 2.
Revised: Using ConceptNet, we obtain concept descriptions for CIFAR10, CIFAR100, and COCOStuff tasks (206, 527, andconcepts, respectively), using the relations described in Section 2.

Original: The concept subspace is obtained by using text embeddings for the concept descriptions generated by the ResNet50 variant of CLIP.
Revised: The concept subspace is obtained by using text embeddings for the concept descriptions generated by the ResNet50 variant of CLIP.

Original: In Table 2, we give an overview of the results.
Revised: In Table 2, we give an overview of the results.

Original: We observe that with a more expressive multimodal bottleneck, we can almost recover the original model accuracy in CIFAR10, and we reduce the performance gap in CIFAR100.
Revised: We observe that with a more expressive multimodal bottleneck, we can almost recover the original model accuracy in CIFAR10, and we reduce the performance gap in CIFAR100.

Original: CLIP contains extensive knowledge of natural images, and thus captures natural image concepts (e.g. red , thorns ).
Revised: CLIP contains extensive knowledge of natural images, and thus captures natural image concepts (e.g. red , thorns ).

Original: However, CLIP is less reliable at representing more granular or domain-specific concepts (e.g. blue whitish veils ) and hence this methodology is less applicable to more domain-specific applications.
Revised: However, CLIP is less reliable at representing more granular or domain-specific concepts (e.g. blue whitish veils ) and hence this methodology is less applicable to more domain-specific applications.

Original: While there is a recent trend in training multimodal models for biomedical applications (Zhang et al., 2020; EleutherAI), these models are trained with smaller datasets and are less competitive.
Revised: While there is a recent trend in training multimodal models for biomedical applications (Zhang et al., 2020; EleutherAI), these models are trained with smaller datasets and are less competitive.

Original: We hope that as these models improve, PCBMs will make it easier to deploy concept bottlenecks in more specialized domains.
Revised: We hope that as these models improve, PCBMs will make it easier to deploy concept bottlenecks in more specialized domains.

Original: Explanations in PCBMs: In Figure 2, we provide sample concept weights in the corresponding
Revised: Explanations in PCBMs: In Figure 2, we provide sample concept weights in the corresponding

Original: PCBMs.
Revised: PCBMs.

Original: For instance in HAM10000, PCBMs use Blue Whitish Veils , Atypical Pigment Networks  and Irregular Streaks to identify malignant lesions, whereas Typical Pigment Networks , Regular
Revised: For instance, in HAM10000, PCBMs use Blue Whitish Veils , Atypical Pigment Networks , and Irregular Streaks to identify malignant lesions, whereas Typical Pigment Networks , Regular

Original: Streaks  and Regular Dots and Globules are used to identify benign lesions.
Revised: Streaks , and Regular Dots and Globules are used to identify benign lesions.

Original: These associations are consistent with medical knowledge, which discussed how these characteristics are associated with malignant and benign lesions (Menzies et al., 1996; Lucieri et al., 2020).
Revised: These associations are consistent with medical knowledge, which discussed how these characteristics are associated with malignant and benign lesions (Menzies et al., 1996; Lucieri et al., 2020).

Original: We observe similar cases in
Revised: We observe similar cases in

Original: CIFAR-10 and CIFAR-100, where the class Rose is associated with the concepts Flower , Thorns and Red .
Revised: CIFAR-10 and CIFAR-100, where the class Rose is associated with the concepts Flower , Thorns and Red .

Original: In Appendix Section F, we further analyze the global explanations of COCO-Stuff and show that PCBM reveals co-occurence biases in the training data.
Revised: In Appendix Section F, we further analyze the global explanations of COCO-Stuff and show that PCBM reveals co-occurence biases in the training data.

Original: Note that the concept weights for PCBM-h are exactly the same as for PCBM since the residual  is fit after the concepts weights are fixed.
Revised: Note that the concept weights for PCBM-h are exactly the same as for PCBM since the residual model is fit after the concepts weights are fixed.

Original: Therefore, PCBM-h provides the same concept interpretations as PCBM.
Revised: Yet we note that the model behavior can potentially differ due to the residual component. Further, prior work showed that estimating the causal effect of a concept requires careful treatment (Goyal et al., 2019), e.g. modeling the concept interactions, and we leave this to future work. [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original:  M ODEL EDITING WITH P OST - HOC C ONCEPT B OTTLENECKS
Revised:  M ODEL EDITING WITH P OST - HOC C ONCEPT B OTTLENECKS

Original: When we observe a counter-intuitive phenomenon or a spurious correlation in the concept bottleneck, can we make the model perform better by simple edit operations?
Revised: When we observe a counter-intuitive phenomenon or a spurious correlation in the concept bottleneck, can we make the model perform better by simple edit operations?

Original: In this section, we show that
Revised: In this section, we show that

Original: PCBMs come with the benefit of easy and global concept-level model feedback.
Revised: PCBMs come with the benefit of easy and global concept-level model feedback.

Original: Koh et al.
Revised: Koh et al.

Original: (2020) demonstrate the ability of CBMs to incorporate local interventions.
Revised: (2020) demonstrate the ability of CBMs to incorporate local interventions.

Original: Namely, if a user identifies a misprediction in the concept bottleneck, they can intervene on the concept prediction, and change the prediction for the particular instance, which we call a local intervention.
Revised: Namely, if a user identifies a misprediction in the concept bottleneck, they can intervene on the concept prediction, and change the prediction for the particular instance, which we call a local intervention.

Original: Here, we are interested in editing models to perform ‘global’ edits.
Revised: Here, we are interested in editing models to perform ‘global’ edits.

Original: Namely, with simple concept-level feedback, we would like to edit a PCBM and change the entire model behavior.
Revised: Namely, with simple concept-level feedback, we would like to edit a PCBM and change the entire model behavior.

Original: For instance, in our experiments, we investigate the use case of adapting a PCBM to a new distribution.
Revised: For instance, in our experiments, we investigate the use case of adapting a PCBM to a new distribution.

Original: Unlike most existing model editing approaches, we do not need any data from either the training or target domains to perform the model edit , which can be a significant advantage in practice when data is inaccessible.
Revised: Unlike most existing model editing approaches, we do not need any data from either the training or target domains to perform the model edit , which can be a significant advantage in practice when data is inaccessible.

Original: Given a trained PCBM, we edit our concept bottleneck by manipulating the concept weights.
Revised: Given a trained PCBM, we edit our concept bottleneck by manipulating the concept weights.

Original: For simplicity, in the experiments below we only focus on positive weights in the linear model.
Revised: For simplicity, in the experiments below we only focus on positive weights in the linear model.

Original: 4. C ONTROLLED EXPERIMENT : EDITING PCBM WHEN THE SPURIOUS CONCEPT IS KNOWN
Revised: 4. C ONTROLLED EXPERIMENT : EDITING PCBM WHEN THE SPURIOUS CONCEPT IS KNOWN

Original: For our editing experiments, we use the Metashift (Liang & Zou, 2022) dataset to simulate distribution shifts.
Revised: For our editing experiments, we use the Metashift (Liang & Zou, 2022) dataset to simulate distribution shifts.

Original: We use 10 different scenarios where there is a distribution shift for a particular class between the training and test datasets.
Revised: We use 10 different scenarios where there is a distribution shift for a particular class between the training and test datasets.

Original: For instance, during training, we only use table images that also contain a dog in the image, and test the model with table images where there is not a dog in the image.
Revised: For instance, during training, we only use table images that also contain a dog in the image and test the model with table images where there is not a dog in the image.

Original: We denote the training domain as table(dog) .
Revised: We denote the training domain as table(dog) .

Original: We give more details and the results of all domains in the
Revised: We give more details and the results of all domains in the

Original: Appendix.
Revised: Appendix.

Original: We use an ImageNet pretrained ResNet50 variant as our backbone and the visual concept bank described in the Experiments section.
Revised: We use an ImageNet pretrained ResNet50 variant as our backbone and the visual concept bank described in the Experiments section.

Original: Given a PCBM, we evaluate three editing strategies :
Revised: Given a PCBM, we evaluate three editing strategies :

Original: Prune : We set the weight of a particular concept on a particular class prediction to 0 ,
Revised: Prune : We set the weight of a particular concept on a particular class prediction to 0 ,

Original: i
Revised: i

Original: e. for a concept indexed by i , we let ˜ w i = 0 .
Revised: e. for a concept indexed by i , we let ˜ w i = 0 .

Original: Prune+Normalize :
Revised: Prune+Normalize :

Original: After applying pruning, we rescale the concept weights.
Revised: After applying pruning, we rescale the concept weights.

Original: Let P denote the indices of positive weights that are pruned, ˜ P denote the indices of weights that remain, and w P , w ˜ P be corresponding weight vectors for the particular class.
Revised: Let P denote the indices of positive weights that are pruned, ˜ P denote the indices of weights that remain, and w P , w ˜ P be corresponding weight vectors for the particular class.

Original: We rescale each weight to match the original norm of the vector by letting ∀ i ∈ ˜ P, ˜ w i =
Revised: We rescale each weight to match the original norm of the vector by letting ∀ i ∈ ˜ P, ˜ w i =

Original: w i (1 + || w P
Revised: w i (1+ || w P

Original: || 

Revised: ||

Original: || w ˜ P
Revised: || w ˜ P

Original: || 1 )
Revised: || 1 )

Original: , leading to || ˜
Revised: , leading to || ˜

Original: w
Revised: w

Original: ||
Revised: ||

Original: =
Revised: =

Original: ||
Revised: ||

Original: w
Revised: w

Original: || 1 .
Revised: || 1 .

Original: The normalization step alleviates the imbalance between class weights upon pruning concepts with large weights for a particular class.
Revised: The normalization step alleviates the imbalance between class weights upon pruning concepts with large weights for a particular class.

Original: Fine-tune (Oracle) : We compare our simple editing strategies to fine-tuning on the test domain, which can be considered an oracle.
Revised: Fine-tune (Oracle) : We compare our simple editing strategies to fine-tuning on the test domain, which can be considered an oracle.

Original: Particularly, we fine-tune the PCBM using samples from the test domain, and then test the fine-tuned model with a set of held-out samples.
Revised: Particularly, we fine-tune the PCBM using samples from the test domain and then test the fine-tuned model with a set of held-out samples.

Original: In the context of Metashift experiments, we simply edit the concept spuriously correlated with a particular class.
Revised: In the context of Metashift experiments, we simply edit the concept spuriously correlated with a particular class.

Original: Concretely, for the domain table(dog) , we prune the weight of the dog concept for the class table .
Revised: Concretely, for the domain table(dog) , we prune the weight of the dog concept for the class table .

Original: In Table 3, we report the result of our editing experiments over 10 scenarios.
Revised: In Table 3, we report the result of our editing experiments over 10 scenarios.

Original: We observe that for PCBMs, the Prune+Normalize strategy can recover almost half of the accuracy gains achieved by fine-tuning on the original test domain.
Revised: We observe that for PCBMs, the Prune+Normalize strategy can recover almost half of the accuracy gains achieved by fine-tuning on the original test domain.

Original: Further, we observe that PCBM-h obtains lower accuracy gains with model editing.
Revised: Further, we observe that PCBM-h obtains lower accuracy gains with model editing.

Original: In PCBM, we can remove the concept from the model, but in
Revised: In PCBM, we can remove the concept from the model, but in

Original: PCBM-h, there may still be leftover information about the spurious concept in the residual part of the model.
Revised: PCBM-h, there may still be leftover information about the spurious concept in the residual part of the model.

Original: This is in line with our expectations, i.e. PCBM-h is a ‘less’ interpretable but more powerful variant of PCBM.
Revised: This is in line with our expectations, i.e. PCBM-h is a ‘less’ interpretable but more powerful variant of PCBM.

Original: While it still offers partial editing improvements that would not be possible with the vanilla model, it does not bring as much improvement as the PCBM.
Revised: While it still offers partial editing improvements that would not be possible with the vanilla model, it does not bring as much improvement as the PCBM.

Original: [Table]
Revised: [Table]

Original: Even though our edit strategy is extremely simple, we can recover almost half of the gains made by fine-tuning the model.
Revised: Even though our edit strategy is extremely simple, we can recover almost half of the gains made by fine-tuning the model.

Original: It is particularly easy-to-use since it can be applied without fine-tuning or using any knowledge or data from the target domain.
Revised: It is particularly easy to use since it can be applied without fine-tuning or using any knowledge or data from the target domain.

Original: However, this methodology requiresthe knowledge of the spurious concept in the training domain.
Revised: However, this methodology requires knowledge of the spurious concept in the training domain.

Original: This may not be realistic in practice, and in the next section we turn to human users for guidance
Revised: This may not be realistic in practice, and in the next section, we turn to human users for guidance.

Original: 4. U SER STUDY : EDITING PCBM WITH HUMAN GUIDANCE
Revised: 4. U SER STUDY : EDITING PCBM WITH HUMAN GUIDANCE

Original: One of the advantages to pruning is that it is naturally amenable to bringing humans in the loop.
Revised: One of the advantages of pruning is that it is naturally amenable to bringing humans into the loop.

Original: Rather than attempting to automate the edits, we can rely on a human to pick concepts that make logical sense to prune.
Revised: Rather than attempting to automate the edits, we can rely on a human to pick concepts that make logical sense to prune.

Original: We show that users make fast and insightful pruning decisions that improve model performance on test data when there is a distribution shift.
Revised: We show that users make fast and insightful pruning decisions that improve model performance on test data when there is a distribution shift.

Original: Notably, these decisions can be made even when the user knows little about the model architecture and has no knowledge of the true underlying spurious correlation in the training data.
Revised: Notably, these decisions can be made even when the user knows little about the model architecture and has no knowledge of the true underlying spurious correlation in the training data.

Original: We conduct a user study where we explore the benefits of human-guided pruning on PCBM and PCBM-h performance.
Revised: We conduct a user study where we explore the benefits of human-guided pruning on PCBM and PCBM-h performance.

Original: [Figure]
Revised: [Figure]

Original: We train PCBMs on MetaShift scenarios, each with a distribution shift between the training and test datasets.
Revised: We train PCBMs on MetaShift scenarios, each with a distribution shift between the training and test datasets.

Original: The user selects concepts to prune from the model.
Revised: The user selects concepts to prune from the model.

Original: Study Design:
Revised: Study Design:

Original: We construct 9 MetaShift scenarios with a distribution shift for one of the classes between the training and testing datasets, which we refer to as the ”shifted class”.
Revised: We construct 9 MetaShift scenarios with a distribution shift for one of the classes between the training and testing datasets, which we refer to as the ”shifted class”.

Original: For instance, all of the training images for the class Keyboard also contain a Cat in the image, whereas test images do not have a cat in the image.
Revised: For instance, all of the training images for the class Keyboard also contains a Cat in the image, whereas test images do not have a cat in the image.

Original: A complete list of the 9 scenarios can be found in the Appendix.
Revised: A complete list of the 9 scenarios can be found in the Appendix.

Original: For each scenario, we train a PCBM and a PCBM-h with CLIP concepts on the training dataset, and task the users with making edits to the trained models by selecting concepts to prune.
Revised: For each scenario, we train a PCBM and a PCBM-h with CLIP concepts on the training dataset and task the users with making edits to the trained models by selecting concepts to prune.

Original: We use CLIP’s ResNet50 variant as our backbone, and leverage ConceptNet to construct the concept subspace.
Revised: We use CLIP’s ResNet50 variant as our backbone, and leverage ConceptNet to construct the concept subspace.

Original: Figure 3 shows the concept-selection interface.
Revised: Figure 3 shows the concept-selection interface.

Original: We refer to the
Revised: We refer to the

Original: Appendix for the full set of instructions given to the user.
Revised: Appendix for the full set of instructions given to the user.

Original: The display consists of the classification task the model was trained on and the concepts with the ten most positive weights for the shifted class.
Revised: The display consists of the classification task the model was trained on and the concepts with the ten most positive weights for the shifted class.

Original: The user selects a subset of the concepts to prune.
Revised: The user selects a subset of the concepts to prune.

Original: Neither the original model accuracy nor the accuracy after pruning are revealed to the user until all scenarios are completed.
Revised: Neither the original model accuracy nor the accuracy after pruning is revealed to the user until all scenarios are completed.

Original: The target audience for the study was machine learning practitioners and researchers;volunteers participated for a total of 30 × 9 = 270 experiments.
Revised: The target audience for the study was machine learning practitioners and researchers;volunteers participated for a total of 30 × 9 = 270 experiments.

Original: The
Revised: The

Original: IRB has determined that this does not require review.
Revised: IRB has determined that this does not require review.

Original: Human-guided editing is fast and improves model accuracy:
Revised: Human-guided editing is fast and improves model accuracy:

Original: Even though we do not impose any time limits, participants make fast pruning decisions.
Revised: Even though we do not impose any time limits, participants make fast pruning decisions.

Original: On average, users prune 3 .
Revised: On average, users prune 3 .

Original: 65 ± 0 .
Revised: 65 ± 0 .

Original: 39 concepts in 34 .
Revised: 39 concepts in 34 .

Original: 3 ± 6 .
Revised: 3 ± 6 .

Original: 4 s per scenario.
Revised: 4 s per scenario.

Original: We evaluate the effectiveness of user pruning by comparing the model accuracy after user pruning to the accuracy of the unedited model on a held-out dataset from the test domain.
Revised: We evaluate the effectiveness of user pruning by comparing the model accuracy after user pruning to the accuracy of the unedited model on a held-out dataset from the test domain.

Original: All 30 / 30 users improve the model accuracy when averaged over scenarios.
Revised: All 30 / 30 users improve the model accuracy when averaged over scenarios.

Original: 8 / 30 users improve model accuracy in all scenarios and 26 / 30 users improve model accuracy in at least 6 scenarios.
Revised: 8 / 30 users improve model accuracy in all scenarios and 26 / 30 users improve model accuracy in at least 6 scenarios.

Original: We compare our human-guided editing strategy to three baselines:
Revised: We compare our human-guided editing strategy to three baselines:

Original: Random Pruning : We uniformly select without replacement a subset of the top ten concepts to prune, matching the number of concepts pruned by the user for a fair comparison.
Revised: Random Pruning : We uniformly select without replacement a subset of the top ten concepts to prune, matching the number of concepts pruned by the user for a fair comparison.

Original: Greedy Pruning (Oracle) :
Revised: Greedy Pruning (Oracle) :

Original: We iteratively select one of the top ten concepts that, when pruned, improves model accuracy the most samples from the test domain; again, we match the number of concepts pruned by the user.
Revised: We iteratively select one of the top ten concepts that, when pruned, improves model accuracy the most samples from the test domain; again, we match the number of concepts pruned by the user.

Original: Fine-tune (Oracle) : The model is fine-tuned on samples from the test domain.
Revised: Fine-tune (Oracle) : The model is fine-tuned on samples from the test domain.

Original: [Table]
Revised: [Table]

Original: We consider greedy pruning and retraining to be oracles, since they require ”leaked” knowledge of the test domain.
Revised: We consider greedy pruning and retraining to be oracles since they require ”leaked” knowledge of the test domain.

Original: Table 4 reports the model performance improvements for PCBM and PCBM-h, averaged over all 270 experiments in the user study.
Revised: Table 4 reports the model performance improvements for PCBM and PCBM-h, averaged over all 270 experiments in the user study.

Original: Improvements within each separate scenario can be found in the Appendix.
Revised: Improvements within each separate scenario can be found in the Appendix.

Original: We note that even with the residual component in PCBM-h, we can still improve model performance by editing the concept bottleneck.
Revised: We note that even with the residual component in PCBM-h, we can still improve model performance by editing the concept bottleneck.

Original: We observe that user pruning achieves marked improvement in model performance compared to the unedited model and attains around 50% of the accuracy gains from fine-tuning, 80% of the gains from greedy pruning.
Revised: We observe that user pruning achieves marked improvement in model performance compared to the unedited model and attains around 50% of the accuracy gains from fine-tuning, 80% of the gains from greedy pruning.

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: Concepts Human-understandable concepts draw increasing interest to interpret model behavior (Kim et al., 2018;
Revised: Concepts Human-understandable concepts draw increasing interest to interpret model behavior (Kim et al., 2018;

Original: Bau et al., 2017; 2020).
Revised: Bau et al., 2017; 2020).

Original: Related work focuses on understanding if neural networks encode and use concepts (Lucieri et al., 2020;
Revised: Related work focuses on understanding if neural networks encode and use concepts (Lucieri et al., 2020;

Original: Kim et al., 2018; McGrath et al., 2021), or generate counterfactual explanations to understand model behavior (Ghandeharioun et al., 2021; Abid et al., 2022; Akula et al., 2020).
Revised: Kim et al., 2018; McGrath et al., 2021), or generate counterfactual explanations to understand model behavior (Ghandeharioun et al., 2021; Abid et al., 2022; Akula et al., 2020).

Original: These works mostly use a set of human-specified concepts to analyze model behavior, however, there is an increasing interest in automatically discovering the concepts that are used by a model (Yeh et al., 2020; Ghorbani et al., 2019; Lang et al., 2021).
Revised: Recent works evaluated the causal validity of explanations (Feder et al., 2021; Elazar et al., 2021; Goyal et al., 2019), e.g. to eliminate potential confounding effects. There is further increasing interest in automatically discovering the concepts that are used by a model (Yeh et al., 2020; Ghorbani et al., 2019; Lang et al., 2021).

Original: Concept-based Models: Concept bottleneck models (CBMs) (Koh et al., 2020) extend the earlier idea (Lampert et al., 2009; Kumar et al., 2009) ofdecomposing a task into two parts by first predicting the concepts, then using concepts predicting the target.
Revised: Concept-based Models: Concept bottleneck models (CBMs) (Koh et al., 2020) extend the earlier idea (Lampert et al., 2009; Kumar et al., 2009) of first predicting the concepts, then using concepts to predict the target.

Original: CBMs bring about interpretability and intervenability benefits, while requiring training the model using concept labels for the entire training dataset, which is a key limitation.
Revised: CBMs bring about interpretability benefits while requiring training the model using concept labels for the entire training dataset, which is a key limitation.

Original: Further, CBMs have not been analyzed in terms ofglobal model edits.
Revised: CBMs have not been analyzed in terms of model edits.

Original: Recent workinspects Concept Bottleneck Models (Mahinpei et al., 2021; Margeloiu et al., 2021) and reveals that end-to-end learned concept representations encode information beyond the desired concept, and may not be interpretable.
Revised: Recent work reveals that end-to-end learned CBMs encode information beyond the desired concept (Mahinpei et al., 2021; Margeloiu et al., 2021).

Original: Concept Whitening (Chen et al., 2020) aims to align individual dimensions of a chosen layer with  a set of pre-defined concepts.
Revised: Concept Whitening (Chen et al., 2020) aims to align each concept with an individual dimension in a layer.

Revised: While a subset of dimensions are aligned, this approach lacks a bottleneck since there are dimensions that are potentially not aligned with a concept (comparable to PCBM-h).

Original: IAIA-BL (Barnett et al., 2021) learns prototypes and uses them as concepts, where the distance to prototypes determine the model prediction.
Revised: Barnett et al. (2021) learns prototypes and uses them as concepts, where the distance to prototypes determine the model prediction.

Original: PCBM-h is inspired by semiparametric models on fitting residuals (H ¨ ardle et al., 2004).
Revised: PCBM-h is inspired by semiparametric models on fitting residuals (H ¨ ardle et al., 2004).

Original: PIE (Wang et al., 2021) implements a similar procedure, where they combine individual features(interpretable) with more complicated interaction terms (a black-box model) in the context of tabular data.
Revised: PIE (Wang et al., 2021) has a similar approach, where they combine individual features with more complicated interaction terms for tabular data.

Original: Model Editing Model editing aims to achieve the removal or modification of information in a given neural network.
Revised: Model Editing Model editing aims to achieve the removal or modification of information in a given neural network.

Original: Several models investigated editing factual knowledge in language models; Zhu et al.
Revised: Several models investigated editing factual knowledge in language models; Zhu et al.

Original: (2020) uses variants of fine-tuning to achieve this objective while retaining performance on unmodified factual knowledge, and Mitchell et al.
Revised: (2020) using variants of fine-tuning to achieve this objective while retaining performance on unmodified factual knowledge and Mitchell et al.

Original: De Cao et al.
Revised: De Cao et al.

Original: ; Hase et al.
Revised: ; Hase et al.

Original: (2021) updates the model by training a separate network to modify model parameters achieving the desired edit.
Revised: (2021) updating the model by training a separate network to modify model parameters to achieve the desired edit.

Original: Similarly, Sotoudeh & Thakur (2021) proposes ”repairing” models by finding minimal parameter updates that satisfies a given specification.
Revised: Similarly, Sotoudeh & Thakur (2021) proposes ”repairing” models by finding minimal parameter updates that satisfy a given specification.

Original: One thread of work focuses on intervening on the latent space of neural networks to alter the generated output towards a desired state, e.g. removal of artifacts or manipulation of object positions (Sinitsin et al., 2020;
Revised: One thread of work focuses on intervening on the latent space of neural networks to alter the generated output towards the desired state, e.g. removal of artifacts or manipulation of object positions (Sinitsin et al., 2020;

Original: Bau et al., 2020; Santurkar et al., 2021).
Revised: Bau et al., 2020; Santurkar et al., 2021).

Original: For instance, Bau et al.
Revised: For instance, Bau et al.

Original: (2020) edits generative models, Santurkar et al.
Revised: (2020) edits generative models, Santurkar et al.

Original: (2021) edits classifiers by modifying ‘rules’, such as making a model perceive the concept of a ‘snowy road‘ as the concept of a ‘road’, and they achieve this by modifying minimal updates to specific feature extractors.
Revised: (2021) edits classifiers by modifying ‘rules’, such as making a model perceive the concept of a ‘snowy road‘ as the concept of a ‘road’, and they achieve this by modifying minimal updates to specific feature extractors.

Original: FIND symbolic
Revised: FIND Right -symbolic

Original: (2021) gives an overview of concept-based models and a discussion on debugging strategies while concurrent work ProtoPDebug (Bontempelli et al., 2022) proposes an efficient debugging strategy for prototype-based concept models.
Revised: (2021) give an overview of concept-based models and a discussion on debugging strategies while concurrent work ProtoPDebug (Bontempelli et al. 2022) proposes an efficient debugging strategy for prototype-based concept models.

Original:  L IMITATIONS AND C ONCLUSION
Revised:  L IMITATIONS AND C ONCLUSION

Original: In this work, we presented Post-hoc CBMs as a way of converting any model into a CBM, retaining the original model performance without losing the interpretability benefits.
Revised: In this work, we presented Post-hoc CBMs as a way of converting any model into a CBM, retaining the original model performance without losing the interpretability benefits.

Original: We leveraged multimodal models as an interface to use concepts, without the laborious concept annotation steps.
Revised: We leveraged multimodal models as an interface to use concepts, without the laborious concept annotation steps.

Original: Further, in addition to the local intervention benefits of CBM, we demonstrated that PCBMs can be leveraged to perform global model interventions.
Revised: Further, in addition to the local intervention benefits of CBM, we demonstrated that PCBMs can be leveraged to perform global model interventions.

Original: Many benefits of CBMs depend on the quality of the concept library.
Revised: Many benefits of CBMs depend on the quality of the concept library.

Original: The concept set should be expressive enough to solve the task of interest.
Revised: The concept set should be expressive enough to solve the task of interest.

Original: Users should be careful about the concept dataset we use to learn concepts, which can reflect various biases.
Revised: Users should be careful about the concept dataset we use to learn concepts, which can reflect various biases.

Original: While there are several such real-world tasks, it is an open question if human-constructed concept bottlenecks can solve larger-scale tasks(e.g.
Revised: While there are several such real-world tasks, it is an open question if human-constructed concept bottlenecks can solve larger-scale tasks(e.g.

Original: ImageNet level).
Revised: ImageNet level).

Original: Hence, finding concept subspaces for models in an unsupervised fashion is an active area of research that will help with  usability and expressivity of concept bottlenecks.
Revised: Hence, finding concept subspaces for models in an unsupervised fashion is an active area of research that will help with the usability and expressivity of concept bottlenecks.

Original: Multimodal models provide an effective interface for conceptlevel reasoning, yet it is unclear what is the optimal way to have humans in the loop.
Revised: Multimodal models provide an effective interface for conceptlevel reasoning, yet it is unclear what is the optimal way to have humans in the loop.

Original: Here, we presented a simple way of taking human input via concept pruning, and how to incorporate richer feedback is an interesting direction of future work.
Revised: Here, we presented a simple way of taking human input via concept pruning, and how to incorporate richer feedback is an interesting direction for future work.

Original: E THICAL S TATEMENT
Revised: E THICAL S TATEMENT

Original: In our user study, we did not collect any user information.
Revised: In our user study, we did not collect any user information.

Original: We applied to the IRB within our institution and “IRB has determined that this research does not involve human subjects as defined in 45 CFR 46.102(f) and therefore does not require review by the IRB”.
Revised: We applied to the IRB within our institution and “IRB has determined that this research does not involve human subjects as defined in 45 CFR 46.102(f) and therefore does not require review by the IRB”.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: The code to reproduce all experiments will be released at redacted for anonymity .
Revised: The code to reproduce all experiments will be released at redacted for anonymity .

Original: All of the backbones we used to construct concept bottlenecks are released public checkpoints, and we stated all of these in the manuscript
Revised: All of the backbones we used to construct concept bottlenecks are released public checkpoints, and we stated all of these in the manuscript

Original: A T RAINING D ETAILS
Revised: A T RAINING D ETAILS

Original: Hyperparameters:
Revised: Hyperparameters:

Original: In all of our experiments ElasticNet sparsity ratio parameter was α = 0 . 99 .
Revised: In all of our experiments ElasticNet sparsity ratio parameter was α = 0 . 99 .

Original: We trained all our models on a single NVIDIA-Titan Xp gpu.
Revised: We trained all our models on a single NVIDIA-Titan Xp gpu.

Original: All of the models were trained for a total number of 10 epochs.
Revised: All of the models were trained for a total number of 10 epochs.

Original: We tune the regularization strength on a subset of the training set, that is kept as a validation set.
Revised: We tune the regularization strength on a subset of the training set, that is kept as a validation set.

Original: PCBMs are fitted using scikit-learn’s SGDClassifier class, withmaximum steps.
Revised: PCBMs are fitted using scikit-learn’s SGDClassifier class, withmaximum steps.

Original: Hybrid parts are trained with PyTorch, where we used Adam as the optimizer with 0 .
Revised: Hybrid parts are trained with PyTorch, where we used Adam as the optimizer with 0 .

Original: 01 learning rate, with 0 .
Revised: 01 learning rate, with 0 .

Original: L 2 regularization on the residual classifier weights, and trained forepochs.
Revised: L 2 regularization on the residual classifier weights, and trained forepochs.

Original: Dataset Details:
Revised: Dataset Details:

Original: Metashift: For all metashift experiments, we have 5 class classification problems.
Revised: Metashift: For all metashift experiments, we have 5 class classification problems.

Original: We have 50 images for each class as the training set, and 50 images for each class in the test dataset, which are different from the training images.
Revised: We have 50 images for each class as the training set, and 50 images for each class in the test dataset, which is different from the training images.

Original: The regularization strength for Metashift experiments is 0 .
Revised: The regularization strength for Metashift experiments is 0 .

Original: CIFAR:
Revised: CIFAR:

Original: We use the original training and test splits of CIFAR datasets.
Revised: We use the original training and test splits of CIFAR datasets.

Original: The regularization strength for CIFAR10 and CIFAR100 is 2 .
Revised: The regularization strength for CIFAR10 and CIFAR100 is 2 .

Original: KN c .
Revised: KN c .

Original: We use linear probing to evaluate the original model 3.
Revised: We use linear probing to evaluate the original model 3.

Original: COCO-Stuff: We use the original training and test splits of COCO dataset.
Revised: COCO-Stuff: We use the original training and test splits of COCO dataset.

Original: We sampletraining and 250 test images from the dataset for each class, and we upsample the images whenever there are not enough images.
Revised: We sample 500 training and 250 test images from the dataset for each class, and we upsample the images whenever there are not enough images.

Original: Recognizing each of the 20 biased classes from (Singh et al., 2020) is treated as a binary classification task where we minimize binary cross entropy loss separately for 20 classes, and compute  mean average precision metric.
Revised: Recognizing each of the 20 biased classes itepsingh2020don is treated as a binary classification task where we minimize binary cross entropy loss separately for 20 classes, and compute the mean average precision metric.

Original: The regularization strength for COCO-Stuff is 0 .
Revised: The regularization strength for COCO-Stuff is 0 .

Original: We evaluate the original model performance using linear probes with CLIP.
Revised: We evaluate the original model performance using linear probes with CLIP.

Original: Ham10k:
Revised: Ham10k:

Original: We split 80% of the HAM10k dataset and use it as the training set, and use the remaining 20% as the test data.
Revised: We split 80% of the HAM10k dataset and use it as the training set, and use the remaining 20% as the test data.

Original: The regularization strength for Ham10k is 2 .
Revised: The regularization strength for Ham10k is 2 .

Original: KN c . 5.
Revised: KN c . 5.

Original: ISIC:
Revised: ISIC:

Original: We use 2000 images for training (400 malignant, 1600 benign) and evaluate the model on a held-out set of 500 images (100 malignant, 400 benign).
Revised: We use 2000 images for training (400 malignant, 1600 benign) and evaluate the model on a held-out set of 500 images (100 malignant, 400 benign).

Original: The regularization strength for ISIC is 0 .
Revised: The regularization strength for ISIC is 0 .

Original: 001 KN c .
Revised: 001 KN c .

Original: We evaluate the original model performance using a linear probe.
Revised: We evaluate the original model performance using a linear probe.

Original: CUB:
Revised: CUB:

Original: We use the training and test splits provided in Koh et al.
Revised: We use the training and test splits provided in Koh et al.

Original: (Koh et al., 2020).
Revised: (Koh et al., 2020).

Original: The regularization strength for CIFAR10 and CUB is 0 .
Revised: The regularization strength for CIFAR10 and CUB is 0 .

Original: 01 KN c .
Revised: 01 KN c .

Original: Visual Concepts : We used Broden Visual concept bank for CIFAR and controlled editing experiments,
Revised: Visual Concepts : We used Broden Visual concept bank for CIFAR and controlled editing experiments,

Original: CUB’s training data for CUB concepts, and derm7pt dataset for dermatology concepts.
Revised: CUB’s training data for CUB concepts, and derm7pt dataset for dermatology concepts.

Original: For each of these, we use 50 pairs of positive and negative images, and learn a linear SVM.
Revised: For each of these, we use 50 pairs of positive and negative images, and learn a linear SVM.

Original: We use the vector normal to the decision boundary as the concept vector.
Revised: We use the vector normal to the decision boundary as the concept vector.

Original: Multimodal concepts: For concept learning, we leveraged the ConceptNet hierarchy.
Revised: Multimodal concepts: For concept learning, we leveraged the ConceptNet hierarchy.

Original: For each classification task, we searched concept net for the class name, and obtained concepts that have the following relation with the query concept: hasA , isA , partOf , HasProperty , MadeOf .
Revised: For each classification task, we searched concept net for the class name and obtained concepts that have the following relation with the query concept: hasA , isA , partOf , HasProperty , MadeOf .

Original: We share the code for obtaining natural language concepts using ConceptNet.
Revised: We share the code for obtaining natural language concepts using ConceptNet.

Original: For each of these concepts, we obtain the text embedding using CLIP and use those as the concept vector.
Revised: For each of these concepts, we obtain the text embedding using CLIP and use those as the concept vector.

Original: B R ESIDUAL COMPONENT INTERVENES ONLY WHEN NECESSARY
Revised: B R ESIDUAL COMPONENT INTERVENES ONLY WHEN NECESSARY

Original: Does PCBM-h alter predictions made by the PCBM?
Revised: Does PCBM-h alter predictions made by the PCBM?

Original: We hypothesized that the residual component would intervene only when the concept bottleneck is not sufficient.
Revised: We hypothesized that the residual component would intervene only when the concept bottleneck is not sufficient.

Original: To better understand the effect of residual predictor, we analyzed the prediction consistency between the two models for CIFAR10 and
Revised: To better understand the effect of residual predictor, we analyzed the prediction consistency between the two models for CIFAR10 and

Original: CIFAR100, where we looked at the models with labeled concepts (i.e. 170 concepts).
Revised: CIFAR100, where we looked at the models with labeled concepts (i.e. 170 concepts).

Original: In Figure 4 and 5, x-axis denotes the confidence of PCBM, the orange line gives the accuracy for samples with the given confidence, and the blue line gives the consistency between PCBM and PCBM-h for the same samples, i.e. whether they make the same prediction.
Revised: In Figure 4 and 5, x-axis denotes the confidence of PCBM, the orange line gives the accuracy for samples with the given confidence, and the blue line gives the consistency between PCBM and PCBM-h for the same samples, i.e. whether they make the same prediction.

Original: In CIFAR10 and CIFAR100, we show that
Revised: In CIFAR10 and CIFAR100, we show that

Original: PCBM and PCBM-h consistency is high when the model confidence is high, and the model accuracy is high.
Revised: PCBM and PCBM-h consistency is high when the model confidence is high, and the model accuracy is high.

Original: Consequently, PCBM-h changes the model prediction mostly when the PCBM prediction is likely a mistake, and the confidence is low.
Revised: Consequently, PCBM-h changes the model prediction mostly when the PCBM prediction is likely a mistake, and the confidence is low.

Original: In Figure 5, we show that all of the predictions changed by PCBM-h are to fix model mistakes.
Revised: In Figure 5, we show that all of the predictions changed by PCBM-h are to fix model mistakes.

Original: Further, in Figure 6 we show the PCBM confidence and the mean absolute deviation between the PCBM confidence and the PCBM-h confidence.
Revised: Further, in Figure 6 we show the PCBM confidence and the mean absolute deviation between the PCBM confidence and the PCBM-h confidence.

Original: We observe that PCBM-h hasa less effect of model confidence as the model gets more confident.
Revised: We observe that PCBM-h has less effect on model confidence as the model gets more confident.

Original: [Figure]
Revised: [Figure]

Original: -h predictions i e. whether both models make the prediction). Namely, at confidence level for the PCBM, accuracy and consistency with the PCBM-h predictions Overall, we that PCBM-h is most likely to change the model prediction when the model is making a mistake, and otherwise predictions are consistent.
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Figure] M e t r i c CIFAR
Revised: [Table]

Original: [Figure] c
Revised: [Equation]

Original: CIFAR

Original: [Figure]
Revised: [Figure]

Original: C C OMPARISON TO CBM
Revised: C C OMPARISON TO CBM

Original: In most of our experiments, there is no clear way to run CBM as a benchmark, as none of the datasets have dense concept annotations - except CUB.
Revised: In most of our experiments, there is no clear way to run CBM as a benchmark, as none of the datasets have dense concept annotations - except CUB.

Original: In CUB, we run CBM as a baseline.
Revised: In CUB, we run CBM as a baseline.

Original: Particularly, we try the joint training strategy from Koh et al.
Revised: Particularly, we try the joint training strategy from Koh et al.

Original: (Koh et al., 2020), where it was reported to give the best results in the CUB dataset.
Revised: (Koh et al., 2020), where it was reported to give the best results in the CUB dataset.

Original: To make the comparison equal, we use the same set of concepts in both models, and use a linear predictor layer.
Revised: To make the comparison equal, we use the same set of concepts in both models, and use a linear predictor layer.

Original: Further, in both cases, we use the same frozen ResNet18 backbone.
Revised: Further, in both cases, we use the same frozen ResNet18 backbone.

Original: We searched over a grid of learning rates from { 0 . 01 , 0 . 01 , 0 . 1 , 1 . 0 } and λ ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } where λ is the coefficient of the concept predictors in the joint training objective (see the CBM paper, Section 3), and the ElasticNet regularization strength in { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } .
Revised: We searched over a grid of learning rates from { 0 . 01 , 0 . 01 , 0 . 1 , 1 . 0 } and λ ∈ { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } where λ is the coefficient of the concept predictors in the joint training objective (see the CBM paper, Section 3), and the ElasticNet regularization strength in { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 } .

Original: In Table 5, we report the model performance.
Revised: In Table 5, we report the model performance.

Original: Overall, we observe some benefit from using dense concept annotations over the entire training dataset, where CBMs achieve a slightly better performance than PCBMs, and the original backbone.
Revised: Overall, we observe some benefit from using dense concept annotations over the entire training dataset, where CBMs achieve a [Figure]

Original: We note that this was at the cost of 112 concept annotations for each of the training samples.
Revised: [Figure]

Original: We further analyze the behavior of CBM under different number of annotations.
Revised: We further analyze the behavior of CBM under different number of annotations.

Original: In Figure 7 we train the CBM with varying number of annotations.
Revised: In Figure 7 we train the CBM with varying number of annotations.

Original: CBMs require dense annotations, e.g.annotations would mean 11200 / 112 = 100 images.
Revised: CBMs require dense annotations, e.g.annotations would mean 11200 / 112 = 100 images.

Original: On the x-axis, we give the number of annotations used to train the CBM.
Revised: On the x-axis, we give the number of annotations used to train the CBM.

Original: On the y-axis, we give the accuracy on the test set.
Revised: On the y-axis, we give the accuracy on the test set.

Original: We observe that CBMs require a much larger amount of annotations to achieve the same accuracy as PCBMs.
Revised: We observe that CBMs require a much larger amount of annotations to achieve the same accuracy as PCBMs.

Original: [Table]
Revised: [Table]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: 6 # of Annotations

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Original: D C ONTROLLED M ETASHIFT E XPERIMENTS FOR M ODEL E DITING 
Revised: D C ONTROLLED M ETASHIFT E XPERIMENTS FOR M ODEL E DITING For

Original: For Metashift, we have 2 tasks.
Revised: Metashift, we have 2 tasks.

Original: Both tasks are 5-class object recognition tasks, where in the first one classes are airplane, bed, car, cow, keyboard , and for the second one we have beach, computer, motorcycle, stove, table .
Revised: Both tasks are 5-class object recognition tasks, where in the first one classes are airplane, bed, car, cow, keyboard , and for the second one we have beach, computer, motorcycle, stove, table .

Original: For each of these, we use a ResNet18 pretrained on ImageNet as the backbone of the P-CBM, and then use 100 images per class to train the concept bottleneck.
Revised: For each of these, we use a ResNet18 pretrained on ImageNet as the backbone of the P-CBM, and then use 100 images per class to train the concept bottleneck.

Original: For all experiments, we use the Adam Optimizer with a learning rate of 0 .
Revised: For all experiments, we use the Adam Optimizer with a learning rate of 0 .

Original: 05 , the regularization parameters 1 1 2 0 0 5 0 1 71 8 8 1 3 1 3 6 # of Annotations 0. [Figure] CBM
Revised: 05 , the regularization parameters λ = 0 .

Original: PCBM λ = 0 .

Original: 05 , α = 0 . 99 .
Revised: 05 , α = 0 . 99 .

Original: Similar to CIFAR experiments, we use the Broden Concept dataset.
Revised: Similar to CIFAR experiments, we use the Broden Concept dataset.

Original: Below we give the entire set of results.
Revised: Below we give the entire set of results.

Original: Train

Original: Test

Original: Model

Original: Original

Original: Prune

Original: Prune+Normalize

Original: Fine-Tune bed(dog) bed(cat)

Original: P-CBM 0.

Original: bed(cat) bed(dog)

Original: P-CBM 0. table(dog) table(cat)

Original: P-CBM 0.

Original: table(cat) table(dog)

Original: P-CBM 0.

Original: table(books) table(dog)

Original: P-CBM 0.

Original: table(books) table(cat)

Original: P-CBM 0.

Original: car(dog) car(cat)

Original: P-CBM 0.

Original: car(cat) car(dog)

Original: P-CBM 0.

Original: [Table]

Original: [Table]

Original: P

Original: [Table]

Original: [Table]

Original: [Table]

Original: table

Original: [Table]

Original: [Table]

Original: table

Original: [Table]

Original: table

Original: [Table]

Original: table

Original: [Table]
Revised: [Table]

Original: E U SER S TUDY E. U SER I NTERFACE
Revised: E U SER S TUDY E. U SER I NTERFACE

Original: Below, we provide screenshots of the initial launch page and the concluding summary page from the user study.
Revised: Below, we provide screenshots of the initial launch page and the concluding summary page from the user study.

Original: For the concept-selection interface, refer to the main paper.
Revised: For the concept-selection interface, refer to the main paper.

Original: E. H UMAN - GUIDED PCBM EDITING : D ETAILED PERFORMANCE RESULTS
Revised: E. H UMAN - GUIDED PCBM EDITING : D ETAILED PERFORMANCE RESULTS

Original: In each of the 9 scenarios in the user study [Table] the underlying classification task is a 5-class object recognition task, and one of the classes has a spurious correlation in the training set.
Revised: In each of the 9 scenarios in the user study, the underlying classification task is a 5-class object recognition task and one of the classes has a spurious correlation in the training set.

Original: For instance, all training images for the class keyboard also contain a cat in the image.
Revised: For instance, all training images for the class keyboard also contain a cat in the image.

Original: In Table [Table] we list the classes in the underlying classification task and the spurious correlation for each scenario.
Revised: In Table 7, we list the classes in the underlying classification task and the spurious correlation for each scenario.

Original: [Table]
Revised: [Table]

Original: Below
Revised: Below

Original: [Table]
Revised: [Table]

Original: we report the test accuracies for each of the 9 scenarios in our user study
Revised: we report the test accuracies for each of the 9 scenarios in our user study, both for the shifted class and for the overall classification task.

Original: [Table]
Revised: [Table]

Revised: painting(lamp) 0.

Revised: 0.617 ± 0.

Revised: 0.640 ± 0.

Revised: 0.659 ± 0.

Revised: pillow(clock) 0.

Revised: 0.776 ± 0.

Revised: 0.815 ± 0.

Revised: 0.815 ± 0. television(fireplace) 0.

Revised: 0.485 ± 0.

Revised: 0.535 ± 0.

Revised: 0.564 ± 0. fork(tomato) 0.

Revised: 0.583 ± 0.

Revised: 0.655 ± 0.

Revised: 0.707 ± 0. car(snow) 0.

Revised: 0.676 ± 0.

Revised: 0.807 ± 0.

Revised: 0.817 ± 0.

Revised: Overall Test Accuracy bed(dog) 0.

Revised: 0.850 ± 0.

Revised: 0.882 ± 0.

Revised: 0.882 ± 0.

Revised: keyboard(cat) 0.

Revised: 0.741 ± 0.

Revised: 0.811 ± 0.

Revised: 0.814 ± 0.

Revised: bed(cat) 0.

Revised: 0.808 ± 0.

Revised: 0.834 ± 0.

Revised: 0.841 ± 0. couch(cat) 0.

Revised: 0.839 ± 0.

Revised: 0.851 ± 0.

Revised: 0.858 ± 0.

Revised: painting(lamp) 0.

Revised: 0.841 ± 0.

Revised: 0.845 ± 0.

Revised: 0.854 ± 0.

Original: both for the shifted class and for the overall classification task.
Revised: pillow(clock) 0. 0.882 ± 0. 0.891 ± 0. 0.891 ± 0. television(fireplace) 0.

Revised: 0.817 ± 0.

Revised: 0.827 ± 0.

Revised: [Table]

Original: F A NALYSIS ON THE COCO-S TUFF B IASES
Revised: F A NALYSIS ON THE COCO-S TUFF B IASES

Original: Singh et al (Singh et al., 2020) identifies co-occurence biases in the COCO-Stuff dataset, where 20 categories frequently co-occur with other identified categories.
Revised: Singh et al (Singh et al., 2020) identifies co-occurence biases in the COCO-Stuff dataset, where 20 categories frequently co-occur with other identified categories.

Original: In Table 10, reader can find the concepts, taken from Singh et al.
Revised: In Table 10, reader can find the concepts, taken from Singh et al.

Original: In Table 10, we report the Top-5 concepts for the PCBM trained with CLIP concepts.
Revised: In Table 10, we report the Top-5 concepts for the PCBM trained with CLIP concepts.

Original: In the Biased Context column, we give the category that co-occurs frequently with the given context.
Revised: In the Biased Context column, we give the category that co-occurs frequently with the given context.

Original: In PCBM Top-5 Concepts column, we give the concepts that have the highestweight for the given category.
Revised: In PCBM Top-5 Concepts column, we give the concepts that have the highest

Original: Looking at PCBM categories, we can see that we can identify the biased contexts.

Original: For instance, cup category has table, dining table , apple has fruit, fresh vegetables , car has traffic circle, intersection in the concepts identified as important, which are parallel to the

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: keyboard(cat)

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: bed

Original: [Table]

Original: [Table]

Original: [Table]

Original: couch

Original: [Table]
Revised: [Table]

Original: painting(lamp)

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: pillow(clock) [Table] [Table] [Table]
Revised: weight for the given category.

Original: [Table]

Original: [Table]

Original: [Table]

Original: television(fireplace)

Original: [Table] [Table] [Table] [Table]
Revised: Looking at PCBM categories, we can see that we can identify the biased contexts.

Original: [Table]

Original: fork(tomato) [Table] [Table] [Table]
Revised: For instance, cup category has table, dining table , apple has fruit, fresh vegetables , car has traffic circle, intersection in the concepts identified as important, which are parallel to the biased context.

Original: [Table]

Original: [Table]

Original: car(snow)

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table] [Table] biased context. In other cases, our model surfaced different potential co-occurence biases, such as chimney for clock , and toilet for hair drier .
Revised: In other cases, our model surfaced different potential co-occurence biases, such as chimney for clock , and toilet for hair drier .

Original: One important limitation for this analysis is that the desired concepts should exist in the concept bank.
Revised: One important limitation for this analysis is that the desired concepts should exist in the concept bank.

Original: To further improve the pipeline, automatic concept discovery approaches shall be a fruitful research direction (Ghorbani et al., 2019;
Revised: To further improve the pipeline, automatic concept discovery approaches shall be a fruitful research direction (Ghorbani et al., 2019;

Original: Yeh et al., 2020).
Revised: Yeh et al., 2020).

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: keyboard(cat) 0.

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: painting(lamp) 0.

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: (fireplace

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: (tomato)

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]