Original: Real world relevance of generative counterfactual explanations
Revised: Real world relevance of generative counterfactual explanations

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: The interpretability of deep learning based algorithms is critical in settings wherethe algorithm must provide actionable information such as clinical diagnoses orinstructions in autonomous driving.
Revised: The interpretability of deep learning based algorithms is critical in settings where the algorithm must provide actionable information such as clinical diagnoses or instructions in autonomous driving.

Original: Image based explanations or feature attributions are an often-proposed solution for natural imaging datasets, but their utilityfor mission critical settings is unclear.
Revised: Image based explanations or feature attributions are an often-proposed solution for natural imaging datasets, but their utility for mission critical settings is unclear.

Original: In this work, we provide image explanations that are both semantically interpretable and assess their utility for real worldrelevance using imaging data extracted from clinical settings.
Revised: In this work, we provide image explanations that are both semantically interpretable and assess their utility for real world relevance using imaging data extracted from clinical settings.

Original: We address theproblem of pneumonia classification from Chest X-ray images where we show that(1) by perturbing specific latent dimensions of a GAN based model, the classifierpredictions can be flipped and (2) the latent factors have clinical relevance.
Revised: We address the problem of pneumonia classification from Chest X-ray images where we show that (1) by perturbing specific latent dimensions of a GAN based model, the classifier predictions can be flipped and (2) the latent factors have clinical relevance.

Original: Wedemonstrate the latter by performing a case study with a board-certified radiologistand identify some latent factors that are clinically informative and others that maycapture spurious correlations
Revised: We demonstrate the latter by performing a case study with a board-certified radiologist and identify some latent factors that are clinically informative and others that may capture spurious correlations.

Original:  Introduction
Revised:  Introduction

Original: Imagine your doctor orders a CT scan, and it indicates an abnormality.
Revised: Imagine your doctor orders a CT scan and it indicates an abnormality.

Original: Naturally, you would askyour doctor "Why?".
Revised: Naturally, you would ask your doctor "Why?".

Original: An effective explanation should be both spatially localized to the abnormalitydiscovered by the CT scan and be easily interpreted by a non-expert.
Revised: An effective explanation should be both spatially localized to the abnormality discovered by the CT scan and be easily interpreted by a non-expert.

Original: Regardless of who examinesthe scan, be it an algorithm or a doctor, a successful explanation must satisfy these criteria to betrustworthy [Lipton, 2018].
Revised: Regardless of who examines the scan, be it an algorithm or a doctor, a successful explanation must satisfy these criteria to be trustworthy [Lipton, 2018].

Original: Effectively explaining complex predictions is critical for follow-up tasks,like deciding how to treat your discovered abnormality.
Revised: Effectively explaining complex predictions is critical for follow-up tasks, like deciding how to treat your discovered abnormality.

Original: However, while explanations are crucial toreal-world impact, existing ML systems remain largely opaque.
Revised: However, while explanations are crucial to real-world impact, existing ML systems remain largely opaque.

Original: To continue expanding ML systemsinto impactful domains, we need explanation systems that aid real world practitioners [Doshi-Velezand Kim, 2017].
Revised: To continue expanding ML systems into impactful domains, we need explanation systems that aid real world practitioners [Doshi-Velez and Kim, 2017].

Original: Explainable machine learning is a rapidly-growing research area [Arrieta et al., 2020].
Revised: Explainable machine learning is a rapidly-growing research area [Arrieta et al., 2020].

Original: A recentand promising direction is to explain model predictions using counterfactuals learned by generativemodels [Wachter et al., 2017, Singla et al., 2019, 2021, Lang et al., 2021].
Revised: A recent and promising direction is to explain model predictions using counterfactuals learned by generative models [Wachter et al., 2017, Singla et al., 2019, 2021, Lang et al., 2021].

Original: However, while recentworks appear to provide meaningful explanations of model predictions, there has been limited workdemonstrating whether experts actually find the explanations helpful in practice [Van Calster et al.
Revised: However, while recent works appear to provide meaningful explanations of model predictions, there has been limited work demonstrating whether experts actually find the explanations helpful in practice [Van Calster et al., 2019].

Original: We take a first step towards stress-testing generative models in real, expert-driven applications.
Revised: We take a first step towards stress-testing generative models in real, expert-driven applications.

Original: Our approach is outlined in Figure 1 for explaining an off-the-shelf pneumonia classifier.
Revised: Our approach is outlined in Figure 1 for explaining an off-the-shelf pneumonia classifier.

Original: We start bytraining a generative model on the domain of interest.
Revised: We start by training a generative model on the domain of interest.

Original: In order to explain the prediction of a classifieron a real image, we project the image into the latent space of the generative model.
Revised: In order to explain the prediction of a classifier on a real image, we project the image into the latent space of the generative model.

Original: We find thatperturbing specific latent dimensions induces a localized change in the image which in turn affectsthe classifier prediction.
Revised: We find that perturbing specific latent dimensions induces a localized change in the image which in turn affects the classifier prediction.

Original: We conduct a clinical review with a domain expert to obtain the real worldrelevance of the chosen latent dimensions.
Revised: We conduct a clinical review with a domain expert to obtain the real world relevance of the chosen latent dimensions.

Original: [Figure]
Revised: [Figure]

Original: In summary, our contribution is two-fold:
Revised: In summary, our contribution is two-fold:

Original: • We introduce an expert-in-the-loop explanation method that audits the predictions of apre-trained classifier using counterfactuals derived from a StyleGAN.
Revised: • We introduce an expert-in-the-loop explanation method that audits the predictions of a pre-trained classifier using counterfactuals derived from a StyleGAN.

Original: Our approach allowsus to identify semantically-meaningful factors that have large impacts on the classifier’spredictions.•
Revised: Our approach allows us to identify semantically-meaningful factors that have large impacts on the classifier’s predictions.

Original:  We then showcase our approach by auditing a Pneumonia classifier trained on Chest X-Rayimages.
Revised: • We then showcase our approach by auditing a Pneumonia classifier trained on Chest X-Ray images.

Original: By collaborating with a radiologist, we ultimately find that our method successfullyidentifies clinically-meaningful factors that convince the model to flip its predictions.
Revised: By collaborating with a radiologist, we ultimately find that our method successfully identifies clinically-meaningful factors that convince the model to flip its predictions.

Original:  Method
Revised:  Method

Original: The objective of our approach to explain the decision of an off-the-shelf classifier that was trainedto predict a pathology from an image inputs.
Revised: The objective of our approach to explain the decision of an off-the-shelf classifier that was trained to predict a pathology from an image inputs.

Original: Specifically, let C : X → y represent the pre-trainedclassifier that takes an image X  as input and predicts the relevant pathology probability y , wherey ∈ [0 , 1] .
Revised: Specifically, let C : X → y represent the pre-trained classifier that takes an image X ∈ R H × W as input and predicts the relevant pathology probability y , where y ∈ [0 , 1] .

Original: The classifier is calibrated such that 0.5 is the decision threshold for a positive prediction.
Revised: The classifier is calibrated such that 0.5 is the decision threshold for a positive prediction.

Original: In order to obtain attributes that meaningfully affect the decision of C , we need a generative model ofthe image manifold that captures the set of factors that can potentially affect the classifier.
Revised: In order to obtain attributes that meaningfully affect the decision of C , we need a generative model of the image manifold that captures the set of factors that can potentially affect the classifier.

Original: For thispurpose, we train a StyleGAN-2 Karras et al.
Revised: For this purpose, we train a StyleGAN-2 Karras et al.

Original: [2020] based generator that has been shown to containdisentangled attributes in its style-based latent space Wu et al.
Revised: [2020] based generator that has been shown to contain disentangled attributes in its style-based latent space Wu et al.

Original: Given a pretrained generative model, one can synthesize fake images by randomly sampling fromthe latent space of the generator.
Revised: Given a pretrained generative model, one can synthesize fake images by randomly sampling from the latent space of the generator.

Original: Thus the true latent vector for a fake image is known.
Revised: Thus the true latent vector for a fake image is known.

Original: However,for a real image X , the true latent vector is unknown.
Revised: However, for a real image X , the true latent vector is unknown.

Original: Finding the latent vector for a real image orinverting the GAN is an active area of research Xia et al.
Revised: Finding the latent vector for a real image or inverting the GAN is an active area of research Xia et al.

Original: Existing methods either use anoptimization based procedure to backproject a given instance into the latent space of a pretrainedgenerative model or train an encoder model whose output is the estimate of the true latent vector.
Revised: Existing methods either use an optimization based procedure to backproject a given instance into the latent space of a pretrained generative model or train an encoder model whose output is the estimate of the true latent vector.

Original: Inthis work, we invert the GAN using backprojection as it does not require external training data and itis better suited for imposing domain specific priors.
Revised: In this work, we invert the GAN using backprojection as it does not require external training data and it is better suited for imposing domain specific priors.

Original: 2.1 Obtaining latent vectors for real images
Revised: 2.1 Obtaining latent vectors for real images

Original: Given an input image X , the backprojection procedure involves performing gradient descent overthe space of the latent vectors optimizing a loss function that encodes a combination of pixel-wise contraints and domain specific constraints.
Revised: Given an input image X , the backprojection procedure involves performing gradient descent over the space of the latent vectors optimizing a loss function that encodes a combination of pixel-wise constraints and domain specific constraints.

Original: Specifically:
Revised: Specifically:

Original: [Equation]
Revised: [Equation]

Original: The contents of the loss function L are as follows:
Revised: The contents of the loss function L are as follows:

Original: • Mean square loss ( L mse ): Enforces similarity at the pixel level.
Revised: • Mean square loss ( L mse ): Enforces similarity at the pixel level.

Original: • Perceptual loss ( L pips ): Enforces perceptual similarity over features across multiple scales.• Domain specific loss ( L cls ): Enforces domain similarity by utilizing a pretrained classifiertrained on the same domain as X .
Revised: • Perceptual loss ( L pips ): Enforces perceptual similarity over features across multiple scales. • Domain specific loss ( L cls ): Enforces domain similarity by utilizing a pretrained classifier trained on the same domain as X .

Original: • Discriminator loss ( L disc ): Adversarial loss term using the pretrained discriminator thatensures that the resultant image is as realistic as possible.
Revised: • Discriminator loss ( L disc ): Adversarial loss term using the pretrained discriminator that ensures that the resultant image is as realistic as possible.

Original: In summary, the full loss function being optimized is:
Revised: In summary, the full loss function being optimized is:

Original: [Equation]
Revised: [Equation]

Original: Note that, each loss term is applied directly on the pixel space and not on the latent space.
Revised: Note that, each loss term is applied directly on the pixel space and not on the latent space.

Original: During theoptimization procedure, we start from initializing z to the mean latent vector computed over 100kimages randomly sampled from G .
Revised: During the optimization procedure, we start from initializing z to the mean latent vector computed over 100k images randomly sampled from G .

Original: Similar to previous works that perform backprojection, we findthat this offers increased stability to the inversion process.
Revised: Similar to previous works that perform backprojection, we find that this offers increased stability to the inversion process.

Original: For each image, 500 gradient descent stepsare performed to obtain the estimated latent vector.
Revised: For each image, 500 gradient descent steps are performed to obtain the estimated latent vector.

Original: While the backprojection procedure described in Eq. 2 can be used for inverting any GAN basedgenerator, specific details for a styleGAN based generator need to be accounted for.
Revised: While the backprojection procedure described in Eq. 2 can be used for inverting any GAN based generator, specific details for a styleGAN based generator need to be accounted for.

Original: As explainedin the StyleSpace work [Wu et al., 2021], different latent spaces of styleGAN offer tradeoffs forinversion fidelity against editability.
Revised: As explained in the StyleSpace work [Wu et al., 2021], different latent spaces of styleGAN offer tradeoffs for inversion fidelity against editability.

Original: Following their suggestion, we invert the latent style space, as itoffers a better trade-off for inversion and editability.
Revised: Following their suggestion, we invert the latent style space, as it offers a better trade-off for inversion and editability.

Original: This is an important concern for our approach,since we need to ensure that the resultant latent vector: (1) offers high fidelity for inversion, so thatwe do not miss any critical detail in medical images and (2) remains in distribution for the generatoreven after applying a perturbation.
Revised: This is an important concern for our approach, since we need to ensure that the resultant latent vector: (1) offers high fidelity for inversion, so that we do not miss any critical detail in medical images and (2) remains in distribution for the generator even after applying a perturbation.

Original: 2.2 Identifying dimensions that change classifier’s decision
Revised: 2 Identifying dimensions that change classifier’s decision

Original: In the previous section, we outlined the approach for inverting the image X to the GAN latent space.
Revised: In the previous section, we outlined the approach for inverting the image X to the GAN latent space.

Original: This operation can be represented as a projection function: P : X →S D s.t G ( SD ) ∼ = 
Revised: This operation can be represented as a projection function: P : X → s.t G ( S ) ∼ = X , where S ∈ R D is the D-dimensional latent vector, commonly referred to as style vector Wu et al.

Original: X with theoptimization function defined in Eq 2.
Revised: The optimization function defined in Eq 2.

Original: In this section, we describe our approach to identify specificlatent dimensions that when perturbed, affect the prediction of an underlying classifier, that is trainedusing data from  X .
Revised: In this section, we describe our approach to identify specific latent dimensions that when perturbed, affect the prediction of an underlying classifier, that is trained using data from domain X .

Original: For each X and latent dimension d , we start from the classifier score on the original input and obtainthe following set of perturbed classifier scores and the corresponding deltas:

Original: [Equation] where X { Sd = k } =
Revised: Given an image X , let S =

Original: G ([ s 1 , s 2 , ..., s d − 1 , k, s d +1 , ..
Revised: [ s 1 , s 2 , ..., s d − 1 , s d , s d +1 , ..

Original: s D ]) i.e. the original style vector with the value forthe
Revised: s D ] be the latent style vector that generated

Original: d th replaced by k .
Revised: X .

Original: In order to quantify the effect of perturbing this dimension on  the classifier, weevaluate the classifier with perturbed inputs corresponding to the extreme values for each dimension{ S dmin , S dmax } .
Revised: Let’s denote the classifier score for the original image as C ( X ; S ) making the dependence on S explicit. To assess the effectiveness of the latent dimension d , we perturb the style value at dimension d by clamping it to maximum and minimum values and record the change.

Original: We estimate the minimum and maximum range for each dimension of the latentvector over a set of randomly sampled images from the latent space of the GAN generator. 
Revised: We estimate the minimum and maximum values for each dimension of the latent vector over a set of randomly sampled images from the latent space of the GAN generator. Let S d

Revised: + denote the perturbed style vector such that the d th dimension is replaced by the maximum: S d + =

Revised: [ s 1 , s 2 , ..., s d − 1 , s maxd , s d +1 , ..

Original: Then, weclamp each dimension of S d separately and independently fromothers to the min and max values forthat corresponding dimension.
Revised: s D ] and corresponding for S d − . Starting from the classifier score on the original input, we obtain the following set of perturbed classifier scores and the corresponding difference scores, represented by △ : [Equation] [Equation]

Revised: [Equation]

Revised: In order to quantify the effect of perturbing this dimension on the classifier, we evaluate the classifier with perturbed inputs corresponding to the extreme values for each dimension { s mind , s maxd } .

Revised: In this procedure, we clamp each dimension of S separately and independently from others to the min and max values for that corresponding dimension.

Original: We aggregate the classifier deltas across a dataset of heldout images and select dimensions whosedelta values across the entire set exceed a preset value δ , which we treat as a hyperparameter.
Revised: We aggregate the classifier deltas across a dataset of heldout images and select dimensions whose delta values across the entire set exceed a preset value δ , which we treat as a hyperparameter.

Original:  Case study: what factors affect an off-the-shelf pathology classifier?
Revised:  Case study: what factors affect an off-the-shelf pathology classifier?

Original: We perform a small-scale case study whose objective is to identify the latent factors that can explainthe predictions made by an off-the-shelf pathology classifier.
Revised: We perform a small-scale case study whose objective is to identify the latent factors that can explain the predictions made by an off-the-shelf pathology classifier.

Original: Specifically, we choose the problem ofpneumonia classification from Chest X-ray (CXR) images and use an off-the-shelf classifier that hasbeen shown to be highly performant on this problem [Cohen et al., 2020].
Revised: Specifically, we choose the problem of pneumonia classification from Chest X-ray (CXR) images and use an off-the-shelf classifier that has been shown to be highly performant on this problem [Cohen et al., 2020].

Original: The classifier was trainedon a combination of several CXR datasets [Wang et al., 2017, Johnson et al., 2019, Irvin et al., 2019]using a multi-label training objective consisting of 18 pathologies.
Revised: The classifier was trained on a combination of several CXR datasets [Wang et al., 2017, Johnson et al., 2019, Irvin et al., 2019] using a multi-label training objective consisting of 18 pathologies.

Original: We only perturb the classifierscore for pneumonia in our experiments.
Revised: We only perturb the classifier score for pneumonia in our experiments.

Original: The clinical annotations used in this work were provided bya board-certified radiologist, who is also a co-author of this work.
Revised: The clinical annotations used in this work were provided by a board-certified radiologist, who is also a co-author of this work.

Original: 3.1 Experimental Setup
Revised: 3.1 Experimental Setup

Original: Dataset and Pretraining For our experiments, we use the publicly available CheXpert dataset [Irvinet al., 2019] that consists of 200k Chest X-ray (CXR) images.
Revised: Dataset and Pretraining For our experiments, we use the publicly available CheXpert dataset [Irvin et al., 2019] that consists of 200k Chest X-ray (CXR) images.

Original: To train the StyleGAN model, we filterthe dataset to contain only frontal AP scans.
Revised: To train the StyleGAN model, we filter the dataset to contain only frontal AP scans.

Original: We split the dataset into training and test splits with thetraining split containing 80% of the images.
Revised: We split the dataset into training and test splits with the training split containing 80% of the images.

Original: The generative model is trained for a total of 120 epochs(or 20k kimgs), early stopping the model based on FID and Precision-Recall scores.
Revised: The generative model is trained for a total of 120 epochs (or 20k kimgs), early stopping the model based on FID and Precision-Recall scores.

Original: Latent Factor Selection We use our approach described in Section2.2 to select the latent factorsthat affect the predictions of our pneumonia classifier.
Revised: Latent Factor Selection We use our approach described in Section2.2 to select the latent factors that affect the predictions of our pneumonia classifier.

Original: We perform the selection on the held outtest split that consist of 2000 images.
Revised: We perform the selection on the held out test split that consist of 2000 images.

Original: We use a classifier delta threshold of δ = 0 . 3 .
Revised: We use a classifier delta threshold of δ = 0 . 3 .

Original: As a result ofour selection process, we end up with a total of 10 dimensions (out of 7424 latent dimensions) thatchange the classifier score beyond δ .
Revised: As a result of our selection process, we end up with a total of 10 dimensions (out of 7424 latent dimensions) that change the classifier score beyond δ .

Original: Setup for Clinical Review For each identified latent factor, we sort the held out set in the decreasingorder of effected change and choose the top 5 images where the magnitude of the change is the largest.
Revised: Setup for Clinical Review For each identified latent factor, we sort the held out set in the decreasing order of effected change and choose the top 5 images where the magnitude of the change is the largest.

Original: This results in a set of 50 images over 10 chosen latent factors.
Revised: This results in a set of 50 images over 10 chosen latent factors.

Original: In order to minimize any bias due to adomain shift, we only use those image where the classifier is extremely confident in its predictioni.e. images where the classifier predicts the correct label with a confidence of 0.9 or more.
Revised: In order to minimize any bias due to a domain shift, we only use those image where the classifier is extremely confident in its prediction i.e. images where the classifier predicts the correct label with a confidence of 0.9 or more.

Original: We alsoensure to pick images where the GAN reconstruction is very accurate and does not remove any detailin the clinically relevant spatial regions.
Revised: We also ensure to pick images where the GAN reconstruction is very accurate and does not remove any detail in the clinically relevant spatial regions.

Original: 3.2 Identifying clinically relevant factors
Revised: 3.2 Identifying clinically relevant factors

Original: [Figure]

Original: We conduct a small-scale clinical review by an clinical radiologist in order to semantically annotatethe latent factors that change the classifier’s predictions.
Revised: We conduct a small-scale clinical review by an clinical radiologist in order to semantically annotate the latent factors that change the classifier’s predictions.

Original: During the review, the expert is shown theinput images and the modified images resulting from the perturbation process.
Revised: During the review, the expert is shown the input images and the modified images resulting from the perturbation process.

Original: A sample input isshown in Figure 2.
Revised: A sample input is shown in Figure 2.

Original: For each image, we include the pneumonia probability output of the off-the-shelfclassifier that we are attempting to explain.
Revised: For each image, we include the pneumonia probability output of the off-the-shelf classifier that we are attempting to explain.

Original: They are also presented with a gif with the modifiedimage stacked on top of the unmodified image in order to better visualize the difference.
Revised: They are also presented with a gif with the modified image stacked on top of the unmodified image in order to better visualize the difference.

Original: They arethen presented with the following queries for each input:
Revised: They are then presented with the following queries for each input:

Original: • Is the change semantically meaningful?
Revised: • Is the change semantically meaningful?

Original: If yes, please provide a clinical description of this change.
Revised: If yes, please provide a clinical description of this change.

Original: • Is the change relevant to the pathology under consideration i.e. pneumonia?
Revised: • Is the change relevant to the pathology under consideration i.e. pneumonia?

Original: The clinically relevant latent factors identified as a result of the review procedure are demonstratedin Figure 3.
Revised: The clinically relevant latent factors identified as a result of the review procedure are demonstrated in Figure 3.

Original: The review data including the responses of the expert are presented in the Appendix 5.
Revised: The review data including the responses of the expert are presented in the Appendix 5.

Original: [Figure] From  limited number of  (50) provided to  factors, they wereable whose  induced clinically  in the  and increased the classifier score for pneumonia.
Revised: From the limited number of images (50) provided to our expert across 10 latent factors, they were able to ascertain 4 factors whose perturbations induced clinically meaningful changes in the image and increased the classifier score for pneumonia.

Original: In addition, out of the 50 instances, 23 were foundto contain perturbations that were relevant to the pathology under consideration i.e. pneumonia.
Revised: In addition, out of the 50 instances, 23 were found to contain perturbations that were relevant to the pathology under consideration i.e. pneumonia.

Original: In addition to being clinically relevant, these factors are able to explain the behavior of the classifier.
Revised: In addition to being clinically relevant, these factors are able to explain the behavior of the classifier.

Original: As Figure 3 demonstrates, the classifier score is susceptible to these identified factors such asincreased opacity in the right medial base or increased density in the right heart border etc.
Revised: As Figure 3 demonstrates, the classifier score is susceptible to these identified factors such as increased opacity in the right medial base or increased density in the right heart border etc.

Original: From thepoint of view of critical applications such as healthcare, our explanation provides an interactive wayto explain the classifier behavior as opposed to more passive explanation provided by approachesbased on feature attributions or spatial relevance alone.
Revised: From the point of view of critical applications such as healthcare, our explanation provides an interactive way to explain the classifier behavior as opposed to more passive explanation provided by approaches based on feature attributions or spatial relevance alone.

Original: It is worth noting that the generative model was provided no knowledge of the classifier or anyother pathology based prior during the training procedure.
Revised: It is worth noting that the generative model was provided no knowledge of the classifier or any other pathology based prior during the training procedure.

Original: The pneumonia classifier was used as adomain based prior only when performing the image reconstruction and the latent factor selectionsteps (Section 2.2).
Revised: The pneumonia classifier was used as a domain based prior only when performing the image reconstruction and the latent factor selection steps (Section 2.2).

Original: The existence of clincially relevant latent dimensions in the latent space of apretrained StyleGAN generative model implies that these dimensions might be generalizable to otherdownstream tasks within the same domain as well. 
Revised: The existence of clinically relevant latent dimensions in the latent space of a pretrained StyleGAN generative model implies that these dimensions might be generalizable to other downstream tasks within the same domain as well. 3.3 Caveats of our approach

Revised: Are all discovered latent dimensions useful?

Original: While some dimensions we discovered did lead to relevant behaviors there were others that wereharder to interpret.
Revised: While some dimensions we discovered did lead to relevant behaviors there were others that were harder to interpret.

Original: Out of the 10 dimensions that were reviewed by our expert, 4 of them presentedwith pathology related changes.
Revised: Out of the 10 dimensions that were reviewed by our expert, 4 of them presented with pathology related changes.

Original: For most examples where any of the other 6 dimensions wereperturbed, the changes were more global – perhaps related to spurious correlations like image qualityrather than pathology-related image features.
Revised: For most examples where any of the other 6 dimensions were perturbed, the changes were more global – perhaps related to spurious correlations like image quality rather than pathology-related image features.

Original: A promising direction for future work would be toexplore interesting behavior among the latent style dimensions that are possible spurious correlations.
Revised: A promising direction for future work would be to explore interesting behavior among the latent style dimensions that are possible spurious correlations.

Revised: Should generator know about the classifier?

Revised: The knowledge of the classifier was provided during the reconstruction stage, not during the training of the generator – the objective is to make the generator become a domain expert.

Original: Related work 
Revised: Infusing knowledge of a specific pathology at the training stage would require training multiple generators for different pathologies. This knowledge is provided this knowledge implicitly when we use the generator to reconstruct a real x-ray image via a classifier based loss. We have made this distinction more explicit. Comparisons against other approaches This work showcases a set of promising initial results making a case for counterfactual explanations.

Revised: As future work, there needs to be a broader study comparing the clinical relevance of counterfactuals (vs) a popular explanation approach, such as salience maps.

Revised: Our experiments would follow previous such studies for saliency maps Saporta et al.

Original: The popular approaches used to explain image classifiers are based on spatial attributions such asheatmaps or salience maps [Selvaraju et al., 2017, Rebuffi et al., 2020] or feature attributions Xu et al.[2020].
Revised: The popular approaches used to explain image classifiers are based on spatial attributions such as heatmaps or saliency maps [Selvaraju et al., 2017, Rebuffi et al., 2020] or feature attributions Xu et al.

Original: Such explanations highlight the regions of the image that are helpful for classifier predictionor visualize concepts modeled by neurons in deep networks.
Revised: Such explanations highlight the regions of the image that are helpful for classifier prediction or visualize concepts modeled by neurons in deep networks.

Original: Counterfactual explanations Wachteret al.
Revised: Counterfactual explanations Wachter et al.

Original: [2017] have emerged as a promising explanation mechanism for back box image classificationmodels especially in cases where the attributes that are used by the classifier are not distinct objectspresent in the image.
Revised: [2017] have emerged as a promising explanation mechanism for back box image classification models especially in cases where the attributes that are used by the classifier are not distinct objects present in the image.

Original: Counterfactual explanations work by providing an alternative set of inputswhere only some attributes are changed while the others are held constant, thus helping isolatehow a classifier is affected by a specific set of attributes.
Revised: Counterfactual explanations work by providing an alternative set of inputs where only some attributes are changed while the others are held constant, thus helping isolate [Figure] how a classifier is affected by a specific set of attributes.

Original: Recent approaches for image basedcounterfactuals [Singla et al., 2019, 2021] have extensively used generative models due to theirability to provide a disentangled latent space Wu et al.
Revised: Recent approaches for image based counterfactuals [Singla et al., 2019, 2021] have extensively used generative models due to their ability to provide a disentangled latent space Wu et al.

Original: [2021] where each attribute can be controlledseparately.
Revised: [2021] where each attribute can be controlled separately.

Original: The work closest to our proposed approach is StylEx
Revised: The work closest to our proposed approach is StylEx

Original: Lang et al.
Revised: Lang et al.

Original: [2021], which obtainsstyle based explanations from the latent space of a pretrained generative model.
Revised: [2021], which obtains style based explanations from the latent space of a pretrained generative model.

Original: The key differencein our work is the application of a gnenerative model based explanation approach to a critical datadomain such as healthcare including an expert-in-the-loop review.
Revised: The key difference in our work is the application of a generative model based explanation approach to a critical data domain such as healthcare including an expert-in-the-loop review.

Original: While StylEx can be appliedsuccessfully to domains where semantic attributes are easily describable, such as faces, animals etc,real world applications require more transparency and interpretability, which is the key contributionof our framework.
Revised: While StylEx can be applied successfully to domains where semantic attributes are easily describable, such as faces, animals etc, real world applications require more transparency and interpretability, which is the key contribution of our framework.

Original:  Conclusion and Future work
Revised:  Conclusion and Future work

Original: Machine learning models are notorious for being black boxes and the larger such models are, theharder it becomes to understand their decisions.
Revised: Machine learning models are notorious for being black boxes and the larger such models are, the harder it becomes to understand their decisions.

Original: Despite being an active research area, the findingsdo not account for diverse expertise of the end users.

Original: In this work, we take a first step towardsattributing real world meaning explaining classifier decisions.
Revised: In this work, we take a first step towards attributing real world meaning for classifier explanations.

Original: We demonstrated our approachon a high impact problem of pneumonia risk computation from Chest X-ray images.
Revised: We demonstrated our approach on a high impact problem of pneumonia risk computation from Chest X-ray images.

Original: We foundsemantically meaningful dimensions within the latent space of a GAN model.
Revised: We found semantically meaningful dimensions within the latent space of a GAN model.

Original: We showed that(1) perturbing those dimensions can create counterfactual inputs that flip the pneumonia classifierdecision and (2) the latent dimensions have clinical relevance by performing a review with a clinicalexpert.
Revised: We showed that (1) perturbing those dimensions can create counterfactual inputs that flip the pneumonia classifier decision and (2) the latent dimensions have clinical relevance by performing a review with a clinical expert.

Original: While our initial results are promising, there are several avenues for improvement includingrobustness of our latent factor selection procedure, improving our GAN model based reconstructionof real images and performing a more rigorous large scale case study with multiple clinical expertsacross different pathologies.
Revised: While our initial results are promising, there are several avenues for improvement including robustness of our latent factor selection procedure, improving our GAN model based reconstruction of real images and performing a more rigorous large scale case study with multiple clinical experts across different pathologies.

Original: We leave these as avenues for future work.
Revised: We leave these as avenues for future work.

Original: AppendixA Data from Clinical Review
Revised: Appendix A Data from Clinical Review

Original: The radiologist was presented with 50 instances with each instance consisting of the original, reconstructed and perturbed images stacked side by side as shown in Figure 2.
Revised: The radiologist was presented with 50 instances with each instance consisting of the original, reconstructed and perturbed images stacked side by side as shown in Figure 2.

Original: We summarized our findings from this review in Section 3.2.
Revised: We summarized our findings from this review in Section 3.2.

Original: Here, we present the response from our expert for the clinicallyrelevant dimensions:
Revised: Here, we present the response from our expert for the clinically relevant dimensions:

Original: [Table]
Revised: [Table]

Revised: possible

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: subtle but possible

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: no

Revised: [Table]

Revised: [Table]

Revised: can be,

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: yes

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: no

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: yes, but probably wouldn’t diagnose

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: yes, but probably wouldn’t diagnose

Revised: [Table]

Revised: yes, but would probably not even notice

Revised: [Table]

Revised: not in isolation without opacity as well

Revised: [Table]

Revised: relevant but not enough alone yes

Revised: [Table]

Revised: no