Original: T HINKING LIKE A MACHINE — GENERATING VISUAL RATIONALES THROUGH LATENT SPACE OPTIMIZATION
Revised: T HINKING LIKE A MACHINE — GENERATING VISUAL RATIONALES THROUGH LATENT SPACE OPTIMIZATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Interpretability and small labelled datasets are key issues in the practical application of deep learning, particularly in areas such as medicine.
Revised: Interpretability and small labelled datasets are key issues in the practical application of deep learning, particularly in areas such as medicine.

Original: In this paper, we present a semi-supervised technique that addresses both these issues simultaneously.
Revised: In this paper, we present a semi-supervised technique that addresses both these issues simultaneously.

Original: We learn dense representations from large unlabelled image datasets, then use those representations to both learn classiﬁers from small labeled sets and generate visual rationales explaining the predictions.
Revised: We learn dense representations from large unlabelled image datasets, then use those representations to both learn classiﬁers from small labeled sets and generate visual rationales explaining the predictions.

Original: Using chest radiography diagnosis as a motivating application we show our method has good generalization ability by learning to represent our chest radiography dataset while training a classiﬁer on an separate set from a different institution.
Revised: Using chest radiography diagnosis as a motivating application we show our method has good generalization ability by learning to represent our chest radiography dataset while training a classiﬁer on an separate set from a different institution.

Original: Our method identiﬁes heart failure and other thoracic diseases.
Revised: Our method identiﬁes heart failure and other thoracic diseases.

Original: For each prediction, we generate visual rationales for positive classiﬁcations by optimizing a latent representation to minimize the probability of disease while constrained by a similarity measure in image space.
Revised: For each prediction, we generate visual rationales for positive classiﬁcations by optimizing a latent representation to minimize the probability of disease while constrained by a similarity measure in image space.

Original: Decoding the resultant latent representation produces an image without apparent disease.
Revised: Decoding the resultant latent representation produces an image without apparent disease.

Original: The difference between the original and the altered image forms an interpretable visual rationale for the algorithm’s prediction.
Revised: The difference between the original and the altered image forms an interpretable visual rationale for the algorithm’s prediction.

Original: Our method simultaneously produces visual rationales that compare favourably to previous techniques and a classiﬁer that outperforms the current state-of-the-art.
Revised: Our method simultaneously produces visual rationales that compare favourably to previous techniques and a classiﬁer that outperforms the current state-of-the-art.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep learning as applied to medicine has attracted much interest in recent years as a potential solution to many difﬁcult problems in medicine, such as the recognition of diseases on pathology slides or radiology images.
Revised: Deep learning as applied to medicine has attracted much interest in recent years as a potential solution to many difﬁcult problems in medicine, such as the recognition of diseases on pathology slides or radiology images.

Original: However, adoption of machine learning algorithms in ﬁelds such as medicine relies on the end user being able to understand and trust the algorithm, as incorrect implementation and errors may have signiﬁcant consequences.
Revised: However, adoption of machine learning algorithms in ﬁelds such as medicine relies on the end user being able to understand and trust the algorithm, as incorrect implementation and errors may have signiﬁcant consequences.

Original: Hence, there has recently been much interest in interpretability in machine learning as this is a key aspect of implementing machine learning algorithms in practice.
Revised: Hence, there has recently been much interest in interpretability in machine learning as this is a key aspect of implementing machine learning algorithms in practice.

Original: We propose a novel method of creating visual rationales to help explain individual predictions and explore a speciﬁc application to classifying chest radiographs.
Revised: We propose a novel method of creating visual rationales to help explain individual predictions and explore a speciﬁc application to classifying chest radiographs.

Original: There are several well-known techniques in the literature for generating visual heatmaps.
Revised: There are several well-known techniques in the literature for generating visual heatmaps.

Original: Gradient based methods were ﬁrst proposed in 2013 described as a saliency map in Simonyan et al.
Revised: Gradient based methods were ﬁrst proposed in 2013 described as a saliency map in Simonyan et al.

Original: (2013), where the derivative of the ﬁnal class predictions is computed with respect to the input pixels, generating a map of which pixels are considered important.
Revised: (2013), where the derivative of the ﬁnal class predictions is computed with respect to the input pixels, generating a map of which pixels are considered important.

Original: However, these saliency maps are often unintelligible as convolutional neural networks tend to be sensitive to almost imperceptible changes in pixel intensities, as demonstrated by recent work in adversarial examples.
Revised: However, these saliency maps are often unintelligible as convolutional neural networks tend to be sensitive to almost imperceptible changes in pixel intensities, as demonstrated by recent work in adversarial examples.

Original: In fact, obtaining the saliency map is often the ﬁrst step in generating adversarial examples as in Goodfellow et al.
Revised: In fact, obtaining the saliency map is often the ﬁrst step in generating adversarial examples as in Goodfellow et al.

Original: Other recent developments in gradient based methods such as Integrated Gradients from Sundararajan et al.
Revised: Other recent developments in gradient based methods such as Integrated Gradients from Sundararajan et al.

Original: (2017) have introduced fundamental axioms, including the idea of sensitivity which helps focus gradients on relevant features.
Revised: (2017) have introduced fundamental axioms, including the idea of sensitivity which helps focus gradients on relevant features.

Original: Occlusion sensitivity proposed by Zeiler & Fergus (2013) is another method which covers parts of the image with a grey box, mapping the resultant change in prediction.
Revised: Occlusion sensitivity proposed by Zeiler & Fergus (2013) is another method which covers parts of the image with a grey box, mapping the resultant change in prediction.

Original: This produces a heatmap where features important to the ﬁnal prediction are highlighted as they are occluded.
Revised: This produces a heatmap where features important to the ﬁnal prediction are highlighted as they are occluded.

Original: Another wellknown method of generating visual heatmaps is global average pooling.
Revised: Another wellknown method of generating visual heatmaps is global average pooling.

Original: Using fully convolutional neural networks with a global average pooling layer as described in Zhou et al.
Revised: Using fully convolutional neural networks with a global average pooling layer as described in Zhou et al.

Original: (2015), we can examine the class activation map for the ﬁnal convolutional output prior to pooling, providing a low resolution heatmap for activations pertinent to that class.
Revised: (2016), we can examine the class activation map for the ﬁnal convolutional output prior to pooling, providing a low resolution heatmap for activations pertinent to that class.

Original: A novel analysis method by Ribeiro et al.
Revised: A novel analysis method by Ribeiro et al.

Original: (2016) known as locally interpretable model-agnostic explanations (LIME) attempts to explain individual predictions by simulating model predictions in the local neighbourhood around this example.
Revised: (2016) known as locally interpretable model-agnostic explanations (LIME) attempts to explain individual predictions by simulating model predictions in the local neighbourhood around this example.

Original: Gradient based methods and occlusion sensitivity can also be viewed in this light — attempting to explain each classiﬁcation by changing individual input pixels or occluding square areas.
Revised: Gradient based methods and occlusion sensitivity can also be viewed in this light — attempting to explain each classiﬁcation by changing individual input pixels or occluding square areas.

Original: However, sampling the neighbourhood surrounding an example in raw feature space can often be tricky, especially for image data.
Revised: However, sampling the neighbourhood surrounding an example in raw feature space can often be tricky, especially for image data.

Original: Image data is extremely complex and high-dimensional — hence real examples are sparsely distributed in pixel space.
Revised: Image data is extremely complex and high-dimensional — hence real examples are sparsely distributed in pixel space.

Original: Sampling randomly in all directions around pixel space is likely to produce non-realistic images.
Revised: Sampling randomly in all directions around pixel space is likely to produce non-realistic images.

Original: LIME’s solution to this is to use superpixel based algorithms to oversegment images, and to perturb the image by replacing each superpixel by its average value, or a ﬁxed pre-determined value.
Revised: LIME’s solution to this is to use superpixel based algorithms to oversegment images, and to perturb the image by replacing each superpixel by its average value, or a ﬁxed pre-determined value.

Original: While this produces more plausible looking images as opposed to occlusion or changing individual pixels, it is still sensitive to the parameters and the type of oversegmentation used — as features larger than a superpixel and differences in global statistics may not be represented in the set of perturbed images.
Revised: While this produces more plausible looking images as opposed to occlusion or changing individual pixels, it is still sensitive to the parameters and the type of oversegmentation used — as features larger than a superpixel and differences in global statistics may not be represented in the set of perturbed images.

Original: This difﬁculty in producing high resolution visual rationales using existing techniques motivates our current research.
Revised: This difﬁculty in producing high resolution visual rationales using existing techniques motivates our current research.

Original:  M ETHODS
Revised:  M ETHODS

Original: We introduce a novel method utilizing recent developments in generative adversarial networks (GANs) to generate high resolution visual rationales.
Revised: We introduce a novel method utilizing recent developments in generative adversarial networks (GANs) to generate high resolution visual rationales.

Original: We demonstrate the use of this method on a large dataset of frontal chest radiographs by training a classiﬁer to recognize heart failure on chest radiographs, a common task for doctors.
Revised: We demonstrate the use of this method on a large dataset of frontal chest radiographs by training a classiﬁer to recognize heart failure on chest radiographs, a common task for doctors.

Original: Our method comprises of three main steps — we ﬁrst use generative adversarial networks to train a generator on an unlabelled dataset.
Revised: Our method comprises of three main steps — we ﬁrst use generative adversarial networks to train a generator on an unlabelled dataset.

Original: Secondly, we use the trained generator as the decoder section of an autoencoder.
Revised: Secondly, we use the trained generator as the decoder section of an autoencoder.

Original: This enables us to encode and decode, to and from the latent space while still producing high resolution images.
Revised: This enables us to encode and decode, to and from the latent space while still producing high resolution images.

Original: Lastly, we train simple supervised classiﬁers on the encoded representations of a smaller, labelled dataset.
Revised: Lastly, we train simple supervised classiﬁers on the encoded representations of a smaller, labelled dataset.

Original: We optimize over the latent space surrounding each encoded instance with the objective of changing the instance’s predicted class while penalizing differences in the resultant decoded image and the original reconstructed image.
Revised: We optimize over the latent space surrounding each encoded instance with the objective of changing the instance’s predicted class while penalizing differences in the resultant decoded image and the original reconstructed image.

Original: This enables us to visualize what that instance would appear as if it belonged in a different class.
Revised: This enables us to visualize what that instance would appear as if it belonged in a different class.

Original: Firstly, we use the Wasserstein GAN formulation by Arjovsky et al.
Revised: Firstly, we use the Wasserstein GAN formulation by Arjovsky et al.

Original: (2017) and ﬁnd that the addition of the gradient penalty term helps to stabilize training as introduced by Gulrajani et al.
Revised: (2017) and ﬁnd that the addition of the gradient penalty term helps to stabilize training as introduced by Gulrajani et al.

Original: Our unlabelled dataset comprises of a set of 98,900 chest radiograph images, which are scaled to 128 by 128 pixels while maintaining their original aspect ratio through letterboxing, and then randomly translated by up to 8 pixels.
Revised: Our unlabelled dataset comprises of a set of 98,900 chest radiograph images, which are scaled to 128 by 128 pixels while maintaining their original aspect ratio through letterboxing, and then randomly translated by up to 8 pixels.

Original: We use a 100 dimensional latent space.
Revised: We use a 100 dimensional latent space.

Original: Our discriminator and generator both use the DCGAN architecture while excluding the batch normalization layers and using Scaled Exponential Linear Units described in Klambauer et al.
Revised: Our discriminator and generator both use the DCGAN architecture while excluding the batch normalization layers and using Scaled Exponential Linear Units described in Klambauer et al.

Original: (2017) as activations except for the ﬁnal layer of the generator which utilized a Tanh layer.
Revised: (2017) as activations except for the ﬁnal layer of the generator which utilized a Tanh layer.

Original: We train the critic for 4 steps for each generator training step.
Revised: We train the critic for 4 steps for each generator training step.

Original: The GAN training process was run for 200k generator iterations before visually acceptable generated images were produced.
Revised: The GAN training process was run for 200k generator iterations before visually acceptable generated images were produced.

Original: ADAM was used as the optimizer with the generator and discriminator learning rates both set to 5 x 10 -5 .
Revised: ADAM was used as the optimizer with the generator and discriminator learning rates both set to 5 x 10 -5 .

Original: In the next step, we use the trained generator as the decoder for the autoencoder.
Revised: In the next step, we use the trained generator as the decoder for the autoencoder.

Original: We ﬁx the weights of the decoder during training and train our autoencoder to reproduce each of the images from the unlabelled dataset.
Revised: We ﬁx the weights of the decoder during training and train our autoencoder to reproduce each of the images from the unlabelled dataset.

Original: The unlabelled dataset was split by patient in a 15 to 1 ratio into a training and validation set.
Revised: The unlabelled dataset was split by patient in a 15 to 1 ratio into a training and validation set.

Original: We minimize the Laplacian loss between the input and the output, inspired by
Revised: We minimize the Laplacian loss between the input and the output, inspired by

Original: Bojanowski et al.
Revised: Bojanowski et al.

Original: Minimal overﬁtting was observed during the training process even when the autoencoder was trained for over 1000 epochs, the reconstruction loss on the validation set was similar to the reconstruction loss on the validation set.
Revised: Minimal overﬁtting was observed during the training process even when the autoencoder was trained for over 1000 epochs, as demonstrated in 2.

Original: We then train a classiﬁer on a smaller labelled dataset consisting of 7,391 chest radiograph images paired with a B-type natriuretic peptide (BNP) blood test that is correlated with heart failure.
Revised: We then train a classiﬁer on a smaller labelled dataset consisting of 7,391 chest radiograph images paired with a B-type natriuretic peptide (BNP) blood test that is correlated with heart failure.

Original: This test is measured in nanograms per litre, and higher readings indicate heart failure.
Revised: This test is measured in nanograms per litre, and higher readings indicate heart failure.

Revised: This is a task of real-world medical interest as BNP test readings are not often available immediately and offered at all laboratories.

Revised: Furthermore, the reading of chest radiograph images can be complex, as suggested by the widely varying levels of accuracy amongst doctors of different seniority levels reported by

Original:  We perform a natural logarithm on the actual  value and divide the resultant number by 10 to scale these readings to between 0 and 1.
Revised: Kennedy et al. We perform a natural logarithm on the actual BNP value and divide the resultant number by 10 to scale these readings to between 0 and 1.

Revised: This task can be viewed as either a regression or classiﬁcation task, as a cut-off value is often chosen as a diagnostic threshold.

Revised: In this paper, we train our network to predict the actual BNP value but evaluate its AUC at the threshold of 100ng/L.

Revised: We choose AUC at this threshold as this is the cut-off suggested by Lokuge et al.

Revised: (2009), and AUC is a widely used metric of comparison in the medical literature.

Original: We augment each labelled image and encode it into the latent space using our previously trained autoencoder.
Revised: We augment each labelled image and encode it into the latent space using our previously trained autoencoder.

Original: To prevent contamination, we separate our images by patient into a training and testing set with a ratio of 4 to 1 prior to augmentation and encoding.
Revised: To prevent contamination, we separate our images by patient into a training and testing set with a ratio of 4 to 1 prior to augmentation and encoding.

Original: We demonstrate the success of simple classiﬁers upon this latent representation, including a 2 layer multilayer perceptron  as well as a linear regressor.
Revised: We demonstrate the success of simple classiﬁers upon this latent representation, including a 2 layer multilayer perceptron with 256 hidden units as well as a linear regressor.

Original: To obtain image speciﬁc rationales, we optimize over the latent space starting with the latent representation of the given example.
Revised: To obtain image speciﬁc rationales, we optimize over the latent space starting with the latent representation of the given example.

Original: We ﬁx the weights of the entire model and apply the ADAM optimizer on a composite objective comprising of the output value of the original predicted class and a linearly weighted mean squared error term between the decoded latent representation and the decoded original representation.
Revised: We ﬁx the weights of the entire model and apply the ADAM optimizer on a composite objective comprising of the output value of the original predicted class and a linearly weighted mean squared error term between the decoded latent representation and the decoded original representation.

Original: We cap the maximum number of iterations at 5000 and set our learning rate at 0.1.
Revised: We cap the maximum number of iterations at 5000 and set our learning rate at 0.1.

Original: We stop the iteration process early if the cutoff value for that class is achieved.
Revised: We stop the iteration process early if the cutoff value for that class is achieved.

Original: The full algorithm is described in Algorithm 1.
Revised: The full algorithm is described in Algorithm 1.

Original: This generates a latent representation with a different prediction from the initial representation.
Revised: This generates a latent representation with a different prediction from the initial representation.

Original: The difference between the decoded generated representation and the decoded original representation is scaled and overlaid over the original image to create the visual rationale for that image.
Revised: The difference between the decoded generated representation and the decoded original representation is scaled and overlaid over the original image to create the visual rationale for that image.

Original: We use gradient descent to optimize the following objective:
Revised: We use gradient descent to optimize the following objective:

Original: [Equation]
Revised: [Equation]

Original: Where X is the reconstructed input image (having been passed through the autoencoder); X target and z target are the output image and its latent representation.
Revised: Where X is the reconstructed input image (having been passed through the autoencoder); X target and z target are the output image and its latent representation.

Original: G is our trained generator neural network.
Revised: G is our trained generator neural network.

Original: α is a coefﬁcient that trades-off the classiﬁcation and reconstruction objectives.
Revised: γ is a coefﬁcient that trades-off the classiﬁcation and reconstruction objectives.

Original: L target is a target objective which can be anegative class probability or in the case of heart failure, predicted
Revised: L target is a target objective which can be a class probability or a regression target.

Original: BNP level.

Original: The critical difference between our objective and the one used for adversarial example generation is that optimization is performed in the latent space, not the image space.
Revised: The critical difference between our objective and the one used for adversarial example generation is that optimization is performed in the latent space, not the image space.

Original: Require: α , learning rate γ , image similarity penalty ρ , cutoff value
Revised: Require: α , learning rate γ , image similarity penalty ρ , cutoff value

Original: Require: x , the initial input f : x → z , a function approximating the mapping between image and latent space g :z → x h( z ) , classiﬁer predicting value from z1: z 0 ← z ← f ( x ) 2: repeat3: d ← (cid:104) ( g ( z ) − g ( z 0 )) 2 (cid:105) 4: y ← h ( z ) 5: z ← z + α ∗ ADAM ( z, y + γd ) 6: until y > ρ 7: return g ( z 0 ) − g ( z )
Revised: Require: x , the initial input f : x → z , a function approximating the mapping between image and latent space g : x h classiﬁer predicting value from z repeat

Original: We also apply our method to external datasets and demostrate good cross-dataset generalization, in particular the National Institutes of Health (NIH) ChestX-ray8 dataset  recently released by Wang et al.
Revised: We also apply our method to external datasets and demostrate good cross-dataset generalization, in particular the National Institutes of Health (NIH) ChestX-ray8 dataset comprising of 108,948 frontal chest radiographs, recently released by Wang et al.

Original: We downsize the provided images to work with our autoencoder and split this by patient into a training, validation and testing set in the 7:1:2 ratio used by the dataset’s authors.
Revised: We downsize the provided images to work with our autoencoder and split this by patient into a training, validation and testing set in the 7:1:2 ratio used by the dataset’s authors.

Original: We encode these images into the latent space and apply a 6 layer fully connected neural network with 100 nodes in each layer utilizing residual connections.
Revised: We encode these images into the latent space and apply a 6 layer fully connected neural network with 100 hidden units in each layer utilizing residual connections.

Revised: We train this with a batch size of 2048.

Original: This architecture is fully described in ﬁgure 1.
Revised: This architecture is fully described in ﬁgure 1.

Original: To evaluate the usefulness of the generated visual rationales, we conduct an experiment where we compare visual rationales generated by a classiﬁer to one which is contaminated.
Revised: To evaluate the usefulness of the generated visual rationales, we conduct an experiment where we compare visual rationales generated by a classiﬁer to one which is contaminated.

Original: We train the classiﬁer directly on the testing examples and over train until almost perfect accuracy on this set is achieved.
Revised: We train the classiﬁer directly on the testing examples and over train until almost perfect accuracy on this set is achieved.

Original: We reason that the contaminated classiﬁer will simply memorize the testing examples and hence will not be able to produce useful rationales.
Revised: We reason that the contaminated classiﬁer will simply memorize the testing examples and hence will not be able to produce useful rationales.

Original: We also apply our method to the well known MNIST dataset and apply a linear classiﬁer with away softmax.
Revised: We also apply our method to the well known MNIST dataset and apply a linear classiﬁer with away softmax.

Original: In order to generate our visual rationales we select an initial class and a target class — we have chosen to transform the digit 9 to the digit 4 as these bear physical resemblance.
Revised: In order to generate our visual rationales we select an initial class and a target class — we have chosen to transform the digit 9 to the digit 4 as these bear physical resemblance.

Original: We alter our optimization objective by adding a negatively weighted term for the predicted probability of the target class as described in Algorithm 2.
Revised: We alter our optimization objective by adding a negatively weighted term for the predicted probability of the target class as described in Algorithm 2.

Original: Algorithm2 Visual rationale generation for multiclass predictors
Revised: Algorithm

Original: Require: α , learning rate γ , image similarity penalty ρ , cutoff value β , target class weighting t , target class
Revised: Require: α , learning rate γ , image similarity penalty ρ , cutoff value β , target class weighting t , target class

Original: Require: x , the initial input f : x → z , a function approximating the mapping between image and latent space g :
Revised: Require: x , the initial input f : x → z , a function approximating the mapping between image and latent space g :

Original: z → x h c  , classiﬁer predicting class probability from z 
Revised: z → x h c ( z ) → P ( c | z ) , classiﬁer predicting class probability from z 1: z 0 ← z ← f ( x ) 2: m ← argmin

Original: : repeat 
Revised: i h i ( z 0 ) 3: repeat 4:

Original:  ← 
Revised: d ← (cid:104) ( g ( z ) − g ( z 0 )) 2 (cid:105) 5: y m ← h m ( z ) 6: y t ← h t ( z ) 7:

Original: 7: z ← z + α  γd  8: until y m >
Revised: z ← z + α ∗ ADAM ( z, y m − βy t + γd ) 8: until y m >

Original:  9: return g 
Revised: ρ 9: return g ( z 0 ) − g

Original: − g
Revised: ( z )

Original:  R ESULTS
Revised:  R ESULTS

Original: To illustrate the ﬁdelity of our autoencoder we reconstruct each image in a smaller labelled set which has not been seen during training.
Revised: To illustrate the ﬁdelity of our autoencoder we reconstruct each image in a smaller labelled set which has not been seen during training.

Original: The reconstructed images are show in Fig.
Revised: The reconstructed images are show in Fig.

Original: These images are obtained by simply encoding the input image into the latent representation and subsequently decoding this representation again.
Revised: These images are obtained by simply encoding the input image into the latent representation and subsequently decoding this representation again.

Revised: MSE loss and the Laplacian loss functions are shown in Fig 2.

Original: In the heart failure classiﬁcation task, we threshold the known BNP values at 100ng/L to get binary labels as suggested by Lokuge et al.
Revised: In the heart failure classiﬁcation task, we threshold the known BNP values at 100ng/L to get binary labels as suggested by Lokuge et al.

Original: Our semi-supervised model achieves an AUC of 0.837 using a linear regressor as our ﬁnal classiﬁer with an AUC curve as shown in Fig 3.
Revised: Our semi-supervised model achieves an AUC of 0.837 using a linear regressor as our ﬁnal classiﬁer with an ROC curve as shown in Fig 4.

Original: This is comparable to the AUC obtained by a multilayer perceptron.
Revised: This is comparable to the AUC obtained by a multilayer perceptron.

Original: In Fig 4 we demonstrate an example of the algorithm’s reconstruction of a chest radiograph from a patient with heart failure, as well as the visualization of the same patient’s chest radiograph without heart failure.
Revised: In Fig 5 we demonstrate an example of the algorithm’s reconstruction of a chest radiograph from a patient with heart failure, as well as the visualization of the same patient’s chest radiograph without heart failure.

Original: We subtract the visualization of the radiograph without heart failure from the original reconstructed image and superimpose this as a heatmap on the original image to demonstrate the visual rationale for this prediction.
Revised: We subtract the visualization of the radiograph without heart failure from the original reconstructed image and superimpose this as a heatmap on the original image to demonstrate the visual rationale for this prediction.

Original: For the same image, we apply the saliency map method, integrated gradients, the occlusion sensitivity method with a window size of 8, as well as LIME to obtain Fig.
Revised: For the same image, we apply the saliency map method, integrated gradients, the occlusion sensitivity method with a window size of 8, as well as LIME to obtain Fig.

Original: 5 for comparison.
Revised: 6 for comparison.

Revised: All of these methods yield noisy and potentially irrelevant features as compared to our method of generating visual rationales.

Original: We apply our classiﬁer as described above to the chest radiograph dataset released by the NIH recently and achieve results similar to or exceeding that of the baseline results reported in the original dataset.
Revised: We apply our classiﬁer as described above to the chest radiograph dataset released by the NIH recently and achieve results similar to or exceeding that of the baseline results reported in the original dataset.

Original: ROC curves are demonstrated in Fig 6.
Revised: ROC curves are demonstrated in Fig 7.

Original: Comparison AUC results are reported in Table 1.
Revised: Comparison AUC results are reported in Table 1.

Original: We show that even without repeating the autoencoder or GAN training process on the new dataset, we are able to classify encoded representations of these chest radiographs with an accuracy comparable to or exceeding the performance of the published baseline network, which utilizes various state of the art network architectures as well as higher resolution images
Revised: We show that even without repeating the autoencoder or GAN training process on the new dataset, we are able to classify encoded representations of these chest radiographs with an accuracy comparable to or exceeding the performance of the published baseline network, which utilizes various state of the art network architectures as well as higher resolution images.

Original: [Table]
Revised: [Table]

Original: We apply our method to the MNIST dataset and demonstrate class switching between digits 9 and 4.
Revised: We apply our method to the MNIST dataset and demonstrate class switching between digits fromto 4 and 3 to 2.

Original: Figure 7.
Revised: Figure 8.

Original: demonstrates the visual rationales for why each digit has been classiﬁed as a 9 rather than a 4, as well as the transformed versions of each digit.
Revised: demonstrates the visual rationales for why each digit has been classiﬁed as a 9 rather than a 4, as well as the transformed versions of each digit.

Original: As expected, the top horizontal line in the digit 9 is removed to make each digit appear as a 4.
Revised: As expected, the top horizontal line in the digit 9 is removed to make each digit appear as a 4.

Original: Interestingly, the algorithm failed to convert several digits into a 4 and instead converts them into other digits which are presumably more similar to that instance, despite the addition of the weighted term encouraging the latent representation to prefer the target class.
Revised: Interestingly, the algorithm failed to convert several digits into a 4 and instead converts them into other digits which are presumably more similar to that instance, despite the addition of the weighted term encouraging the latent representation to prefer the target class.

Revised: This type of failure is observed more in digits that are less similar to each other, such as from converting from the digits 3 to 2, as simply removing the lower curve of the digit may not always result in a centered ”two” digit.

Revised: This precludes the simple interpretation that we are able to attribute to the 9 to 4 task.

Original: This behaviour is not noted in our chest radiograph dataset as we are able to convert every image from the predicted class to the converse.
Revised: This behaviour is not noted in our chest radiograph dataset as we are able to convert every image from the predicted class to the converse, which is presumably due to the smaller differences between chest X-rays with and without heart failure.

Revised: Similarly, the time taken to generate a visual rationale depends on the conﬁdence of the classiﬁer in its prediction, as the algorithm runs until the input has been altered sufﬁciently or a maximum number of steps (in our case 500) have been completed.

Revised: In the case of converting digit 9s to 4s - we were able to generate 1000 visual rationales in 1 minute and 58 seconds.

Original: We compare this with the occlusion sensitivity and saliency map method demonstrated in Fig.8.
Revised: We compare this with the occlusion sensitivity and saliency map method demonstrated in Fig.

Original: Lastly, we contaminate our heart failure classiﬁer as described in the methods section and compare visual rationales generated by the contaminated classiﬁer with those generated previously.
Revised: Lastly, we contaminate our heart failure classiﬁer as described in the methods section and compare visual rationales generated by the contaminated classiﬁer with those generated previously.

Original: Fig 9.
Revised: Fig 10.

Original: demonstrates images where both classiﬁers predict the presence of heart failure.
Revised: demonstrates images where both classiﬁers predict the presence of heart failure.

Original: The rationales from the contaminated classiﬁer focus on small unique aspects of the image and largely do not correspond to our notion of what makes a chest radiograph more likely to represent heart failure, namely enlarged hearts and congested lung ﬁelds.
Revised: The rationales from the contaminated classiﬁer focus on small unique aspects of the image and largely do not correspond to our notion of what makes a chest radiograph more likely to represent heart failure, namely enlarged hearts and congested lung ﬁelds.

Revised: To demonstrate this we present 100 images classiﬁed as having a BNP level of over 100ng/L to two expert reviewers, equally split between a contaminated or a normally trained classiﬁer.

Revised: Each image and the associated visual rationale was presented to the reviewers who were blinded to the origin of the classiﬁer.

Revised: Reviewers were tasked in selecting features from a provided list which they felt corresponded with the visual rationales.

Original:   D ISCUSSION
Revised: Each reviewer rated each image twice. Aggregated results from experts are presented in Table 2. This clearly shows that the contaminated classiﬁer indeed produces less interpretable visual rationales.  D ISCUSSION

Original: We show in this work that using the generator of a GAN as the decoder of an autoencoder is viable and produces high quality autoencoders.
Revised: We show in this work that using the generator of a GAN as the decoder of an autoencoder is viable and produces high quality autoencoders.

Original: The constraints of adversarial training force the generator to produce realistic radiographs for a given latent space, in this case a 100-dimensional space normally distributed around 0 with a standard deviation of 1.
Revised: The constraints of adversarial training force the generator to produce realistic radiographs for a given latent space, in this case a 100-dimensional space normally distributed around 0 with a standard deviation of 1.

Original: This method bears resemblance to previous work done on inverting GANS done by Creswell &
Revised: This method bears resemblance to previous work done on inverting GANS done by Creswell &

Original: Bharath (2016), although we are not as concerned with recovering the exact latent representation but rather the ability to recreate images from our dataset.
Revised: Bharath (2016), although we are not as concerned with recovering the exact latent representation but rather the ability to recreate images from our dataset.

Original: It is suggested in previous work in Kumar et al.
Revised: It is suggested in previous work in Kumar et al.

Original: (2017) that directly training a encoder to reverse the mapping learnt by the generator in a decoupled fashion does not yield good results as the encoder never sees any real images during training.
Revised: (2017) that directly training a encoder to reverse the mapping learnt by the generator in a decoupled fashion does not yield good results as the encoder never sees any real images during A A B

Revised: B

Revised: Cardiomegaly

Original: By training upon the loss between the real input and generated output images we overcome this.
Revised: Effusion Pacemaker Airspace opacity Contaminated

Revised: A

Revised: A

Revised: B

Revised: B

Revised: Cardiomegaly

Original:  We further establish the utility of this encoder by using encoded latent representations to predict outcomes on unseen datasets, including one not from our institution.
Revised: Effusion Pacemaker Airspace opacity We further establish the utility of this encoder by using encoded latent representations to predict outcomes on unseen datasets, including one not from our institution.

Original: We achieve this without retraining our encoder on these unseen datasets, suggesting that the encoder has learnt useful features about chest radiographs in general.
Revised: We achieve this without retraining our encoder on these unseen datasets, suggesting that the encoder has learnt useful features about chest radiographs in general.

Original: Our primary contribution in this paper however is not the inversion of the generator but rather the ability to generate useful visual rationales.
Revised: Our primary contribution in this paper however is not the inversion of the generator but rather the ability to generate useful visual rationales.

Original: For each prediction of the model we generate a corre- sponding visual rationale with a target class different to the original prediction.
Revised: For each prediction of the model we generate a corresponding visual rationale with a target class different to the original prediction.

Original: We display some examples of the rationales this method produces and inspect these manually to check if these are similar to our understanding of how to interpret these images.
Revised: We display some examples of the rationales this method produces and inspect these manually to check if these are similar to our understanding of how to interpret these images.

Original: The ability to autoencode inputs is essential to our rationale generation although we have not explored in-depth in this paper the effect of different autoencoding algorithms (for instance variational autoencoders) upon the quality of the generated rationales.
Revised: The ability to autoencode inputs is essential to our rationale generation although we have not explored in-depth in this paper the effect of different autoencoding algorithms (for instance variational autoencoders) upon the quality of the generated rationales, as our initial experiments with variational and vanilla autoencoders were not able to reconstruct the level of detail required.

Original: For chest radiographs, common signs of heart failure are an enlarged heart or congested lung ﬁelds, which appear as increased opacities in the parts of the image corresponding to the lungs.
Revised: For chest radiographs, common signs of heart failure are an enlarged heart or congested lung ﬁelds, which appear as increased opacities in the parts of the image corresponding to the lungs.

Original: The rationales generated by the normally trained classiﬁer in Fig 9 appear to be consistent with features described in the medical literature while the contaminated classiﬁer is unable to generate these rationales.
Revised: The rationales generated by the normally trained classiﬁer in Fig 10 appear to be consistent with features described in the medical literature while the contaminated classiﬁer is unable to generate these rationales.

Original: We also demonstrate the generation of rationales with the MNIST dataset where the digit 9 is transformed into 4 while retaining the appearance of the original digit.
Revised: We also demonstrate the generation of rationales with the MNIST dataset where the digit 9 is transformed into 4 while retaining the appearance of the original digit.

Original: We can see that the transformation generally removes the upper horizontal line of the 9 to convert this into a 4.
Revised: We can see that the transformation generally removes the upper horizontal line of the 9 to convert this into a 4.

Original: Interestingly, some digits are not successfully converted.
Revised: Interestingly, some digits are not successfully converted.

Original: Even with different permutations of delta and gamma weights in Algorithm 2 some digits remain resistant to conversion.
Revised: Even with different permutations of delta and gamma weights in Algorithm 2 some digits remain resistant to conversion.

Original: We hypothesize that this may be due to the relative difﬁculty of the chest radiograph dataset compared to MNIST - leading to the extreme conﬁdence of the MNIST model that some digits are not the target class.
Revised: We hypothesize that this may be due to the relative difﬁculty of the chest radiograph dataset compared to MNIST — leading to the extreme conﬁdence of the MNIST model that some digits are not the target class.

Original: This may cause vanishingly small gradients in the target class prediction, preventing gradient descent from achieving the target class.
Revised: This may cause vanishingly small gradients in the target class prediction, preventing gradient descent from achieving the target class.

Original: We compare the visual rationale generated by our method to various other methods including integrated gradients, saliency maps, occlusion sensitivity as well as LIME in Fig.
Revised: We compare the visual rationale generated by our method to various other methods including integrated gradients, saliency maps, occlusion sensitivity as well as LIME in Fig.

Original: All of these methods share similarities in that they attempt to perturb the original image to examine the impact of changes in the image on the ﬁnal prediction, thereby identifying the most salient elements.
Revised: All of these methods share similarities in that they attempt to perturb the original image to examine the impact of changes in the image on the ﬁnal prediction, thereby identifying the most salient elements.

Original: In the saliency map approach, each individual pixel is perturbed, while in the occlusion sensitivity method, squares of the image are perturbed.
Revised: In the saliency map approach, each individual pixel is perturbed, while in the occlusion sensitivity method, squares of the image are perturbed.

Original: LIME changes individual superpixels in an image by changing all the pixels in a given superpixel to the average value.
Revised: LIME changes individual superpixels in an image by changing all the pixels in a given superpixel to the average value.

Original: This approach fails on images where the superpixel classiﬁcation is too coarse, or where the classiﬁcation is not dependent on high resolution details within the superpixel.
Revised: This approach fails on images where the superpixel classiﬁcation is too coarse, or where the classiﬁcation is not dependent on high resolution details within the superpixel.

Original: To paraphrase Sundararajan et al.
Revised: To paraphrase Sundararajan et al.

Original: (2017), attribution or explanation for humans relies upon counterfactual intuition — or altering the image to remove the cause of the predicted outcome.
Revised: (2017), attribution or explanation for humans relies upon counterfactual intuition — or altering the image to remove the cause of the predicted outcome.

Original: Model agnostic methods such as gradient based methods, while fulﬁlling the sensitivity and implementation invariance axioms, do not acknowledge the natural structure of the inputs.
Revised: Model agnostic methods such as gradient based methods, while fulﬁlling the sensitivity and implementation invariance axioms, do not acknowledge the natural structure of the inputs.

Original: For instance, this often leads to noisy pixel-wise attribution as seen in Fig.
Revised: For instance, this often leads to noisy pixel-wise attribution as seen in Fig.

Original: This does not ﬁt well with our human intuition as for many images, large continuous objects dominate our perception and we often do not expect attributions to differ drastically between neighbouring pixels.
Revised: This does not ﬁt well with our human intuition as for many images, large continuous objects dominate our perception and we often do not expect attributions to differ drastically between neighbouring pixels.

Original: Fundamentally these other approaches suffer from their inability to perturb the image in a realistic fashion, whereas our approach perturbs the image’s latent representation, enabling each perturbed image to look realistic as enforced by the GAN’s constraints.
Revised: Fundamentally these other approaches suffer from their inability to perturb the image in a realistic fashion, whereas our approach perturbs the image’s latent representation, enabling each perturbed image to look realistic as enforced by the GAN’s constraints.

Original: Under the manifold hypothesis, natural images lie on a low dimensional manifold embedded in pixel space.
Revised: Under the manifold hypothesis, natural images lie on a low dimensional manifold embedded in pixel space.

Original: Our learned latent space serves as a approximate but useful coordinate system for the manifold of natural images.
Revised: Our learned latent space serves as a approximate but useful coordinate system for the manifold of natural images.

Original: More speciﬁcally the image (pardon the pun) of the generator G [ R d ] is approximately the set of ‘natural images’ (in this case radiographs) and small displacements in latent space around a point z closely map into the tangent space of natural images around G ( z ) .
Revised: More speciﬁcally the image (pardon the pun) of the generator G [ R d ] is approximately the set of ‘natural images’ (in this case radiographs) and small displacements in latent space around a point z closely map into the tangent space of natural images around G ( z ) .

Original: Performing optimization in latent space is implicitly constraining the solutions to lie on the manifold of natural images, which is why our output images remain realistic while being modiﬁed under almost the same objective used for adversarial image generation.
Revised: Performing optimization in latent space is implicitly constraining the solutions to lie on the manifold of natural images, which is why our output images remain realistic while being modiﬁed under almost the same objective used for adversarial image generation.

Original: Hence, our method differs from these previously described methods as it generates high resolution rationales by switching the predicted class of an input image while observing the constraints of the input structure.
Revised: Hence, our method differs from these previously described methods as it generates high resolution rationales by switching the predicted class of an input image while observing the constraints of the input structure.

Original: This can be targeted at particular classes, enabling us answer the question posed to our trained model — ’Why does this image represent Class A rather than Class B?’
Revised: This can be targeted at particular classes, enabling us answer the question posed to our trained model — ‘Why does this image represent Class A rather than Class B?’

Original: There are obvious limitations in this paper in that we do not have a rigorous deﬁnition of what interpretability entails, as pointed out by Sundararajan et al.
Revised: There are obvious limitations in this paper in that we do not have a rigorous deﬁnition of what interpretability entails, as pointed out by Sundararajan et al.

Original: An intuitive understanding of the meaning of interpretability can be obtained from its colloquial usage — as when a teacher attempts to teach by example, an interpretation or explanation for each image helps the student to learn faster and generalize broadly without needing speciﬁc examples.
Revised: An intuitive understanding of the meaning of interpretability can be obtained from its colloquial usage — as when a teacher attempts to teach by example, an interpretation or explanation for each image helps the student to learn faster and generalize broadly without needing speciﬁc examples.

Original: Future work could focus on the measurement of interpretability by judging how much data a second model requires when learning from the predictions and interpretations provided by another pretrained model.
Revised: Future work could focus on the measurement of interpretability by judging how much data a second model requires when learning from the predictions and interpretations provided by another pretrained model.

Original: Maximizing the interpretability of a model may be related to the ability of models to transfer information between each other, facilitating learning without resorting to the use of large scale datasets.
Revised: Maximizing the interpretability of a model may be related to the ability of models to transfer information between each other, facilitating learning without resorting to the use of large scale datasets.

Revised: Such an approach could help evaluate non-image based visual explanations such as sentences, as described in Hendricks et al.

Original: Other technical limitations include the difﬁculty of training a GAN capable of generating realistic images larger than 128 by 128 pixels.
Revised: Other technical limitations include the difﬁculty of training a GAN capable of generating realistic images larger than 128 by 128 pixels.

Original: This limits the performance of subsequent classiﬁers in identifying small features.
Revised: This limits the performance of subsequent classiﬁers in identifying small features.

Original: This can be seen in the poor performance of our model in detecting nodules, a relatively small feature, compared to the baseline implementation in the NIH dataset.
Revised: This can be seen in the poor performance of our model in detecting nodules, a relatively small feature, compared to the baseline implementation in the NIH dataset.

Original: In conclusion we describe a method of semi-supervised learning and apply this to chest radiographs, using local data as well as recent datasets.
Revised: In conclusion, we describe a method of semi-supervised learning and apply this to chest radiographs, using local data as well as recent datasets.

Original: We show that this method can be leveraged to generate visual rationales and demonstrate these qualitatively on chest radiographs as well as the well known
Revised: We show that this method can be leveraged to generate visual rationales and demonstrate these qualitatively on chest radiographs as well as the well known

Original: MNIST set.
Revised: MNIST set.