user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Thinking like a machine — generating visual rationales through latent space optimization

Paper content:
Interpretability and small labelled datasets are key issues in the practical application of deep learning, particularly in areas such as medicine.
In this paper, we present a semi-supervised technique that addresses both these issues simultaneously.
We learn dense representations from large unlabelled image datasets, then use those representations to both learn classifiers from small labeled sets and generate visual rationales explaining the predictions.
Using chest radiography diagnosis as a motivating application, we show our method has good generalization ability by learning to represent our chest radiography dataset while training a classifier on an separate set from a different institution.
Our method identifies heart failure and other thoracic diseases.
For each prediction, we generate visual rationales for positive classifications by optimizing a latent representation to minimize the probability of disease while constrained by a similarity measure in image space.
Decoding the resultant latent representation produces an image without apparent disease.
The difference between the original and the altered image forms an interpretable visual rationale for the algorithm's prediction.
Our method simultaneously produces visual rationales that compare favourably to previous techniques and a classifier that outperforms the current state-of-the-art.
Deep learning as applied to medicine has attracted much interest in recent years as a potential solution to many difficult problems in medicine, such as the recognition of diseases on pathology slides or radiology images.
However, adoption of machine learning algorithms in fields such as medicine relies on the end user being able to understand and trust the algorithm, as incorrect implementation and errors may have significant consequences.
Hence, there has recently been much interest in interpretability in machine learning as this is a key aspect of implementing machine learning algorithms in practice.
We propose a novel method of creating visual rationales to help explain individual predictions and explore a specific application to classifying chest radiographs.There are several well-known techniques in the literature for generating visual heatmaps.
Gradient based methods were first proposed in 2013 described as a saliency map in BID11 , where the derivative of the final class predictions is computed with respect to the input pixels, generating a map of which pixels are considered important.
However, these saliency maps are often unintelligible as convolutional neural networks tend to be sensitive to almost imperceptible changes in pixel intensities, as demonstrated by recent work in adversarial examples.
In fact, obtaining the saliency map is often the first step in generating adversarial examples as in BID3 .
Other recent developments in gradient based methods such as Integrated Gradients from BID12 have introduced fundamental axioms, including the idea of sensitivity which helps focus gradients on relevant features.Occlusion sensitivity proposed by Zeiler & Fergus (2013) is another method which covers parts of the image with a grey box, mapping the resultant change in prediction.
This produces a heatmap where features important to the final prediction are highlighted as they are occluded.
Another wellknown method of generating visual heatmaps is global average pooling.
Using fully convolutional neural networks with a global average pooling layer as described in BID15 , we can examine the class activation map for the final convolutional output prior to pooling, providing a low resolution heatmap for activations pertinent to that class.A novel analysis method by BID10 known as locally interpretable model-agnostic explanations (LIME) attempts to explain individual predictions by simulating model predictions in the local neighbourhood around this example.
Gradient based methods and occlusion sensitivity can also be viewed in this light -attempting to explain each classification by changing individual input pixels or occluding square areas.However, sampling the neighbourhood surrounding an example in raw feature space can often be tricky, especially for image data.
Image data is extremely complex and high-dimensional -hence real examples are sparsely distributed in pixel space.
Sampling randomly in all directions around pixel space is likely to produce non-realistic images.
LIME's solution to this is to use superpixel based algorithms to oversegment images, and to perturb the image by replacing each superpixel by its average value, or a fixed pre-determined value.
While this produces more plausible looking images as opposed to occlusion or changing individual pixels, it is still sensitive to the parameters and the type of oversegmentation used -as features larger than a superpixel and differences in global statistics may not be represented in the set of perturbed images.
This difficulty in producing high resolution visual rationales using existing techniques motivates our current research.
We show in this work that using the generator of a GAN as the decoder of an autoencoder is viable and produces high quality autoencoders.
The constraints of adversarial training force the generator to produce realistic radiographs for a given latent space, in this case a 100-dimensional space normally distributed around 0 with a standard deviation of 1.This method bears resemblance to previous work done on inverting GANS done by BID2 , although we are not as concerned with recovering the exact latent representation but rather the ability to recreate images from our dataset.
It is suggested in previous work in BID8 that directly training a encoder to reverse the mapping learnt by the generator in a decoupled fashion does not yield good results as the encoder never sees any real images during training.
By training upon the loss between the real input and generated output images we overcome this.We further establish the utility of this encoder by using encoded latent representations to predict outcomes on unseen datasets, including one not from our institution.
We achieve this without retraining our encoder on these unseen datasets, suggesting that the encoder has learnt useful features about chest radiographs in general.Our primary contribution in this paper however is not the inversion of the generator but rather the ability to generate useful visual rationales.
For each prediction of the model we generate a corresponding visual rationale with a target class different to the original prediction.
We display some examples of the rationales this method produces and inspect these manually to check if these are similar to our understanding of how to interpret these images.
The ability to autoencode inputs is essential to our rationale generation although we have not explored in-depth in this paper the effect of different autoencoding algorithms (for instance variational autoencoders) upon the quality of the generated rationales, as our initial experiments with variational and vanilla autoencoders were not able to reconstruct the level of detail required.For chest radiographs, common signs of heart failure are an enlarged heart or congested lung fields, which appear as increased opacities in the parts of the image corresponding to the lungs.
The rationales generated by the normally trained classifier in FIG0 to be consistent with features described in the medical literature while the contaminated classifier is unable to generate these rationales.We also demonstrate the generation of rationales with the MNIST dataset where the digit 9 is transformed into 4 while retaining the appearance of the original digit.
We can see that the transformation generally removes the upper horizontal line of the 9 to convert this into a 4.
Interestingly, some digits are not successfully converted.
Even with different permutations of delta and gamma weights in Algorithm 2 some digits remain resistant to conversion.
We hypothesize that this may be due to the relative difficulty of the chest radiograph dataset compared to MNIST -leading to the extreme confidence of the MNIST model that some digits are not the target class.
This may cause vanishingly small gradients in the target class prediction, preventing gradient descent from achieving the target class.We compare the visual rationale generated by our method to various other methods including integrated gradients, saliency maps, occlusion sensitivity as well as LIME in Fig. 6 .All
of these methods share similarities in that they attempt to perturb the original image to examine the impact of changes in the image on the final prediction, thereby identifying the most salient elements. In
the saliency map approach, each individual pixel is perturbed, while in the occlusion sensitivity method, squares of the image are perturbed. LIME
changes individual superpixels in an image by changing all the pixels in a given superpixel to the average value. This
approach fails on images where the superpixel classification is too coarse, or where the classification is not dependent on high resolution details within the superpixel. To paraphrase
BID12 , attribution or explanation for humans relies upon counterfactual intuition -or altering the image to remove the cause of the predicted outcome. Model agnostic
methods such as gradient based methods, while fulfilling the sensitivity and implementation invariance axioms, do not acknowledge the natural structure of the inputs. For instance,
this often leads to noisy pixel-wise attribution as seen in Fig. 6 . This does not
fit well with our human intuition as for many images, large continuous objects dominate our perception and we often do not expect attributions to differ drastically between neighbouring pixels.Fundamentally these other approaches suffer from their inability to perturb the image in a realistic fashion, whereas our approach perturbs the image's latent representation, enabling each perturbed image to look realistic as enforced by the GAN's constraints.Under the manifold hypothesis, natural images lie on a low dimensional manifold embedded in pixel space. Our learned latent
space serves as a approximate but useful coordinate system for the manifold of natural images. More specifically
the image (pardon the pun) of the generator G [R d ] is approximately the set of 'natural images' (in this case radiographs) and small displacements in latent space around a point z closely map into the tangent space of natural images around G(z). Performing optimization
in latent space is implicitly constraining the solutions to lie on the manifold of natural images, which is why our output images remain realistic while being modified under almost the same objective used for adversarial image generation.Hence, our method differs from these previously described methods as it generates high resolution rationales by switching the predicted class of an input image while observing the constraints of the input structure. This can be targeted at
particular classes, enabling us answer the question posed to our trained model -'Why does this image represent Class A rather than Class B?'There are obvious limitations in this paper in that we do not have a rigorous definition of what interpretability entails, as pointed out by BID12 . An intuitive understanding
of the meaning of interpretability can be obtained from its colloquial usage -as when a teacher attempts to teach by example, an interpretation or explanation for each image helps the student to learn faster and generalize broadly without needing specific examples.Future work could focus on the measurement of interpretability by judging how much data a second model requires when learning from the predictions and interpretations provided by another pretrained model. Maximizing the interpretability
of a model may be related to the ability of models to transfer information between each other, facilitating learning without resorting to the use of large scale datasets. Such an approach could help evaluate
non-image based visual explanations such as sentences, as described in BID5 .Other technical limitations include the
difficulty of training a GAN capable of generating realistic images larger than 128 by 128 pixels. This limits the performance of subsequent
classifiers in identifying small features. This can be seen in the poor performance
of our model in detecting nodules, a relatively small feature, compared to the baseline implementation in the NIH dataset.In conclusion, we describe a method of semi-supervised learning and apply this to chest radiographs, using local data as well as recent datasets. We show that this method can be leveraged
to generate visual rationales and demonstrate these qualitatively on chest radiographs as well as the well known MNIST set.

assistant: text: We propose a method of using GANs to generate high quality visual rationales to help explain model predictions.