  As AI-based medical devices are becoming more common in imaging fields like
radiology and histology, interpretability of the underlying predictive models
is crucial to expand their use in clinical practice. Existing heatmap-based
interpretability methods such as GradCAM only highlight the location of
predictive features but do not explain how they contribute to the prediction.
In this paper, we propose a new interpretability method that can be used to
understand the predictions of any black-box model on images, by showing how the
input image would be modified in order to produce different predictions. A
StyleGAN is trained on medical images to provide a mapping between latent
vectors and images. Our method identifies the optimal direction in the latent
space to create a change in the model prediction. By shifting the latent
representation of an input image along this direction, we can produce a series
of new synthetic images with changed predictions. We validate our approach on
histology and radiology images, and demonstrate its ability to provide
meaningful explanations that are more informative than GradCAM heatmaps. Our
method reveals the patterns learned by the model, which allows clinicians to
build trust in the model's predictions, discover new biomarkers and eventually
reveal potential biases.
