Weakly supervised Unet: an image classifier which learns to explain itself
Authors: O’Shea Robert John , Horst Carolyn , Manickavasagar Thubeena , Hughes Daniel , Cusack James , Tsoka Sophia , Cook Gary , Goh Vicky 
Published: September 2022

ABSTRACTBackgroundExplainability is a major limitation of current convolutional neural network (CNN) image classifiers. A CNN is required which supports its image-level prediction with a voxel-level segmentation.MethodsA weakly-supervised Unet architecture (WSUnet) is proposed to model voxel classes, by training with image-level supervision. WSUnet computes the image-level class prediction from the maximal voxel class prediction. Thus, voxel-level predictions provide a causally verifiable saliency map for the image-level decision.WSUnet is applied to explainable lung cancer detection in CT images. For comparison, current model explanation approaches are also applied to a standard CNN. Methods are compared using voxel-level discrimination metrics and a clinician preference survey.ResultsIn test data from two external institutions, WSUnet localised the tumour precisely at voxel-level (Precision: 0.93 [0.93-0.94]), achieving superior voxel-level discrimination to the best comparator (AUPR: 0.55 [0.54-0.55] vs. 0.36 [0.35-0.36]). Clinicians preferred WSUnet predictions in most test instances (Clinician Preference Rate: 0.72 [0.68-0.77]).ConclusionsWSUnet is a simple extension of the Unet, which facilitates voxel-level modelling from image-level labels. As WSUnet supports its image-level prediction with a causative voxel-level segmentation, it functions as a self-explaining image classifier.Graphical AbstractThe weakly-supervised Unet converts voxel-level predictions to image-level predictions using a global max-pooling layer. Thus, loss is computed at image-level. Following training with image-level labels, voxel-level predictions are extracted from the voxel-level output layer.FundingAuthors acknowledge funding support from the UK Research & Innovation London Medical Imaging and Artificial Intelligence Centre; Wellcome/Engineering and Physical Sciences Research Council Centre for Medical Engineering at King’s College London [WT 203148/Z/16/Z]; National Institute for Health Research Biomedical Research Centre at Guy’s & St Thomas’ Hospitals and King’s College London; National Institute for Health Research Biomedical Research Centre at Guy’s & St Thomas’ Hospitals and King’s College London; Cancer Research UK National Cancer Imaging Translational Accelerator [C1519/A28682]. For the purpose of open access, authors have applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.HIGHLIGHTSWSUnet is a weakly supervised Unet architecture which can learn semantic segmentation from data labelled only at image-level.WSUnet is a convolutional neural network image classifier which provides a causally verifiable voxel-level explanation to support its image-level prediction.In application to explainable lung cancer detection, WSUnet’s voxel-level output localises tumours precisely, outperforming current model explanation methods.WSUnet is a simple extension of the standard Unet architecture, requiring only the addition of a global max-pooling layer to the output.