Title: An Explainable 3D Convolutional Neural Network with Reliable Feature Selection and Hybrid 3D Image Block Ranking
Primary area: interpretability and explainable AI
Keywords: computer vision; convolutional neural networks; feature selection; explainability; hybrid; image classification

Abstract: An explainable 3D imaging challenge is effectively interpreting the rational relationship among top-ranked 3D image blocks, relevant features selected from extracted 3D feature maps, and decisions of a 3D convolutional neural network (CNN). We propose an explainable 3D CNN that integrates a robust feature selection (FS) method with a new hybrid block ranking algorithm to uncover the spatial relationship among 3D image blocks, selected features, and clinical diagnosis. The 3D image block ranking pipeline begins with a multi-FS procedure that removes irrelevant features that are out of an object and produces diverse selected feature sets. Then the selected feature sets are used to construct complementary 3D distribution, ranking, and average-ranking feature matrices that quantify importance levels for both relevant 3D image blocks and decisions. Finally, the novel hybrid 3D image block ranking algorithm leverages the complementary 3D feature matrices to reliably generate a block ranking map (BRM) with 3D block ranking numbers. Simulation results using 982 3D ADNI data for Alzheimer’s disease (AD) diagnosis (3-class classification) indicate that the top-ranked 3D image blocks contain brain areas associated with AD diagnosis for two different simulations using overlapping neighboring and non-overlapping neighboring 3D image blocks. A medical doctor may conveniently use the BRM’s top-ranked 3D image blocks of a patient’s 3D image visualized by axial 2D patches, coronal 2D patches, and sagittal 2D patches to efficiently make an explainable and correct medical diagnosis. Importantly, the new 3D CNN with FS is better than the traditional 3D CNN without FS in terms of test accuracy, F1-score, AUC, and model size for both AD diagnosis and autism diagnosis (binary classification). Thus, the new 3D CNN with the reliable FS and the hybrid 3D image block ranking is more effective than the traditional 3D CNN for explainable, accurate, and memory-efficient 3D image classification applications.