# A Review of Explainable Deep Learning Cancer Detection Models in Medical Imaging
## 3 Explanation Methods for Deep Learning Cancer Detection Models
### Ad-Hoc  
Ad-Hoc explanation techniques modify the training procedure and/or network architecture to learn explainable features. An additional proposed approach to learn explainable features was the work by the authors of [130] using supervised iterative descent. The authors' approach learns explainable features for image registration by incorporating a shared modality-agnostic feature space and using segmentations to derive a weak supervisory signal for registration. The authors report their experiments \"demonstrate that these features can be plugged into conventional iterative optimizers and are more robust than state-of-the-art hand-crafted features for aligning MRI and CT images\". This is significant with respect to medical imaging because of the multi-modality aspect which is common to medical imaging. The authors of [131] propose a two part explainable deep learning network. One part extracts features and predicts while the other part generates an explanation. The authors create a mathematical model to extract high-level explainable features. The explanation model reports information such as what are the most important features, what is their rank, and how does each one influence the prediction. Capsule networks have been recently utilized due to the impression they are more explainable compared to CNN [132]. The authors of [133] state that convolutions neural networks struggle with explainability therefore they propose explainable capsules, or X-Caps. X-Caps utilize a capsule network architecture that is designed to be inherently explainable. The authors design to capsule network to learn an explainable feature space by encoding high-level visual attributes within the vectors of a capsule network to perform explainable image-based diagnosis. The authors report accuracy similar to that of unexplainable models demonstrating accuracy and explainability are not mutually exclusive. The authors define a routing sigmoid as follow. \[r_{i,j}=\frac{exp(b_{i,j})}{exp(b_{i,j})+1} \tag{7}\]  
where \(r_{i,j}\) are the routing coefficients determined by the dynamic routing algorithm for child capsule \(i\) to parent capsule \(j\) and the initial logits, \(b_{i,j}\) are the prior probabilities that the prediction vector for capsule \(i\) should be routed to parent capsule \(j\). \[L_{r}=\frac{\gamma}{H\times W}\sum_{x}^{W}\sum_{y}^{H}||R^{x,y}-O_{r}^{x,y}||\ with\ R^{x,y}=I^{x,y}\times S^{x,y}|S^{x,y}\in 0,1 \tag{8}\]  
Let the critical response maps \(A(x|c)\) for a given CT scan slice image \(x\) for each prediction \(c\) be computed via back-propagation from the last layer of the last explainable sequencing cell in the proposed SISC radiomic sequencer. The notation used in this study is based on the study done by the authors of [134] for consistency. The last layer in the explainable sequencing cell at the end of the proposed SISC radiomic sequencer contains \(N=2\) nodes, equal to the number of possible predictions (i.e., benign and malignant). The output activations of this layer are followed by global average pooling and then a softmax output layer. So, to create the critical maps for each possible prediction, the back-propagation starts with the individual prediction nodes in the last layer to the input space. For a single layer \(l\), The deconvoluted output response \(r_{l}\) is given by,  
\[h_{l}=\sum_{k=1}^{K}f_{k,l}*p_{k,l} \tag{9}\]  
where \(f_{k}\) is the feature map and \(p_{l}\) is the kernel of layer \(l\). the symbol \(*\) represents the convolution operator. Therefore, the critical response map \(A(x|c)\), for a given prediction \(c\) is defined as,  
\[A(x|c)=D_{1}^{\prime}M_{1}...D_{L-1}M_{L-1}^{\prime}D_{L}^{c}F_{L} \tag{10}\]  
where \(M^{\prime}\) is the unpooling operation and \(D_{L}^{c}\) is the convolution operation at the last layer with kernel \(p_{L}\) replaced by zero except at the \(c\)th location corresponding to the prediction \(c\).