# Explaining Deep Learning Models for Tabular Data Using Layer-Wise Relevance Propagation
## 3 Proposed Approach
### Second Phase  
XAI Technique (LRP, SHAP, and LIME)  
In the second phase, once the 1D-CNN is trained, we use that trained deep model as our trial model for explainability. LRP uses the trained model to generate a heatmap based on its relevance values. Our interest in the use of heatmaps is to find and determine what set of features are the most relevant in the prediction of True Positives (1) and True Negatives (0). Furthermore, our objective is to show the important features not only for an individual sample (local analysis) but also for the whole testing set as an overall global pattern learned by the classifier (global analysis). We have also generated heatmaps from the trained model for comparison with both SHAP and LIME. We use the default versions of LRP, SHAP, and LIME, without tuning of parameters or use of variants, in order to get a baseline comparison of the three methods. Figure 3 shows the structure of how in general LRP calculate relevance whereas Figure 4 shows the LRP heatmaps at instance and class level, with the colour scheme reflecting feature importance based on (pre-normalised) LRP values. In the following sub-sections, a brief description of how these techniques work is given. **Layer-wise Relevance Propagation (LRP):** LRP is one of the main algorithms for the explainability of networks that uses the back-propagation algorithm [41]. LRP explains a classifier's prediction specific to a given data point by attributing 'Relevance Values' (Ri) to important components of the input by using the topology of the trained model itself. It is efficiently utilised on images/videos and text where the output predicted value is used to calculate the relevance value for the neurons in the lower layer. The higher the impact of a neuron in the forward pass, the higher its relevance in the backward pass. This relevance calculation follows through to the input where the highly relevant neurons/features are or will have higher values compared to other neurons. As a result, when visualised, the important input neurons can be clearly highlighted based on which final decision was taken in the output layer. Figure 3 shows the flow of the relevance value calculation. LRP is currently being widely used with CNNs, and to a lesser extent for LSTM in the XAI domain. Improvements in LRP are an active area of research. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model Name** & **LR** & **BatchSize** & **Iterations** & **Network Structure** \\ \hline M-1D-CNN-1-28 & 0.00001 & 300 & 15,000 & C25-C50-C100-F2200-O2 \\ \hline M-1D-CNN-2-28 & 0.00001 & 200 & 15,000 & C25-C50-C100-F2200-F500-F10-O2 \\ \hline M-1D-CNN-3-16 & 0.00001 & 300 & 15,000 & C25-C50-C100-F200-O2 \\ \hline M-1D-CNN-4-16 & 0.00001 & 300 & 15,000 & C25-C50-C100-C200-F1600-F800-O2 \\ \hline M-1D-CNN-1-31 & 0.00001 & 300 & 15,000 & C25-C50-C100-C200-F4400-O2 \\ \hline M-1D-CNN-2-31 & 0.0001 & 300 & 15,000 & C25-C50-C100-C200-F4400-O2 \\ \hline M-1D-CNN-3-31 & 0.0001 & 200 & 15,000 & C25-C50-C100-C200-F4400-O2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Proposed networks structure. Here convolution, fully connected, and output layers are represented by C, F, and O. The number shows the number of kernels/neurons in that layer. **Local Interpretable Model-agnostic Explanations:** LIME [32] is currently one of the most well-known methods to explain any classification model from a local point of view. This method is considered agnostic because it does not make any assumptions about how the classifier works. It simply builds a surrogate simple model, intrinsically interpretable, such as a linear regression model around each prediction between the input variables and the corresponding outcome variables. The use of a simple model in a local way allows for easier interpretation of the behaviour of the classification model in the vicinity of the instance being predicted. LIME attempts to understand the classification model by perturbing the input variables of a data sample and understanding how the predictions change.