## 5 Experiment
### Experiment with the Interpretation Process  
Here is an example of how the proposed method can extract function identifiers used to detect malicious applications. For this example, we selected a app (sha2: ae1377 abf1755523 fe 96a41456c88230f239 ec106041b91 ad6282c739072 aae0) from the CICMalDroid2020 dataset that is categorized as part of the SMS malware family. The following Figure 4 shows the attention map generated by the Attention Rollout and the filtering of results to keep only the high attention regions of the attention map. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Accuracy** & **F1 Score** & **Precision** & **Recall** & **F1 Score** & **Precision** & **Recall** \\ \hline \multicolumn{7}{c}{**Weighted Avg**} & \multicolumn{3}{c}{**Macro Avg**} \\ \hline our proposed model & 0.8681 & 0.8701 & 0.8753 & 0.8681 & 0.8422 & 0.8386 & 0.8521 \\ CNN [7] & 0.8107 & 0.8093 & 0.8090 & 0.8107 & 0.7734 & 0.7809 & 0.7676 \\ ResNet18 & 0.8859 & 0.8859 & 0.8869 & 0.8859 & 0.8617 & 0.8658 & 0.8585 \\ ResNet50 & 0.8525 & 0.8534 & 0.8553 & 0.8236 & 0.8190 & 0.8157 & 0.8236 \\ DenseNet121 & 0.9072 & 0.9071 & 0.9071 & 0.9072 & 0.8866 & 0.8883 & 0.8852 \\ MobileNetV2 & 0.7944 & 0.7925 & 0.7913 & 0.7944 & 0.7423 & 0.7464 & 0.7392 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluation of malware type detection on CICMalDroid2020 dataset for our proposed model. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Accuracy** & **F1 Score** & **Precision** & **Recall** & **F1 Score** & **Precision** & **Recall** \\ \hline \multicolumn{7}{c}{**Weighted Avg**} & \multicolumn{3}{c}{**Macro Avg**} \\ \hline our proposed model & 0.8027 & 0.7743 & 0.7775 & 0.8027 & 0.3113 & 0.5219 & 0.2509 \\ ResNet18 & 0.8005 & 0.7737 & 0.7762 & 0.8005 & 0.2748 & 0.4800 & 0.2207 \\ ResNet50 & 0.7843 & 0.7514 & 0.7528 & 0.7843 & 0.2230 & 0.3767 & 0.1803 \\ DenseNet121 & 0.8047 & 0.7770 & 0.7764 & 0.8047 & 0.2397 & 0.3831 & 0.1983 \\ MobileNetV2 & 0.7701 & 0.7258 & 0.7333 & 0.7701 & 0.1776 & 0.3803 & 0.1381 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Malware-type detection scores for our proposed model and four popular CNN-based models on the AndroZoo dataset. The DEX is converted to an RGB-based image. This image is fed into the model to obtain the model's detection results. Then, the method in Section 4.4.2 is used to build the model's attention map for the DEX image. The attention map is drawn as shown in Figure 4a, where areas closer to red are less critical, and areas closer to blue are more important. To select the high attention regions of the attention map, we used Equations (12) and (13) to keep only values above a certain percentile. In Figure 4, typically, the 93 rd or 95 th percentile is a reasonable threshold for identifying valuable regions. Figure 4: The attention maps that map have been filtered to remove values below a certain threshold percentage. (**a**–**i**) are the percentile values, which means 0, 25, 50, 75, 90, 93, 95, 97, and 100. Figure 3: ROC curves for ‘addisplay’ and ‘spyware’ categories alongside the macro average ROC curve for all categories within the AndroZoo dataset. As shown in Figure 4f,g, the high attention regions are clearly visible. Once the region is selected, the Algorithm 2 is used to change it to the DEX offset value. Figure 5 shows how well the selected regions were converted to dex offset values with Algorithm 2. Both images are identical. Notice that the high attention score regions extracted from the attention map are correctly converted to dex offset values. Comparison with Malware Detection Interpretation Methods  
Our proposed method is compared with existing studies that attempt to classify malicious apps and describe malicious behavior. Table 7 shows studies related to the interpretation of malicious app detection models. One of the related studies, [PERSON] et al. [7], proposed cumulative heatmaps to provide interpretability to image-based detection models.