## 5 Results  
Tables 2 and 3 present the results calculated for each algorithm, indicated in the first column, derived from the stratified cross-validation experiment with k equal to 10 and averaged between them. In particular, with regard to the metrics of Precision, Recall, F1-Score, they are to be considered calculated with macro average. The reference dataset for Table 2 is _malware-analysis-datasets-api-call-sequences_, while Table 3 presents the results for dataset _APIMDS_. Table 2 highlights the differences between the algorithms. In this specific case, the deep learning techniques NODE and TabNet, although showing an average F1-macro score in line with the state of the art, have F1-macro score and Area Under the ROC curves lower when compared to algorithms using trees such as Random Forest, XGBoost, and CatBoost. Finally, CatBoost presents the best performance, which seems to obtain the best trade-off between precision and recall. The dataset _APIMDS_ consists of 300 goodware class examples (labeled with 0) and 23,146 malware class examples (labeled with 1). Again, shallow learning techniques using trees such as Random Forest, Cat Boost and XGBoost perform slightly better than their deep learning counterparts such as TabNet and NODE. Specifically, it can be seen that, as in the case of the previous dataset, XGBoost and CatBoost are the best techniques both from an AUC ROC and F1-macro score point of view. XGBoost is slightly better than CatBoost in this experiment because it has better recall; however, it is also relevant to note how robust CatBoost is to the dataset change, i.e., its ability to perform with accuracies in line with the state of the art regardless of dataset. It is now of interest to understand what features the best performing algorithms, such as CatBoost, deem most important for classifying instances as malware and as goodware using the explainable artificial intelligence SHAP technique. Table 4 illustrates the training and prediction times of the treated algorithms as applied to dataset _malware-analysis-datasets-api-call-sequences_. The time intervals represented in the table are expressed in seconds. To achieve a more accurate estimate, the times reported in the table were averaged on the number of iterations completed for each step by each algorithm, i.e., 100. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Algorithm** & **Training Time** & **Prediction Time** \\ \hline Random Forest & 12.84419 & 0.07022 \\ CatBoost & 6.70016 & 0.00566 \\ XGBoost & 7.50087 & 0.01600 \\ TabNet & 38.97393 & 0.16049 \\ ExtraTrees & 4.22829 & 0.07658 \\ NODE & 49,774.68524 \({}^{\star}\) & 31.14673 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Training and prediction times measured in seconds—_malware-analysis-datasets-api-call-sequences_ dataset. \({}^{\star}\) This value is relative to a single training task. Because it is enormous, it is impossible to average over 100 iterations. Instead, the prediction time was averaged. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & **Precision** & **Recall** & **F1 Score** & **AUC ROC** \\ \hline Random Forest & 0.965 & 0.809 & 0.869 & 0.8094 \\ CatBoost & 0.962 & 0.82 & 0.877 & 0.8201 \\ XGBoost & 0.9575 & 0.7816 & 0.8473 & 0.7816 \\ ExtraTrees & 0.97 & 0.7411 & 0.8162 & 0.7411 \\ NODE & 0.9532 & 0.7238 & 0.7971 & 0.7238 \\ TabNet & 0.8985 & 0.7674 & 0.8166 & 0.7674 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Metrics averaged over 10 folds—_malware-analysis-datasets-api-call-sequences_ dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & **Precision** & **Recall** & **F1 Score** & **AUC ROC** \\ \hline Random Forest & 0.9914 & 0.9215 & 0.9532 & 0.9215 \\ CatBoost & 0.9955 & 0.9399 & 0.9655 & 0.9399 \\ XGBoost & 0.9944 & 0.9619 & 0.9779 & 0.9632 \\ ExtraTrees & 0.9949 & 0.9132 & 0.9498 & 0.9132 \\ NODE & 0.9948 & 0.9166 & 0.9519 & 0.0022 \\ TabNet & 0.9843 & 0.9264 & 0.9529 & 0.9264 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Metrics averaged over 10 folds—_APIMDS_ dataset.