## 4 Experimentation and Evaluation
### Multiclass Classification
#### 4.2.7 Choice of the Best Model  
Once the analysis was completed and the best model of each type was chosen, an evaluation was performed to determine the optimal model for binary and multiclass classification. Furthermore, a comparison of the SHAP values of the models was conducted to observe the relationships and conclusions of the various attributes used. As for binary classification, a comparison of the models can be seen in Figure 14.  
Figure 14: Comparison of best models for binary classification. \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**Model** & **Dataset** & **Hidden** & **Hidden** & **Hidden** & **Hidden** & **Model** \\
**Type** & **Dataset** & **Layer 1** & **Layer 2** & **Layer 3** & **Layer 4** & **Type** \\ \hline Binary & Imbalanced & 8 neurons & 5 neurons & 6 neurons & 8 neurons & Binary \\ \hline Binary & Imbalanced & 7 neurons & 5 neurons & 7 neurons & 8 neurons & Binary \\ \hline Binary & Balanced & 10 neurons & 7 neurons & 8 neurons & 11 neurons & Binary \\ \hline Binary & Balanced & 10 neurons & 6 neurons & 9 neurons & 10 neurons & Binary \\ \hline Multiclass & Imbalanced & 16 neurons & 9 neurons & 7 neurons & 9 neurons & Multiclass \\ \hline Multiclass & Imbalanced & 15 neurons & 9 neurons & 7 neurons & 9 neurons & Multiclass \\ \hline Multiclass & Balanced & 14 neurons & 7 neurons & 5 neurons & 6 neurons & Multiclass \\ \hline Multiclass & Balanced & 13 neurons & 8 neurons & 5 neurons & 6 neurons & Multiclass \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hypreparameters of the MLP neural network. Figure 13: Influence of each feature by average SHAP values—MLP Multiclass Classifier. Upon analyzing the results, similar values were observed for most metrics. However, the model with the highest values in Precision, Recall, and F1 was the random forest, which was subsequently chosen for the XAI model. Continuing with the SHAP values observed in previous sections, models with the same characteristics were compared, specifically those trained with the balanced dataset. These models were decision tree and XGBoost, both of which had nearly identical SHAP values, with significant importance placed on the _Sttl_ and _ct_state_ttl_ features. On the other hand, the SHAP values for anomaly classification in the random forest model showed a different importance, with very high and even values among the _Dbytes_, _Dmean_, _Dintptk_, _ct_state_ttl_, _Sbytes_, and _dload_ features. This result could indicate that the model is more stable due to the greater influence of multiple features, meaning it uses more information from the traces and does not primarily depend on two fields of information. This is illustrated in Figures 2, 4, and 10. Figure 15 represents the sensitivity of the different features for identifying an anomalous trace, helping to better understand the chosen model for demonstration. Several conclusions can be drawn from the most important features, which can be compared with the real-time performance validated in Section 4.2:  
* _Ct_state_ttl_: A relatively low value directly influences the classification of a trace as an anomaly;
* _Sttl_: An extremely low value influences the classification as an anomaly, but an extremely high value influences it to an even greater extent. This indicates that extreme values in this field are crucial for classifying a trace as anomalous;
* _Dmean_: It can be observed that a range of average or slightly low values to the highest possible values does not influence the labeling of a trace as an anomaly or only slightly influences the classification as normal traffic. However, very low values significantly influence the classification as an anomaly;
* _Dbytes_: Average or slightly low values compared to the average influence the classification of the trace as not normal, while high or extremely low values considerably influence the classification as abnormal traffic;  
Figure 15: Variation of feature values for classifying a trace as an anomaly in the Random Forest model. Binary Classification.