## 4 Experiment and Results
### Binary Classification
#### 4.1.2 Performance Evaluation  
We employed the commonly used performance metrics to evaluate the effectiveness of the selected ML models. These metrics include accuracy, precision, recall, F1-score, and ROC-AUC Score (Receiver Operating Characteristic Area Under the Curve) [50; 51]. Additionally, to gain deeper insights into the model's performance, we analyzed the confusion matrix, which provides a detailed breakdown of the model's predictions. This allows us to understand the distribution of true positives (TPs), true negatives (TNs), false positives (FPs), and false negatives (FNs), as summarized in Table 5. \begin{table}
\begin{tabular}{l l} \hline \hline
**Algorithm** & **Identified Best Parameters (Based on the Defined Parameter Grid)** \\ \hline RF & bootstrap: True, criterion: gini, max\_features: auto, min\_samples\_leaf: 1, \\  & min\_samples\_split: 2, n\_estimators: 100 \\ LR & fit\_intercept: True, max\_iter: 100, penalty: l2, solver: sag, tol: 0.0001 \\  & criterion: gini, max\_depth: 10, max\_features: None, max\_leaf\_nodes: None, \\ DT & min\_impurity\_decrease: 0.0, min\_samples\_leaf: 5, min\_samples\_split: 2, \\  & splitter: best \\ GB & learning\_rate: 0.1, max\_depth: 5, min\_samples\_leaf: 1, min\_samples\_split: 10, n\_estimators: 300 \\ XGBoost & colsample\_bytree: 0.8, gamma: 0, learning\_rate: 0.1, max\_depth: 5, \\  & n\_estimators: 300, subsample: 0.8 \\ MLP & activation: tanh, alpha: 0.0001, hidden\_layer\_sizes: (50, 50), learning\_rate: \\  & constant, solver: adam \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Identified best hyperparameters (binary). * True Positive (TP): The model correctly predicts attack traffic. * False Negative (FN): The model incorrectly predicts attack traffic as legitimate traffic. * False Positive (FP): The model incorrectly predicts legitimate traffic as attack traffic. * True Negative (TN): The model correctly predicts legitimate traffic. The confusion matrix also facilitates the computation of previously mentioned key performance metrics:  
* Accuracy measures the proportion of true results (both true positives and true negatives) among the total number of cases examined, computed as follows [50; 52]: \[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+ \text{FN}}\] (6)
* Precision indicates the proportion of true positive results among all positive results (true positives and false positives). This is particularly important when the cost of false positives is high, such as incorrectly flagging legitimate traffic as an attack. It is computed as follows [50; 52]: \[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}\] (7)
* Recall represents the proportion of true positive results among all actual positive instances (true positives and false negatives). This metric is significant when the cost of false negatives is high, such as missing an actual attack. It is computed as follows [50; 52]: \[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}\] (8)
* F1-score represents the harmonic mean of precision (P) and recall (R), providing a single metric that balances both precision and recall [50]: \[\text{F1 score}=2\cdot\frac{P\cdot R}{P+R}\] (9)
* ROC-AUC score is a performance measurement for classification problems across various threshold settings [50]. The AUC score, which ranges from 0 to 1, quantifies the classifier's ability to discriminate between positive (attack) and negative (legitimate) classes. A higher AUC value indicates better model performance. The ROC curve is created by plotting the true positive rate (TPR) against the false positive rate (FPR) at different threshold levels [51]. Table 6 summarizes the performance metrics obtained from the employed model. Searching Time (s) in Table 6, refers to the computational time required by hyperparameter optimization procedures (GridSearchCV for traditional ML models and Random Search via Keras Tuner for the DNN) to identify optimal hyperparameter combinations used for subsequent model training.