## 4 Dataset
### Data Acquisition
This proportion strikes a balance between having sufficient data for reliable parameter optimization and retaining a separate portion for testing. Importantly, the validation set helps to prevent overfitting by ensuring that the model's performance is evaluated on data that are not directly used for training, providing an early indication of how well the model can generalize to new data. The test set, also accounting for 15%, was reserved exclusively for evaluating the model's final performance on unseen data. This percentage provides a sufficient sample size to obtain statistically significant results and ensures a reliable assessment of the model's generalization ability. Using equal proportions for validation and testing maintains consistency and avoids skewed evaluations, as both sets are derived from the same data distribution. We evaluated the performance of four ML models (LR, RF, XGBoost, and SVM) using key evaluation metrics, including the accuracy, precision, recall, and F1-score. Table 4 and Figure 4 present the average detection performance of these models. Figure 5 illustrates the models' confusion matrices. Precision, a measure of the proportion of true positives among all positive predictions, was similarly high across all models. XGB achieved a perfect precision score of 1.00, while LR and SVM followed closely with scores of 0.996 and RF achieved a score of 0.986. These results reflect the models' reliability in minimizing false positives when predicting the positive class. Recall, which quantifies the ability of a model to identify all true positive instances, paralleled the precision results. XGB and SVM both achieved perfect recall of 1.00, whereas LR and RF scored slightly lower at 0.996 and 0.986, respectively. This demonstrates that the models were equally adept at minimizing false negatives. The F1-scores, which balance precision and recall, also underscore the models' robust performance. XGB and SVM achieved perfect scores of 1.00, indicating an optimal tradeoff between precision and recall. LR and RF, with F1-scores of 0.996 and 0.986, respectively, demonstrated slightly lower but still excellent performance. Figure 4: Performance results for the different ML models. Table 5 quantitatively compares the proposed tool with recently developed tools discussed in Section 2 which utilize the same ML algorithms. Comparing these results with previous studies, our implementations of LR, XGBoost, and SVM notably outperformed the benchmarks in terms of classification accuracy and precision [38; 55; 60]. RF scored similarly to existing results, while XGB consistently achieved superior performance across all metrics. Because our proposed tool works with online data instead of relying on static offline data, we believe that our approach can also enhance similar models proposed in other studies [63; 64; 65; 66]. Given the uniformly high performance of the models, selecting which one to use for deployment may depend on factors such as computational efficiency, interpretability, \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Metric**} & **Logistic Regression (LR)** & **Random Forest (RF)** & \multirow{2}{*}{**XGBoost**} & **Support Vector Machine (SVM)** \\ \hline Accuracy & 0.99 & 0.99 & 1.00 & 0.99 \\ \hline Precision & 0.996 & 0.986 & 1.00 & 0.996 \\ \hline Recall & 0.996 & 0.986 & 1.00 & 0.996 \\ \hline F1-score & 0.996 & 0.986 & 1.00 & 1.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Model performance. Figure 5: Confusion matrices for the different models: (**a**) Random Forest (RF), (**b**) Logistic Regression (LR), (**c**) XGBoost, (**d**) Support Vector Machine (SVM). and application-specific requirements. For example, the simplicity and interpretability of LR make it a suitable choice when model transparency is crucial. Conversely, XGB's unmatched accuracy makes it ideal for high-stakes environments where predictive precision is paramount.