## 3 Methodology
### SEBD Architecture and Framework Phases
#### 4.1.4 Evaluation Using Other Classifiers

For the fourth evaluation, we used different classifiers, including random forest (RF) with the cross-validation method to observe the predictions of SEBD using both datasets (group 1 and group 2). We used a support vector machine (SVM), naive Bayes (NB), k-nearest neighbors (KNN), and an ensemble voting classifier, and compared the precision, recall, and f1 measures. The results in Figure 5A show that, for the group 1 dataset, SEBD achieved very high recall scores (0.995 and 1), which measures the proportion of actual positive instances that were correctly identified by the model. A high recall score means that the model is able to detect most of the positive instances. For the precision scores of group 1, which measured the proportion of positive predictions that were actually positive, SEBD achieved 0.858 to 0.99. These high precision scores mean that the model produced a low number of false-positive predictions. An F1 score of 89% represents a balanced measure of precision and recall. As for F1, SEBD achieved a high F1 score (between 0.92 to 0.99), which indicates that the model performed well in terms of both precision and recall. For the group 2 dataset, as shown in Figure 5B, SEBD maintained higher scores for recall, and both RF and the ensemble classifier performed best in both datasets. The overall scores indicate that SEBD is able to effectively identify bot accounts. Figure 4: Receiver operating characteristic (ROC) curve for benchmark dataset using training/test with group 1 dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Accuracy** & **F1** & **Recall** & **Precision** & **MCC** \\ \hline Ref [54] & 0.8273 & 0.8967 & 0.8708 & 0.9245 & 0.3780 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Result of benchmark dataset with RF training/test using group 1 dataset. ### SEBD with PAC Parameters

To evaluate SEBD with PAC parameters, we used group 2 hashtags, as shown in Table 2. This is due to the fact that all group 1 hashtags were collected and processed at once. Therefore, we used a different hashtag status to measure the model performance using PAC parameters (epsilon = 0.05, delta = 0.05). The results of the application of the PAC framework, in terms of false-positive rates and false-negative rates, are shown in Table 6. The given epsilon value of 0.05 means that the model was confident, with a probability of 1 \(-\) 0.05 = 0.95, that is, its error rate was less than 0.05. The given delta value of 0.05 means that the model was accurate, with a probability of 1 \(-\) 0.05 = 0.95, that is, its error rate was less than 0.05. SEBD showed a false-positive rate (FPR) of 0.025 and a false-negative rate (FNR) of 0.032 in the whole group 2 dataset. This means that, out of 100 instances where the model predicted a positive result, 25 were actually false-positive (incorrect) predictions and, out of 100 instances where the ground truth was positive, 32 predictions were missed (incorrect) by the model. Therefore, based on these results, the model seems to have good accuracy and confidence levels. The results of the group 2 dataset using PAC learning parameters show that the model was able to effectively generalize to unseen data, as shown in Figure 6A. The empirical error and true error are close to each other, which means that the model performed well on both the training and test data, and there was no over-fitting issue, as shown in Figure 6B.