## 3 Methodology
### SEBD Architecture and Framework Phases
* An individual hashtag vs. a group of hashtags in a similar location and a similar domain. Thus, using this variety of hashtag statuses/domains/locations, the expected performance of our model was determined. This would help to evaluate its performance with regard to PAC learning parameters to ensure the accuracy and confidence of our model. ## 4 Results and Discussion

### SEBD Evaluation

We evaluated SEBD using common evaluation metrics: accuracy score, F1 score, and MCC. The accuracy score aimed to evaluate the model correctness. Recall indicates the completeness of the model and precision represents the exactness or the predictive power of the model. The F1 score and Matthews correlation coefficient (MCC) were used as more balanced evaluation metrics for the binary classification task. They are defined in the following equations [48]:

\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{5}\]

\[Precision=\frac{TP}{TP+FP} \tag{6}\]

\[Recall=\frac{TP}{TP+FN} \tag{7}\]

\[F1-measure=\frac{2*Precision*Recall}{Precision+Recall} \tag{8}\]

\[MCC=\frac{\text{TP}\times\text{TN}-\text{FP}\ \times\text{FN}}{\sqrt{(\text{TP}+ \text{FP})(\text{TP}+\text{FN})(\text{TN}+\text{FP})(\text{TN}+\text{FN})}} \tag{9}\]

where TP represents true-positive samples, TN true-negative samples, FN false-negative samples, and FP false-positive samples. To evaluate SEBD, we applied four evaluations methods to understand how well the model performs in identifying bot accounts in a stream-based manner. We limited the evaluation to the prediction of labels and excluded the performance evaluation of concept-drift or sliding windows. #### 4.1.1 Evaluation Using Holdout Method

We used the holdout method to test a 30% split of all the data that SEBD predicted, and trained the model with the rest (70% split) using cross-validation (10 folds). We used the whole dataset (group 1 and group 2). We trained a random forest classifier with 100 estimators (trees). The mean F1 score for using cross-validation was 0.962. The performance results are shown in Table 3. SEBD achieved a score of 0.969 for accuracy, of 0.962 for recall, of 0.998 for precision, and of 0.91 for MCC. These results indicate that SEBD is effective at identifying bot accounts. Figure 2 shows the confusion matrix (A) and an AUC curve score of 0.98 (B) for SEBD. The pre-trained model (Bot-MGAT) that SEBD is built on integrates a multi-view graph attention network. It was trained on both fellowship links and interaction links using transfer-learning approach [17]. Unlike GCN or RGCN, the use of multi-views serves on better understanding of the graph structure and the links between the graph's nodes. This helps the model to capture the non-linear relationships between nodes more accurately, which can be useful for classification of graph nodes and link prediction. Table 3 shows SEBD compared to baseline work. In [41], the authors proposed an inductive representation learning technique utilizing GCNN to detect Twitter spam bots. Using a neural network and propagation techniques, they learned the embedding of nodes in the graph structure. They evaluated their approach using bot datasets that had old generations of spam bot accounts, which proved to be easier to detect and distinguish than recent generations of bots [49]. In [40], the proposed framework encodes different categories of characteristics, such as description, profile, and tweet, as numeric features. This approach avoids the need for feature engineering while constructing a heterogeneous graph with various types of

Figure 2: SEBD results using holdout method and cross-validation with random forest. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Ref\#** & **Approach** & **Accuracy** & **F1** & **MCC** \\ \hline [PERSON] et al. [41] & GCNN & 0.94 & 0.84 & - \\ BotRGCN [40] & RGCN & 0.862 & 0.8707 & 0.7021 \\ SEBD & MGAT & 0.969 & 0.98 & 0.91 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: SEBD results compared to baseline work. edges between Twitter users.