# Social Botomics: A Systematic Ensemble ML Approach for Explainable and Multi-Class Bot Detection
## 4 Methodology
### Binary Bot Classification  
As discussed in Section 2, since we go beyond the conventional binary bot detection approach, we need to define a basic model which will serve as our Baseline Binary Bot detection classifier, namely BBC. Our initial goal is to select the classification algorithm that will ultimately be chosen for further experiments.To achieve this, we experimented with several classification algorithms, namely Logistic Regression (LR), Linear Discriminant Analysis (LDA), Decision Tree Classifier (CART), Multi Layer Perceptron (MLP), AdaBoost (ADA), and Random Forest (RF). We split our dataset into a 75-25% ratio for training and testing the models. For each of these algorithms we performed a GridSearch process for hyperparameter tuning and better parameter selection. The produced models were tested on the test sample (previously unseen data) and they were evaluated by measuring the accuracy using 10-fold cross validation. Eventually, our analysis proved that Random Forest had a clear dominance compared with the other algorithms. Its performance has also been recognized as the most reliable, as discussed in similar research [5; 8; 9; 42; 56], and is well aligned with an explainable approach for classification in bot detection experiments. We have included the results of other classification algorithms in Figure 4.  
Beyond the bot classification baseline, we followed a proper imbalance handling approach to tune and adjust the uneven amount of instances, as described in Section 3.1. To tackle this problem, right after the selection of the proper classification algorithm,  
Figure 4: Random Forest performs better than the other classifiers. The figure on the right shows, that apart from highly accurate predictions, Random Forest is among the most stable, with relatively small fluctuations in the 10-fold validation test. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Feature Category** & **Number of Features** & **Average Calculation Time** & **Version** \\ \hline \(U\) & 28 & 0.73 s & Full and Pruned \\ \hline \(T\) & 69 & 0.19 s & Full \\ \hline \(T\) & 29 & 0.14 s & Pruned \\ \hline \(N\) & 62 & 0.015 s & Full \\ \hline \(C\) & 190 & 5.02 s & Full \\ \hline \(C\) & 182 & 7.2 s & Pruned \\ \hline \(S\) & 58 & 0.42 s & Full and Pruned \\ \hline \(H\) & 13 & 0.003 s & Full and Pruned \\ \hline \multicolumn{3}{c}{_Total_} & full: 420/pruned: 309 & full: 6.4/pruned: 8.5 & - \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Overview of feature characteristics per version. The abbreviations of the first column refer to the features category, see Section 4.1. The social neighborhood feature category (N) is available in the full-features version. The pruned-features version is used for accounts that either have no tweets or no retweets. we experimented with three balancing methods, namely SMOTE-Tomek [64], SMOTE-ENN [65] and ADASYN [63], to select the one that guaranteed a realistic equalizing of cases while maintaining an optimal performance. More specifically, in Table 4, we present the performance metrics for the BBC classifier, using the three different imbalance handling methods that we have already mentioned. Our experimentation runs showed that ADASYN performs better than the other two, preserving a balance between precision and over-fitting. Next we trained our algorithm on two different datasets, as defined in the feature extraction process. The first dataset includes the users that comply with all the limitations posed by the full-features set, while the second includes users that have at least two tweets or retweets and can be used to extract the shortened feature set. The two final datasets are the following:  
1. Full-features Dataset : 46,819 (28,449 humans--18,370 bots)
2. Pruned-features Dataset : 68,437 (30,435 humans--38,002 bots)  
Finally, with respect to the efficiency of our model, we used different feature combinations, among the ones that were identified earlier.