# Enhancing Detection of Arabic Social Spam Using Data Augmentation and Machine Learning
## 5 Discussion  
The research framework presented in this paper applied different methods to improve Arabic spam detection on social networks, considering Twitter as a case study. As shown in the first experiment Section 4.1 which follows the current state of the art of research work in the field of Arabic spam detection, only one of the content features (TF/IDF) was used with different classifiers on the extracted dataset (this is used as a baseline method to be compared with the next experiments). The best classifier tends to predict true non-spam rather than true spam. With recall rates of 29%, 31%, and 55% for NB, LR, and SVC, respectively, which is not good compared to the current research work. Therefore, in the second experiment Section 4.2, more features are considered for different categories, such as content, interaction, and user features. This led to predicting more true spam tweets. The recall rates for the correct classification of the spam class increased in LR and SVC, with rates of 59% and 65%, respectively, and the macro F1 score of the LR and SVM classifiers improved by approximately 4% to 7%, while the AUC of the three classifiers improved by approximately 3% to 4 %.  
The main problem when applying those classifiers was the unbalancing between spam and non-spam tweets. Therefore, we considered data augmentation to increase the number of spammed tweets, resulting in 6228 total tweets. Table 5 shows the results achieved by experiment Section 4.3, which reveals that adding an augmented dataset outperforms the two previous experiments, and achieved the maximum macro F1 score value of 89%, an accuracy value of 92%, and an AUC score value of 96%. The results achieved outperform the work [53], which already targets Arabic religious hate speech classification, which is part of the spam content categories that we use. This work is closest to our work considering that they used the same classifiers and an Arabic dataset of similar size, where they reached 79% accuracy and 84% AUC. Another previous work [50] that already targets Arabic spam tweets detection using the same classifiers and considering content features,  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{8}{c}{**Augmented Dataset**} \\ \cline{2-9}  & **Class** & **Precision** & **Recall** & **F1** & **Macro-F1** & **Accuracy** & **AUC** \\ \hline \multirow{2}{*}{NB} & 0 & 0.78 & 0.78 & 0.78 & 0.85 & 0.885 & 0.85 \\  & 1 & 0.92 & 0.92 & 0.92 & 0.85 & 0.885 & 0.85 \\ \hline \multirow{2}{*}{LR} & 0 & 0.81 & 0.85 & 0.83 & \multirow{2}{*}{0.87} & \multirow{2}{*}{0.913} & \multirow{2}{*}{0.95} \\  & 1 & 0.95 & 0.93 & 0.94 & & & \\ \hline \multirow{2}{*}{SVC} & 0 & 0.76 & 0.93 & 0.84 & \multirow{2}{*}{0.89} & \multirow{2}{*}{0.921} & \multirow{2}{*}{0.96} \\  & 1 & 0.98 & 0.92 & 0.95 & & & \\ \hline \hline \end{tabular}
\end{table}
Table 5: Augmented dataset—Experiment 3 confusion matrix.  
Figure 6: Plot of augmented dataset results (Experiment 3).  
the proposed work outperforms that work, and for a fair evaluation, their dataset, which contains 3503 tweets, is less than our dataset, where they achieved an accuracy rate of 87%.  
Unlike the non-augmented dataset experiments, the confusion matrix of LinearSVC for the augmented dataset tends to predict true spam and improve the percentage of detected spam tweets. In addition, we have noticed that the data augmentation technique is effective, bringing a consistent improvement to the baseline classifiers. The spam class correct classification has increased and is close to the non-spam class rates, with recall rates of 78%, 85%, and 93% for NB, LR, and SVC, respectively. As a result, the Macro F1 score of all classifiers is improved by approximately 28% to 32% and up to 38% in spam class recall, and the AUC of all classifiers is improved by around 20% to 26%. Figure 7 shows the ROC curve analysis of the three experiments.