# Detecting Suspicious Texts Using Machine Learning Techniques
## 5 Experiments
### Discussion  
After analyzing the experimental results, we can summarize that LR, DT, and RF do well with a unigram feature while MNB and SGD obtained maximum accuracy with unigram and bigram feature combination. In both cases, the tf-idf feature extraction technique is employed. Classifiers performed poorly for trigram features. After comparing BoW and the tf-idf extraction technique, we noticed an allusive rise for the weighted features of the texts. This increase happens because the BoW emphasizes the most frequent words only, while the tf-idf emphasizes the context-related words more. LR, RF, MNB, and SGD performed excellently on every feature combined with a little deviation of (0.5-0.8)% between them. However, the performance of the decision tree is inferior compared to others due to its limited ability to learn complex rules from texts. The AUC value is another performance measure that indicates the model's capability to make a distinction between classes. SGD obtained the highest value of 0.893 AUC for the tf-idf, and LR and RF achieved the maximum AUC value of 0.878 for the BoW feature extraction. After analysis, the reason behind the superior performance of SGD classifier is pointed out. Here, SGD represents a linear classifier that is already proven as a well-suited model for binary classification like ours [42]. It uses a simple approach of discriminative learning which can find global minima more efficiently, thus resulting in better accuracy. By comparing these ML classifiers in terms of their execution time, no significant difference has been reported. All the classifiers have completed their execution before the 50 s mark. Since the machine learning-based techniques mainly utilized word-level features, it is difficult to adopt the sentence-level meanings appropriately. The system can not predict the class accurately for this reason. Therefore, to shed light on the tendency for which text is complicated to predict in \begin{table}
\begin{tabular}{c c} \hline \hline
**Techniques** & **Accuracy(\%) on SBTD** \\ \hline SVM + BoW [19] & 81.14 \\ LR + unigram + bigram [49] & 82.07 \\ DT + tf-idf [50] & 77.92 \\ Naïve Bayes [51] & 81.77 \\ LR + BoW [30] & 82.28 \\ Our Proposed Technique & 84.57 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Performance comparison after employing existing techniques on our dataset. suspicious detection, we analyze the predicted results. Consider an example, (_Banglish form:_ \"[PERSON] khela pare na take Bangladesh cricket team theke ber kore dewa dorkar\"). (_English translation:_ [PERSON] cannot play, he needs to be dropped from the Bangladesh cricket team). This text may excite the fans of [PERSON] because it conveys the disgraceful message about him. The proposed approach should classify these texts as the suspicious class rather than the non-suspicious class. Thus, the classification discrepancies happen due to the inability to capture the semantic relation between words and sentence-level meanings of the texts. It is always challenging to classify such type of text because these types of texts did not have any words that directly provoke people or pose any threat. The proposed approach encountered a limited number of such texts during the training phase and hence it failed to predict the class correctly. These deficiencies can be dispelled by employing neural network-based architecture with the addition of diverse data in the existing corpus. Although the result of the proposed SGD based model is quite reasonable compared to the previous approaches, there are scopes to increase the overall accuracy of suspicious Bengali text detection. Firstly, the proposed model did not consider the semantic relation between words in the texts. For this reason, ML-based classifiers show poor accuracy for a non-suspicious class that has long texts. The semantic relationship and corresponding machine learning rule-based model [52; 53] could also be effective depending on the data characteristics.