## 4 Experimental Results and Discussions
\[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+ \text{FN}} \tag{1}\]  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{2}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{3}\]  
\[\text{F}_{1}\text{Score}=\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}} \tag{4}\]  
In our tables, bold line indicates the best results obtained in the corresponding experiment instance. The validation results for the language models trained with these parameters are presented in Table 5, where the best result, 91.56%, belongs to Electra BERT Base. Following this, the BERT Base Turkish language model, which has high performance in Turkish language processing tasks, achieves a score of 91.56%. Looking at the test results in Table 6, it can be observed that language models specifically trained for Turkish outperform multilingual models. The F1 scores for Electra Base Turkish and BERT Base Turkish models are 90.64% and 90.21%, respectively. In contrast, the multilingual models, BERT Base Multilingual and XLM-RoBERTa Base, performed with lower F1 scores of 84.76% and 78.11%, respectively, on Turkish data. The confusion matrices presented in Figure 4 were generated using Python, specifically with the Matplotlib and Seaborn libraries, based on the experimental results obtained in this study. Figure 4 shows the confusion matrices for each language model trained. Upon examining the confusion matrices, it can be seen that the models specifically trained for Turkish make fewer errors and demonstrate better class separation. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Validation Results**} \\ \hline
**Model** & **Accuracy** & **Recall** & **Precision** & **F1** \\ \hline Electra Base Turkish & **91.56\%** & **91.57\%** & **91.61\%** & **91.58\%** \\ \hline Bert Base Turkish & 91.16\% & 91.18\% & 91.17\% & 91.18\% \\ \hline TurkishBERTweet & 88.48\% & 88.50\% & 88.53\% & 88.51\% \\ \hline Bert Base Multilang & 85.96\% & 85.93\% & 86.17\% & 85.99\% \\ \hline Turkish RoBERTa base & 85.72\% & 85.74\% & 85.83\% & 85.77\% \\ \hline XLM- RoBERTa Base & 78.15\% & 78.16\% & 78.14\% & 78.13\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Validation results of the models compared after training with the FSMTSA dataset. The results shown in Table 7 reveal that deep learning-based language models outperform traditional machine learning algorithms. The highest accuracy rate of 88.36% is achieved with the Linear SVC (Support Vector Classifier) algorithm. Due to the linear separability assumption, the logistic regression algorithm may not fully model semantic relationships arising from context and suffixes in Turkish. If this model is not supported by morphological processing, the misclassification rate may increase due to word variations. Logistic Regression follows in second place with an accuracy rate of 87.44%. The other traditional methods such as Naive Bayes and KNN show lower performance, with accuracy rates of 85.77% and 81.05%, respectively. One reason for this result is that root-based feature extraction is not performed, the model cannot fully grasp the meaning changes arising from word context and suffixes in Turkish. This can decrease the model's performance. The KNN algorithm is in fourth place since it works directly at the word level and may struggle to capture the correct similarity between neighbors due to the agglutinative structure of Turkish. This could negatively impact the model's performance. Decision Tree follows in last place with the lowest accuracy rate of 76.03%. Due to word suffixes and syntactic flexibility in Turkish, the tree structure can become very complex. The branching structure may grow excessively, losing its ability to generalize. As a result, the overfitting level of the model may increase.