# SMS Spam Detection System Based on Deep Learning Architectures for Turkish and English Messages
## 5 Results  
In this study, CNN, GRU, and a hybrid CNN + GRU deep learning algorithm were implemented in order to classify text messages as unsolicited and solicited. The experiments were conducted by using the same data distribution in all algorithms. The accuracy, recall, precision, and F1-score metrics were used to compare the performances of the classifiers. Table 6 demonstrates a comparison between the results of the studies in the literature and the proposed model in terms of English SMS spam detection; while Table 7 demonstrates a comparison between the results of the studies in the literature and the proposed model in terms of Turkish SMS spam detection. For our model, on the other hand, we obtained an accuracy value on a dataset generated by combining both datasets. Moreover, our purpose in using a hybrid method was to use deep learning models more efficiently as well as to resolve the issue of long training times, which is frequently encountered in some deep learning algorithms, and to prevent the overfitting problem. Furthermore, the statistics of the proposed algorithms are presented in Figure 5. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Reference** & **Recall** & **Accuracy** & **F1-Score** & **Precision** \\ \hline CNN [32] & 0.91 & 0.98 & 0.94 & 0.97 \\ CNN + LSTM [35] & - & 0.98 & 0.94 & - \\ CNN [36] & 0.90 & 0.96 & 0.94 & - \\ BIISTM [37] & 0.91 & 0.93 & 0.94 & 0.96 \\ KNN + SVM + RSO [39] & - & 0.99 & - & - \\ Naive Bayes [43] & - & 0.98 & - & - \\ CNN & 0.89 & 0.98 & 0.93 & 0.96 \\ GRU & 0.85 & 0.98 & 0.92 & 0.99 \\ Proposed Model & 0.99 & 0.99 & 0.99 & 0.99 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of the algorithms on the English dataset. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **CNN** & **GRU** & **CNN + GRU** \\ \hline Feature Layer & Fully Conn. & Fully Conn. & Fully Conn. \\ Embedding Size & 32 & 32 & 32 \\ Dropout & 0.15 & 0.15 & 0.15 \\ Epoch & 80 & 80 & 80 \\ Learning Rate & 0.1 & 0.01 & 0.001 \\ Classifying Layer & Fully Conn. & Fully Conn. & Fully Conn. \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Parameters of the deep learning layers. A total of 956 messages among 965 non-spam messages and 659 messages among 665 spam messages in the test dataset were correctly detected by our model. The confusion matrix is presented in Figure 6. The confusion matrix holds the comparison between the predictions made by the model in the dataset and the actual results. The strengths and weaknesses of the model can be comprehensively understood through the comparison matrix. In the experimental study conducted, it was determined that the learning process was finalized at the 72 nd epoch. A total of 80 epochs and five iterations in each epoch were used in the study. The ROC curve obtained as a result of the study is shown in Figure 7.  
The hybrid CNN + GRU approach achieved a value of 0.990797, the CNN algorithm 0.981933, and the GRU algorithm 0.980803 in the accuracy metric. In the precision metric, on the other hand, the GRU algorithm yielded the best result, with a value of 0.997219, while the CNN algorithm achieved 0.964571 and the CNN + GRU algorithm attained 0.990673. The CNN + GRU algorithm performed best on the recall metric with a value of 0.993762, the CNN algorithm obtained 0.899287 while the GRU algorithm achieved a value of 0.854675. Finally, on the F1-score, CNN + GRU performed best compared to the other algorithms, with a value of 0.992215. The F1-Score value was 0.920461 for the GRU algorithm and 0.930250 for the CNN algorithm. The CNN + GRU model's ability to capture complex expressions renders it more advantageous in comparison with the other models in detecting SMS spam messages, while maintaining both high precision and recall. This demonstrates that CNN + GRU stands out as the most reliable model. Offering less computational complexity makes this model essentially significant in environments where minimizing false positives and false negatives is of critical importance. This result reveals  
Figure 5: Results of algorithms.