# A Hybrid CNN-LSTM Model for SMS Spam Detection in Arabic and English Messages
## 3 The Proposed Model
### Machine Learning Modeling
#### 3.3.2 Deep Learning
* The confusion matrix is a table which indicates the following measures:
* True Positives (TP): the cases when the actual class of the message was 1 (Spam) and the predicted is also 1 (Spam)
* True Negatives (TN): the cases when the actual class of the message was 0 (Not-Spam) and the predicted is also 0 (Not-Spam)
* False Positives (FP): the cases when the actual class of the message was 0 (Not-Spam), but the predicted is 1 (Spam). * False Negatives (FN): the cases when the actual class of the message was 1 (Spam) but the predicted is 0 (Not-Spam). * Accuracy: is the number of messages that were correctly predicted divided by the total number of predicted messages. \[Accuracy=\frac{(TP+TN)}{(TP+FP+FN+TN)}\] (7)
* Precision: is the proportion of positive predictions (Spam) that are truly positives. \[Precision=\frac{TP}{TP+FP}\] (8)
* **Recall**: is the proportion of actual Positives that are correctly classified. \[Recall=\frac{TP}{TP+FN}\] (9)
* F1-Score: is the harmonic mean of precision and recall. \[F1-Score=\frac{2\times Precision\times Recall}{Precision+Recall}\] (10)
* Receiver Operating Characteristic (ROC) is the plot of the true positive rate (TPR) [Equation (11)] against the false positive rate (FPR) [Equation (12)] at various threshold settings. \[TPR=\frac{TP}{TP+FN}\] (11)

Figure 4: An example of an Arabic spam message. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{3}{c}{**Number of the Messages**} & **Training Size** & **Testing Size** \\ \hline Spam & 785 & & \\ Not-Spam & 7519 & 80\% & 20\% \\ Total & 8304 & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Statistics of the dataset. \[FPR=\frac{FP}{FP+TN} \tag{12}\]
* AUC (Area under the ROC Curve) is an aggregate measure of performance across all possible classification thresholds. ### Parameters of the Deep Learning Layers

In the deep learning part of our model, we tested three architectures: CNN, LSTM, and a combination of CNN and LSTM together. In Figures 5-7, we present using the Keras API the parameters of these three architectures. ### Experimental Results

To classify the text messages into Spam and Not-Spam, we tried in this paper several machine learning algorithms: (i) Support Vector Machine, (ii) K-Nearest Neighbors, (iii) Multinomial Naive Bayes, (iv) Decision Tree, (v) Logistic Regression, (vi) Random Forest, (vii) AdaBoost, (viii) Bagging classifier, (ix) Extra Trees, (x) CNN, (xi) LSTM, and (xii) a hybrid CNN-LSTM algorithm. To train these classifiers, we used the same distribution of data for all algorithms: 80% for training and 20% for the test. We calculated five measures to compare the performance of the classifiers: Accuracy, Precision, Recall, F1-Score, and ROC_AUC. In Table 3 and Figure 8, we show the results that were found for all of the previously mentioned algorithms. Starting by the accuracy, the hybrid model CNN-LSTM gave the best score with a value equal to 0.983745, followed by the CNN algorithm with an accuracy of 0.981939. For the precision, Random Forest was the best with a value equal to 1, CNN-LSTM is in fourth position with a precision of 0.953947. Concerning the Recall, CNN-LSTM and Multinomial Naive Bayes share the best score with a value equal to 0.878788. Subsequently, for the F1-Score, CNN-LSTM is once again the best with a score of 0.914826 against 0.905063 for the CNN algorithm. Finally, for the AUC measurement, our CNN-LSTM model gave the best score with a value of 0.937054, Multinomial Naive Bayes is the second with a score of 0.934046. In Figures 9 and 10, we present the confusion matrix and the ROC curve of the CNN-LSTM model. In conclusion, the hybrid model that is based on CNN and LSTM that we have proposed in this paper, has shown its effectiveness by giving the best result in four among five evaluation measures. The obtained results prove that a good combination of deep learning algorithms, as we have done in this paper, represents a very promising solution to create efficient systems for text messages classification and spam detection.