# A Hybrid CNN-LSTM Model for SMS Spam Detection in Arabic and English Messages
## 3 The Proposed Model
### Machine Learning Modeling
#### 3.3.2 Deep Learning
Each unit, at a time step \(t\), is composed of a cell \(c_{t}\) (the memory part of LSTM) and three gates to regulate the flow of information inside the LSTM unit: an input gate \(i_{t}\), an output gate \(o_{t}\) and a forget gate \(f_{t}\). These gates collectively decide how to update the current memory cell \(c_{t}\) and the current hidden state \(h_{t}\). The transition functions between the LSTM units are defined, as follows [19]:

\[ \begin{split} i_{t}&=\sigma(W_{i}.[h_{t-1}+b_{i}])\\ f_{t}&=\sigma(W_{f}.[h_{t-1},x_{t}]+b_{f})\\ q_{t}&=\tanh(W_{q}.[h_{t-1},x_{t}]+b_{q})\\ o_{t}&=\sigma(W_{o}.[h_{t-1},x_{t}]+b_{o})\\ c_{t}&=f_{t}\odot c_{t-1}+i_{t}\odot q_{t}\\ h_{t}&=o_{t}\odot\tanh(c_{t})\end{split}  \tag{5}\]

Here, \(x_{t}\) is the input vector of the LSTM unit, \(\sigma\) is the sigmoid function, \(\tanh\) denotes the hyperbolic tangent function, the operator \(\odot\) denotes the element-wise product, and \(W\), and \(b\) are the weight matrices and bias vector parameters that need to be learned during training. In the architecture of our model, we used a single LSTM layer placed directly after the MaxPooling. This layer contains 64 LSTM units using a Dropout equal to 0.2 as a regularization parameter to prevent the model from overfitting. Algorithm 3 describes the detailed working of the LSTM algorithm. ```
Input:\(x\) Output:\(h\)foreachtime step \(t\)do \(i_{t}=\sigma(W_{i}. [h_{t-1}+b_{i}])\) \(f_{t}=\sigma(W_{f}. [h_{t-1},x_{t}]+b_{f})\) \(q_{t}=\tanh(W_{q}. [h_{t-1},x_{t}]+b_{q})\) \(o_{t}=\sigma(W_{o}. [h_{t-1},x_{t}]+b_{o})\) \(c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot q_{t}\) \(h_{t}=o_{t}\odot\tanh(c_{t})\) end
```

**Algorithm 3**LSTM

**Dense** is the last layer of our model. It is also called the fully connected layer, and it is used to classify text messages according to the output of the LSTM layer. Since our classification model is binary, we used a Dense layer with a single neuron and a sigmoid activation function to give predictions of 0 or 1 for the two classes (Not-spam and spam). The sigmoid function is a logistic function that returns a value between 0 and 1, as defined by the formula in Equation (6):

\[f(x)=\frac{1}{1+e^{-x}} \tag{6}\]

### Classification

Classification is the final step in our model. Regardless of the used machine learning method, the goal here is to classify the text messages given as input to this model into two classes: spam and not-spam. The spam class refers to any undesirable message containing temptations of phishing or theft of information. The not-spam class designates normal messages that represent no danger for the users. ## 4 Experimental Evaluation

In this section, we present the results that were obtained from the experimental tests that we conducted on the model presented in this paper. We also present a comparison between the different machine learning techniques tried in our approach. The implementation of this model was performed in Python 3.7 with the help of the TensorFlow environment and the Keras 2.0 API. ### Dataset Description

For the evaluation of our approach, we used two types of dataset: (i) the SMS Spam dataset from the UCI Repository [25] and (ii) a set of Arabic messages collected from local smartphones. The SMS Spam dataset is a public set of SMSs labeled messages that have been collected for SMS spam research. It contains 5574 English messages that were labeled according being legitimate (ham) or spam. For the second dataset, it contains a set of 2730 Arabic messages that were collected from several local smartphones in Saudi Arabia and which are labeled as spam and not-spam. For example, Figure 4 contains an Arabic spam message attempting to steal the confidential information of the user's bank card. In Table 2, we show some statistics regarding the dataset used. ### Evaluation Measures

To evaluate the performance of the proposed model, we used the standard metrics for classification tasks, such as Accuracy, Precision, Recall, F1-Score, Confusion Matrix, ROC curve, and AUC.