# SMS Spam Detection System Based on Deep Learning Architectures for Turkish and English Messages
## 3 Proposed Model
The number of neurons and activation function selections were carefully conducted with the aim of preventing any kind of overfitting problem in the datasets used in the study. The proposed model contains a low-level convolutional component learning from short-term temporal dependencies and a high-level recurrent component processing information during a long-term period. The inputs of the model are message data and the outputs are the predictions produced by the structures. Figure 3 explains how the CNN + GRU approach operates. The output of the preprocessing layer is connected with a CNN layer with the dimensions of \"Convolution 1D\", as explained above. The dimensions of the CNN layer output are reduced by the \"Max-Pooling\" layer. Then, a GRU layer is added to the system. Finally, sigmoid is used as the activation function for the binary classification of the data. The role and structure of each layer are described in detail as follows. Consider the vector signified by the \(i\)th word constituting any text message within the dataset as \(z_{i}\in R^{d},\ z\in R^{L}\). The value signified by d represents the input message and R represents the word vector. In other words, the expression \(R^{d}\) means a d-dimensional word vector. Here, consider the input message as d and message length as L. Contemplate a k-consecutive-word vector and \(w_{s}\) as a window vector regarding each position f in the text message, as follows:  
\[W_{s}=[z_{s},z_{s+1},\ldots\ldots,z_{s+k-1}] \tag{1}\]  
Figure 2: Steps in word embedding. The convolution layer involves a filter \(p\in R^{kd}\) implemented on the w window in order to produce a new feature map \(c\in R^{L-k+1}\), which is included in the convolution layer. For the window vector represented by \(w_{s}\), each feature element \(c_{s}\) is calculated as follows:  
\[c_{s}=f(w_{s}\otimes p+b) \tag{2}\]  
Here, an element-basis product is considered. \(b\in R\) is a term and f is a non-linear function. In the study, the ReLU function was used as the activation function [52]. This function is defined as follows:  
\[f(x)=MAX(0,x) \tag{3}\]  
The CNN method used in the extraction of features from the text data cannot establish a relation between the present and past information. This process can be achieved with a \"gated recurrent unit\", which is another deep learning method. The GRU was designed to solve the vanishing gradient problem brought by an RNN. It is considered an LSTM variation. The results offered by these two methods bearing similar structures are equally perfect in certain instances. The GRU algorithm involves fewer numbers of tensor operations compared to the LSTM algorithm. In consequence, GRU operates faster in training the model compared to the LSTM algorithm. These two algorithms have advantages and disadvantages dependent on the situation in which they are used. Differing from LSTM, the GRU method is generated by only two gates, with no internal cell state. In the case of the internal cell state, data in a GRU recurrent unit are added into the hidden state of the GRU. As presented in Figure 4, the update gate and reset gate are two different gates of a GRU. The data transmitted to the GRU neural network are the data at time t. The input data are \(z_{1}\), \(z_{2}\), \(z_{3}\), \(\ldots z_{t}\)[53]. \[h_{t}=\sigma(V_{h}*[r_{t-1},z_{t}]) \tag{4}\]  
\[m_{t}=\sigma(V_{n}*[r_{t-1},z_{t}]) \tag{5}\]  
The output of the reset gate at time t is denoted by \(h_{t}\), and the output of the update gate at time t is denoted by \(m_{t}\). With the input of the update gate, the weight at the time \(r_{t-1}\) is indicated by \(V_{n}\). With the entry of the reset gate, the weight at the time \(r_{t-1}\) is indicated by \(V_{k}\). In the formula, the standard GRU output at time t \(-\) 1 is denoted by \(r_{t-1}\). In the dense layer, text messages are classified according to the output of the GRU layer. The sigmoid function was used here. The sigmoid function returns a value between 0 and 1 [54].