# Detection of Suicide Ideation in Social Media Forums Using Deep Learning
## 4 Methodology
### Proposed Network Model
The LSTM layer structure with  
Figure 2: LSTM-CNN Architecture for Suicide Ideation Detection. input sequences X = (\(x_{t}\)) with a \(d\)-dimensional word embedding vector, while H represents the number of LSTM hidden layer nodes [46]. \[f_{t}=\sigma(W_{f}x_{t}+U_{f}h_{t-1}+b_{f}) \tag{2}\]  
\[i_{t}=\sigma(W_{i}x_{t}+U_{i}h_{t-1}+b_{i}) \tag{3}\]  
\[o_{t}=\sigma(W_{o}x_{t}+U_{d}h_{t-1}+b_{o}) \tag{4}\]  
\[u_{t}=tanh(W_{u}x_{t}+U_{u}h_{t-1}+b_{u}) \tag{5}\]  
\[c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot U_{t} \tag{6}\]  
\[h_{t}=o_{t}\odot tanh(c_{t}) \tag{7}\]  
In above mentioned equations, \(\delta\) stands for a logistic sigmoid function, while \(\circlearrowleft\) represents element-wise multiplication. \(W_{f}\) and \(U_{f}\), two weights matrices, and \(b_{f}\) a bias vector are applied for the forget gate \(f_{t}\) which are similar for the input gate \(i_{t}\), memory cell \(c_{t}\), tanh layer \(u_{t}\), output gate \(o_{t}\) and hidden state \(h_{t}\). Forget gate controls the information sent to the memory cell. This data selection is decided by sigmoid function. Input gate selects which new information will be kept in the memory cell. Memory cell stores the data at each step, and this way ensures long-distance correlations with new input. After the information is updated or ignored through sigmoid layer, tanh layer decides information's level of importance (\(-1\) to \(1\)). These two values are multiplied to update the memory of the new cell state. It is then added to old memory \(c_{t-1}\) resulting in \(c_{t}\)[50; 51]. In the output gate, the amount of information from the internal memory cell is exposed based on the output cell state which is expressed by the hidden unit \(h_{t}\) at time \(t\) which later will be fed to the CNN layer. _Convolutional Layer_  
Convolutional layer is a part of CNN neural network initially designed for an image recognition with a strong performance ability [52; 53]. In recent years, however, CNN has become an incredibly versatile model used for a wide range of multiple text classification tasks with considerable results [54; 32; 55]. When applying CNN on a well-structured and organized text, the model will discover and learn patterns that would otherwise be lost in a feed-forward network. For instance, a word \"down\" in the context of \"down to earth\" and \"feeling down\" has a different sentiment. In addition, CNN can extract features regardless of where they occur in a sentence [56]. CNN is similar to Feed-forward Neural Networks where the connections between the nodes do not form a cycle. Thus, a single neuron in CNN represents a region within an input sample such as a piece of image or text, in our convolution layer we follow the work by [46]. After each feature sequence is extracted by the LSTM model which is H = \([h_{1},h_{2},h_{3},...,h_{T}]^{T}\) where \(h_{t}\) stands for a m-dimensional feature vector of the \(t^{th}\) word in the text sequence where T is the number of LSTM expansion steps equal to the text sequence length. \(H\in R^{m\times T}\) is the CNN input matrix with fixed-length inputs; thus, every input length is standardized to T by trimming the longer sentences and padding the shorter sentences with zeros. The convolutional filter is \(F\in R^{j\times k}\) where j is the number of the words in the window, k is the dimension of the word embedding vector. The convolutional filter F = \([F_{0},F_{2},...,F_{m-1}]\) will generate one value as follows at time step \(t\). Equation (8):  
\[O_{F_{i}}=ReLU[(\sum_{i=0}^{m-1}h_{I+i}^{T}F_{i})+b] \tag{8}\]  
where \(b\) is a bias, and \(F\) and \(b\) are the parameters of this single filter. Finally, a feature map is generated on which ReLU activation function is applied to remove non-linearity. Its mathematical expression is as follows:  
\[X=[x_{1};x_{2};x_{3};...;x_{T}]^{T\times d} \tag{1}\]0In our experiment, we use multiple convolutional filters with various parameter initializations to extract multiple maps from the text [46].