# Important Trading Point Prediction Using a Hybrid Convolutional Recurrent Neural Network
## 4 Deep Learning Framework for Important Trading Point Prediction
### Hybrid Convolutional Recurrent Neural Network
Suppose the hidden state at the previous time step \(t-1\) is \(h_{t-1}\):  
The input gate \(i_{t}\), which determines the allowed number of candidate hidden values \(\tilde{c}_{t}\) updated into the memory cell, is calculated by  
\[i_{t}=\text{sigmoid}(W_{i}\cdot[\text{Conv}_{t},X_{t},h_{t-1}]+b_{i}). \tag{5}\]  
The candidate hidden value \(\tilde{c}_{t}\) is calculated by  
\[\tilde{c}_{t}=\tanh(W_{c}\cdot[\text{Conv}_{t},X_{t},h_{t-1}]+b_{c}). \tag{6}\]  
The forget gate \(f_{t}\), which controls how much previous information should be kept in the new cell, is calculated by  
\[f_{t}=\text{sigmoid}\left(W_{f}\cdot[\text{Conv}_{t},X_{t},h_{t-1}]+b_{f} \right). \tag{7}\]  
Suppose the old information is represented by \(c_{t-1}\); then, the information stored in the memory unit is updated as follows:  
\[c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot\tilde{c}_{t}. \tag{8}\]  
The output gate \(o_{t}\), which defines the proportion of information that can be output, is calculated by  
\[o_{t}=\text{sigmoid}(W_{o}\cdot[\text{Conv}_{t},X_{t},h_{t-1}]+b_{o}). \tag{9}\]  
Then the output of the LSTM cell \(h_{t}\) is expressed as:  
\[h_{t}=o_{t}\odot\tanh(c_{t}), \tag{10}\]where \(\odot\) denotes the element-wise product; the three types of gating units use sigmoid(-) as the activation function, and the hyperbolic tangent tanh(-) is adopted as the activation function for input modulation and output. For convenience, in this study, we use the function LSTM(-,\(r\),) as shorthand for the LSTM model in (11):  
\[(h_{t},c_{t})=\text{LSTM}(\text{Conv}_{t},x_{t},h_{t-1},c_{t-1},W,b), \tag{11}\]  
where \(W\) and \(b\) include all of the weight matrices and bias vectors mentioned in (5)-(9), which are determined in the training process. As illustrated in Figure 4, we adopt stacked LSTM with two layers. The advantages of stacked LSTM are obvious: (1) stacking LSTM layers enables the characteristics of raw temporal data to be learned from different aspects at each time step; (2) the model parameters are distributed over the whole space of the model without increasing the memory capacity, which enables the model to refine the nonlinear operations of raw signals and accelerate convergence. The improved performance is shown in the experiments. For the purpose of avoiding overfitting and achieving better generalization, we apply layers of dropout and batch normalization after each LSTM. The stacked LSTM model can be described as follows:  
\[\left(h_{t}^{1},c_{t}^{1}\right)=\text{LSTM}\,1\left(\text{Conv}_{t},x_{t},h_ {t-1}^{1},c_{t-1}^{1},W^{1},b^{1}\right) \tag{12}\]  
\[x_{j}^{l}=f\left(\sum_{i\in M_{j}}x_{i}^{l-1}*k_{ij}^{l}+b_{j}^{l}\right), \tag{4}\]0  
Then, the output of the second LSTM layer \(h_{t}^{2}\) is fed to the fully connected layer for prediction, and the overall output of the HCRNN model is expressed as  
\[x_{j}^{l}=f\left(\sum_{i\in M_{j}}x_{i}^{l-1}*k_{ij}^{l}+b_{j}^{l}\right), \tag{4}\]1  
where \(W_{2}\) is a weight matrix, \(b_{2}\) is the bias vector, and \(y_{t}\) is a prediction value between 0 and 1, which indicates the probability of belonging to a SIP. Considering that the goal of this task is to classify each time point into one of two categories--belonging to a SIP  
\[x_{j}^{l}=f\left(\sum_{i\in M_{j}}x_{i}^{l-1}*k_{ij}^{l}+b_{j}^{l}\right), \tag{4}\]2 Structure of the proposed hybrid convolutional recurrent neural network (HCRNN) model for important trading point prediction, where detailed layer connections are indicated. (\"1\") and not belonging to a SIP (\"0\")--we set a threshold \(\theta\) to classify the output, which is expressed in (15):  
\[x_{j}^{l}=f\left(\sum_{i\in M_{j}}x_{i}^{l-1}*k_{ij}^{l}+b_{j}^{l}\right), \tag{4}\]3  
where \(s_{t}\) is the prediction type of time point \(t\). This means that if \(y_{t}\) is above the threshold \(\theta\), the prediction type will be defined as a time point belonging to a SIP (\"1\"); otherwise, the prediction type will be defined as not belonging to a SIP (\"0\").