## 4 Stock Price Prediction Model Methodology and Research Design
### Model Architecture  
As discussed earlier, deep supervised learning models have demonstrated superior performance on the benchmark LOB dataset compared to other methods. As confirmed in the work by [PERSON] et al. [15], convolution layers are very helpful for handling high-dimensional data such as LOB and OF tensors. They serve to automatically extract the most relevant features from these data. To fully account for the time-series nature of the data, recurrent layers are added to the architecture. These layers facilitate the transfer of hidden state information from prior observations to the next ones. These two types of layers, in addition to a standard max-pooling layer applied to make the model less sensitive to small changes in the input data and a dense layer as an output layer, are the key building blocks of the proposed deep neural network architecture. In Figure 8, the architecture of the proposed model is depicted at a conceptual level, whereas Figure A1 in Appendix A depicts the detailed architecture. The input layer receives processed and normalised data in the form of tensors with a shape of [100; 70]. In these R100 \"\(\times\)\" R70 tensors, R100 corresponds to the 100 most recent timestamps, and R70 relates to the limit order book and order flow features. These input data are transmitted through nine consecutive convolution layers, each with 32 filters, followed by a max-pooling layer (kernel size = 1 \(\times\) 8; stride = 1 \(\times\) 1), forming the first block of convolution layers. The kernel size of the first and sixth layers is 1 \(\times\) 2; the kernel size of the second, third, fifth, sixth, eighth, and ninth layers is 4 \(\times\) 1, and the kernel size of the seventh layer is 1 \(\times\) 10. The second, third, fifth, sixth, eighth, and ninth layers have the same padding. All the layers in the first block have a standard stride of 1 \(\times\) 1, except for the first and fourth layers, which have a stride of 1 \(\times\) 2. For all the convolution layers, the parametric rectified linear unit activation function is applied. It was first introduced by [PERSON] et al. [38] and is a generalisation of the popular rectified unit activation function originally presented by [PERSON] et al. [39]. The parametric rectified linear unit activation function multiplies negative input values \(y_{i}\) by coefficient \(a_{i}\) to obtain outputs \(f(y_{i})\), while for positive input values, the output equals the input:  
\[y_{i}>0:f(y_{i}) =y_{i}\] \[y_{i}<0:f(y_{i}) =a_{i}\times y_{i}\]  
The original rectified unit activation function assumes zero output for negative input values, and the output equals the input for positive input values:  
\[y_{i}>0:f(y_{i}) =y_{i}\] \[y_{i}<0:f(y_{i}) =0\]  
Figure 8: Diagram of the TFF-CL-GRU neural network architecture. The motivation behind opting for the PReLU lies in its ability to introduce nonlinearity to the network while addressing certain limitations associated with traditional rectified linear unit (ReLU) activations. Unlike the standard ReLU, which can suffer from the \"dying ReLU\" problem by setting all negative values to zero, the PReLU allows for the adaptation of the negative slope during training. This adaptability contributes to improved convergence during training and better gradient flow, enhancing the model's ability to learn complex representations from financial data. Additionally, the PReLU has demonstrated effectiveness in mitigating the vanishing gradient problem, which can be especially pertinent in deep neural network architectures. Although the parametric rectified linear unit function adds minor extra computation costs to learn parameter \(a_{i}\), it has been demonstrated that it allows for faster convergence of deep learning models such as TFF-CL-GRU. The second block consists of three sub-blocks that are concatenated along axis 3. Each of these three sub-blocks receives the outputs from the max-pooling layer of the first block as input.