## 3 Research Methodology
### Long Short-Term Memory  
Long short-term memory (LSTM) addresses a significant challenge encountered in recurrent neural networks (RNNs), namely the issue of vanishing or exploding gradients [36]. The vanishing gradient problem hinders the ability of recurrent neural networks (RNNs) to effectively learn when there exists a time lag of more than 5-10 distinct time steps between input events and target signals. In contrast, the long short-term memory (LSTM) model demonstrates the ability to establish connections between temporal intervals as short as the minimum time lag and as long as 1000 discrete time steps. This is achieved through the utilization of specialized units known as cells, which consist of constant error carousels (CECs) that enforce a continuous flow of error. Cell access is provided through multiplicative gate units [37].  
The concealed stratum of a conventional long short-term memory (LSTM) network is comprised of memory blocks. A memory block consists of a set of memory cells anda pair of multiplicative gate units that facilitate the transfer of input and output signals to and from all the cells within the block [38]. The memory cell is equipped with a cell error constant (CEC) mechanism that effectively addresses the issue of vanishing gradient error. This mechanism ensures that the local backflow error of the cell remains constant, without diminishing or amplifying, during periods when the cell is not being subjected to new input or error signals. The two gating units, namely the input gate and the output gate, serve the purpose of protecting the CEC (cellular error correction) mechanism from both forward and backward error flow, respectively. The state of the cell is determined by the activation of the CEC [39]. The computation of the activation of the input gate y'in and the activation of the output gate y'out at discrete time steps \(t=1,2,\ldots\) proceeds as follows:  
\[net_{outj}(t)=\sum_{m}w_{outj}y^{m}(t-1),\;y^{outj}(t)=f_{outj}\Big{(}net_{outj }(t)\Big{)} \tag{5}\]  
\[net_{in_{j}}(t)=\sum_{m}w_{in_{j}m}y^{m}(t-1),\;y^{in_{j}}(t)=f_{in_{j}}\Big{(} net_{in_{j}}(t)\Big{)} \tag{6}\]  
where \(j\) represent th memory block, \(f\) is the logistic sigmoid in the range \([0,\;1]\), and \(w_{in}\) is the connection weight from the unit \(m\) to the unit \(l\). In order to compute the internal state of a given memory cell \(S_{c}(t)\), the squashed gate input to the state at the recent time step \(S_{c}(t-1)\) where \((t>0)\) can be added via the following equation:  
\[net_{c_{j}^{c}}(t)=\sum_{m}w_{c_{j}^{c}m}y^{m}(t-1) \tag{7}\]  
\[S_{c_{j}^{c}}(t)=S_{c_{j}^{c}}(t-1)+y^{in_{j}}(t)g\Big{(}net_{c_{j}^{c}}(t) \Big{)} \tag{8}\]  
where \(c_{j}^{c}\) represent cell \(v\) of memory block \(j\), the squashing of the cell input is performed by \(g\), and \(S_{c_{j}^{c}}(0)=0\). In order to determine the output of a cell \(y^{c}\), the internal state \(S_{c}\) is squashed using an output squashing function \(h\) and gating it with the activation of the output gate \(y^{out}\) expressed as:  
\[y^{i^{c}}_{j}(t)=y^{outj}(t)h\Big{(}S_{c_{j}^{c}}(t)\Big{)} \tag{9}\]  
where \(h\) represents a centered sigmoid in the range \([-1,\;1]\). The output units \(K\) of a network with layered topology consisting of a hidden layer with memory blocks and a standard input and output layer can be defined by the equation:  
\[net_{k}(t)=\sum_{m}w_{kn}y^{m}(t-1) \tag{10}\]  
\[y^{k}(t)=f_{k}(net_{k}(t)) \tag{11}\]  
where \(f_{k}\) represents the squashing function with a logistic sigmoid in the range \([0,\;1]\) and \(m\) ranges over all input units and the cells in the hidden layer. LSTM is capable of solving tasks with complex long time lags that were never solved using RNNs.