## 2 Research Framework/Methodology
### Long Short-Term Memory (LSTM)  
Recurrent neural networks (RNNS) are the building blocks of neurons that connect the inputs, hidden layers, and outputs and process the selective parts of sequence data by an activation function at a time t. It processes the sequenced data by getting the previously hidden state ht-1 and new input data \(\mathbf{x}_{t}\), and multiplying it with the weights of inputs, adding up the biases as a feed for the activation function [20]. The architecture of an LSTM model consists of an input gate, output gate, memory gate, and a forget gate, as shown in Figure 4. These gates update the information flow through each block of LSTM. These gates include the activation functions such as the sigmoid function and tanh function, and operations such as addition and multiplication. The forget gate decides how much information needs to be kept or discarded. The memory gate decides the amount of information needed to be stored in the cell state, and the input gate updates the previous information as an input to analyze for the output. The output gate provides the output for the current block by analyzing the cell state. The LSTM model decides which information to keep or discard with the help of these gates. Hence, keeping the information of the previous cell/ frame can overcome the problem of the gradient descent.  
Figure 3: The GoogleNet architecture and Structure of the Inception block.  
For a time t, the updated cell state can be calculated from Equations (2) to (7) [14]. In these equations, \(\delta\) is the activation function sigmoid (defined as \(\delta\)(x) = (1 + e\({}^{-}\)x)\({}^{-1}\)), \(\otimes\) is the pointwise multiplication operation, \(\mathbf{i}_{\text{t}}\), \(\mathbf{f}_{\text{t}}\), \(\mathbf{o}_{\text{t}}\), \(\mathbf{g}_{\text{t}}\), and \(\mathbf{c}_{\text{t}}\) are the input gate, forget gate, output gate, and cell state, respectively, and \(\mathrm{Ws}\) and \(\mathrm{Vs}\) are the coefficient matrixes.  
\[\mathbf{i}_{\text{t}} = \delta(W_{\text{xi}}\mathbf{x}_{\text{t}}+\text{ }V_{\text{hi}}\mathbf{h}_{\text{t}-1}+\text{ }\text{b}_{\text{i}}) \tag{2}\] \[\mathbf{f}_{\text{t}} = \delta(W_{\text{xf}}\mathbf{x}_{\text{t}}+\text{ }V_{\text{hi}}\mathbf{h}_{\text{t}-1}+\text{ }\text{b}_{\text{f}})\] (3) \[\mathbf{o}_{\text{t}} = \delta(W_{\text{xo}}\mathbf{x}_{\text{t}}+\text{ }V_{\text{ho}}\mathbf{h}_{\text{t}-1}+\text{ }\text{b}_{\text{o}})\] (4) \[\mathbf{g}_{\text{t}} = \tan\text{h}\;(W_{\text{xc}}\mathbf{x}_{\text{t}}+\text{ }V_{\text{hc}}\mathbf{h}_{\text{t}-1}+\text{ }\text{b}_{\text{c}})\] (5) \[\mathbf{c}_{\text{t}} = \mathbf{f}_{\text{t}}\,\otimes\,\mathbf{c}_{\text{t}-1}+\, \mathbf{i}_{\text{t}}\,\otimes\,\mathbf{g}_{\text{t}}\] (6) \[\mathbf{h}_{\text{t}} = \mathbf{o}_{\text{t}}\,\otimes\,\tan\text{h}(\mathbf{c}_{\text{t}}) \tag{7}\]  
In general, the input is fed into a single layer LSTM model for the activation and processing of the output. The multi-layer LSTM models are used to boost the process and get better performance for time series problems. In this way, the layers of LSTM are stacked on each other, and each layer received the hidden state of the previous as an input and processed the frame in the same direction [14]. While in bidirectional LSTM (BiLSTM), the output of the current cell depends upon the previous frames as well as the upcoming frame. In BiLSTM, there are two single LSTM stacked on each other with the reverse direction of information exchange. LSTM and bidirectional LSTM (BiLSTM) layers learn unidirectional and bidirectional long-term dependencies, respectively, between time steps in time series and sequence data. The overview of the CNN-BiLSTM structure is shown in Figure 5. The cell's output is computed based on the hidden state of both forward and backward LSTM layers. The output of a frame at a time t is computed from the two consecutive frames, \(\mathbf{t}-1\) and \(\mathbf{t}+1\)[28].