## 3 Materials and Methods
### Methodology
#### 3.4.6 Predictive Model: Long Short-Term Memory (LSTM) Architecture  
LSTM is a widely used artificial recurrent neural network (RNN) model to deal with sequential data. Since LSTM process the single data point and the sequential data points, it is efficient to train sequential data using LSTM. Some examples of sequential data points are text dataset, time-series dataset, voice dataset and video dataset. Sequential data maintains long-term dependencies, whereas LSTM is capable of learning long-term dependencies. LSTM is a modified version of RNN and RNN is capable of remembering previous data points. The basic architecture of LSTM is followed by the RNN model. An RNN architecture consists of three layers: input layer, hidden layer, and output layer [59; 60]. The fundamental state (current state) of RNN architecture is as follows:  
\[h_{t}=f(h_{t-1},x_{t};\theta) \tag{9}\]  
Here, \(h_{t}\) refers to the current hidden state, \(f\) refers to the function of the previous hidden state \(h_{t-1}\) and the current input \(x_{t}\), \(\theta\) refers to the parameters of that function. The primary mechanism of an RNN architecture is that the hidden layer's input is the current input and output derived from the earlier hidden state. Therefore, the hidden layer works as conditional neural memory and remembers the sequential data. The process is shown in a textual format and in Figure 1:  
The drawback of the RNN architecture is that it forgets the necessary data when a very large dataset is used. The nature of time-series data is that the current data depends on the previous data. Hence, time-series data has long-term dependency over time. The LSTM model proposed by [PERSON] [61] is an improved version of RNN that can memorize the long-term dependency data by forgetting the unnecessary data and memorizing the necessary data at every updatio step of gradient descent [62]. LSTM  
Figure 1: Recurrent neural network (RNN) structure. architecture comprises of four parts: a cell, an input gate, an output gate, and a forget cell [32]. The forget cell forgets the unnecessary data and remembers only the necessary data. The forget gate is responsible for deciding which information should be discarded based on the state \(h(t-1)\) and input \(x(t)\) at the state \(c(t-1)\). The forget gate's sigmoid function keeps all 1s and discards all 0s between 0 and 1 values at each cell state. The value '1' is considered as the necessary value, and '0' is considered as the unnecessary value [14; 61; 63; 64]. The equation of the forget gate state is as follows:  
\[f_{t}=\sigma(W_{f}.[h_{t-1},x_{t}]+b_{f}) \tag{10}\]  
where \(f_{t}\) refers to the current forget state, \(\sigma\) refers to the sigmoid activation function, \(W_{f}\) refers to the weights of the forget gate, \(h_{t-1}\) refers to the output from the previous hidden state, \(x_{t}\) refers to current input and \(b_{f}\) refers to the bias of the forget gate function. After forgetting the unnecessary value, new values need to be updated in the cell state. The process has three parts:  
1. A sigmoid layer called the \"input gate layer\" decides which values to update. 2. A tanh layer creates a vector of new candidate values to add to the state. 3. Combination of step 1 and 2 creates an update to the state. The sigmoid layer state's equation is: \[i_{t}=\sigma(W_{i}.[h_{t-1},x_{t}]+b_{i})\] (11)  
Sigmoid layer from Equation (11) decides which value should be updated; tanh layer from Equation (12) creates a vector of new candidates for creating a new value to the state \(C(t)\). The tanh layer's state equation is:  
\[\tilde{C}(t)=\tanh(W_{c}.[h_{t-1},x_{t}]+b_{C}) \tag{12}\]  
The updation of the new cell at state \(C(t)\) occurs by adding \(\tilde{C}(t)*i_{t}\) with \(C_{t-1}*f_{t}\). The updation state's equation is:  
\[C_{t}=C_{t-1}*f_{t}+\tilde{C}(t)*i_{t} \tag{13}\]  
Finally, the output is filtered out by a sigmoid Equation (14) and a tanh Equation (15) function to decide which output needs to be kept.