## 3 Proposed Methodology
### Proposed Hybrid LSTM-GRU Model  
LSTMs have a separate cell state and hidden state, giving them robust control over memory [25]. However, this can lead to retaining information that might not be needed. GRUs have a simpler gating mechanism and combine the cell state and hidden state, making them computationally efficient but somewhat limited in handling long-term dependencies [24]. By combining both LSTM and GRU mechanisms, this model allows for more granular control over which information is retained or discarded. This makes it more adaptable to different types of sequences, where both short- and long-term memory might be needed. The structure of the hybrid model is shown in Figure 5.  
The reset gate in this hybrid model functions as it does in a GRU, deciding how much of the previous hidden state \((h_{t-1})\) should be \"reset\" or ignored when computing the candidate state. This allows the model to selectively drop irrelevant information from previous states. \[r_{t}=\sigma(W_{r}\cdot[h_{t-1},x_{t}]+b_{r}) \tag{11}\]  
Figure 5: Proposed architecture of LSTM-GRU hybrid model. where \(h_{t-1}\) and \(x_{t}\) represent the previous hidden state and input, respectively; \(W_{r}\) represents weight matrices for the reset gate; and \(b_{r}\) represents the bias term. Like the forget gate in a typical LSTM cell, the hybrid model's forget gate operates similarly. It determines the proportion of the previous cell state \((C_{t-1})\) that should be \"forgotten\" or discarded. Every data point in the cell state \((C_{t-1})\) is given a value between 0 and 1 after applying a sigmoid activation function to the previous hidden state \((h_{t-1})\) and the output from the reset gate \((r_{t})\). \[f_{t}=\sigma(W_{f}\cdot[h_{t-1},r_{t}]+b_{f}) \tag{12}\]  
where \(r_{t}\) represents the reset gate and \(W_{f}\) and \(b_{f}\) represent weight matrices and bias terms for the forget gate. The forget gate gives the flexibility to discard unimportant information, helping to mitigate issues like the vanishing gradient problem and enabling the model to remember important information over long time sequences. The update gate \(z_{t}\) controls how much of the hidden state \(h_{t}\) should be updated with new information and how much of the previous hidden state should be retained. This gate blends the previous hidden state with the candidate's hidden state (the new information) to create the next hidden state. \[z_{t}=\sigma(W_{z}\cdot[h_{t-1},x_{t}]+b_{z}) \tag{13}\]  
where \(z_{t}\) represents the update gate and \(W_{z}\) and \(b_{z}\) represent the weight matrices and bias terms for the update gate. The update gate helps maintain the continuity of trends over time while allowing flexibility to adjust for recent changes in time series data. The input gate helps the model decide whether to integrate a recent trend or anomaly into the memory based on its importance for predicting future values. The input gate controls the extent to which the new candidate \(\tilde{c}_{t}\) values (generated for the cell state) should influence the overall memory, allowing for selective integration of new information into the long-term cell state. This is beneficial for remembering important events or trends from the time-series data, allowing the model to capture long-term dependencies. Input gate and candidate cell state equations are given below. \[i_{t}=\sigma(W_{i}\cdot[h_{t-1},z_{t}]+b_{i}) \tag{14}\]  
\[\tilde{C}_{t}=\tanh(W_{i}\cdot[h_{t-1},z_{t}]+b_{i}) \tag{15}\]  
where \(i_{t}\) and \(\tilde{C}_{t}\) represent the input gate and candidate cell state and \(W_{i}\) and \(b_{i}\) represent the weight matrices and bias terms of the input gate. The cell state \(C_{t}\) is updated by integrating the candidate cell state (modulated by the input gate) with the previous cell state (modulated by the forget gate). \[C_{t}=(f_{t}\cdot C_{t-1}+i_{t}\cdot\tilde{C}_{t}) \tag{16}\]  
The output gate regulates the output based on both the update gate and the hidden state.