## 3 Materials and Methods
### Standard LSTM  
LSTM networks, first introduced by [PERSON] and [PERSON] in 1997, are a specialized type of recurrent neural network (RNN) designed to address the vanishing gradient problem. Unlike traditional RNNs, which struggle to capture long-range dependencies in sequential data, LSTMs incorporate a memory cell mechanism that selectively retains and discards information over extended time steps. This key advantage makes LSTMs particularly effective for applications such as time-series forecasting, natural language processing (NLP), speech recognition, and anomaly detection in IoT systems. The LSTM architecture, as illustrated in Figure 3, consists of multiple gates and memory cells that regulate the flow of information. In Figure 3, the architecture visually represents how the information propagates through the LSTM unit. The blue circles indicate elements-wise operations such as multiplication and addition, while the yellow sigmoid and _tanh_ activations control information processing. The previous hidden state and input are processed through weight matrices, and updated values propagate through the memory cell, ultimately generating \(h_{t}\) and \(c_{t}\)[55; 56; 57; 58]. Unlike conventional RNNs, which rely solely on hidden states, LSTMs maintain a cell state (\(c_{t}\)) that enables long-term storage of relevant information. This memory cell is updated through three primary gates: the forget gate, the input gate, and the output gate. Each of these components is governed by specific activation functions and weight matrices. The _forget gate_ (\(f_{t}\)), defined by Equation (28), determines how much of the past cell state  
Figure 3: Standard LSTM architecture demonstrating the flow of information through forget, input, and output gates. (\(c_{t-1}\)) should be retained or discarded. It takes as input the previous hidden state (\(h_{t-1}\)) and the current input (\(x_{t}\)), applies a sigmoid activation function, and produces an output between 0 and 1. If the value is close to 0, the past memory is mostly forgotten; if it is close to 1, the memory is preserved. \[f_{t}=\sigma(\textbf{W}_{hf}h_{t-1}+\textbf{W}_{if}x_{t}+\textbf{B}_{hf}+ \textbf{B}_{if}) \tag{28}\]  
where \(f_{t}\) denotes the forget gate; \(\textbf{W}_{hf}\), \(\textbf{W}_{if}\) are weight matrices; \(\textbf{B}_{hf}\), \(\textbf{B}_{if}\) are bias terms; and \(\sigma\) represents the sigmoid activation function. The _input gate_ (\(i_{t}\)) and the _candidate memory update_ (\(g_{t}\)), given in Equations (29) and (30), work together to decide how much new information should be stored in the memory cell. The input gate applies a sigmoid activation function, while the candidate update uses a hyperbolic tangent function to generate new candidate values. These values are then multiplied element-wise to regulate how much information should be added to the cell state. \[g_{t}=\tanh(\textbf{W}_{hg}h_{t-1}+\textbf{W}_{ig}x_{t}+\textbf{B}_{hg}+ \textbf{B}_{ig}) \tag{29}\]  
\[i_{t}=\sigma(\textbf{W}_{hi}h_{t-1}+\textbf{W}_{ii}x_{t}+\textbf{B}_{hi}+ \textbf{B}_{ii}) \tag{30}\]  
where \(g_{t}\) denotes the candidate generation, \(i_{t}\) represents the input gate; \(\textbf{W}_{hg}\), \(\textbf{W}_{hi}\), \(\textbf{W}_{ig}\), \(\textbf{W}_{ii}\) are weight matrices; and \(\textbf{B}_{hg}\), \(\textbf{B}_{hi}\), \(\textbf{B}_{ig}\), \(\textbf{B}_{ii}\) are bias terms. The cell state update, expressed in Equations (31) and (32), combines the previous cell state (\(c_{t-1}\)) and the newly computed candidate state (\(g_{t}^{\prime}\)), which is derived by multiplying the candidate state with the input gate. The forget gate determines how much of the past information is retained, while the input gate modulates the newly added information. \[g_{t}^{\prime}=g_{t}\odot i_{t}, \tag{31}\]  
\[c_{t}=(f_{t}\odot c_{t-1})+g_{t}^{\prime} \tag{32}\]  
where \(g_{t}^{\prime}\) is the modulated candidate state; \(c_{t}\) is the updated cell state; and \(\odot\) represents the element-wise multiplication operator.