## 5 De-Noising Filter Method and Experimental Results
### De-Noising Filter Model  
Figure 7 presents the LSTM layered architecture. This design demonstrates the flow of a time series X (inputs) with N features of length S through an LSTM layer. The outputs are a time series \"h\" with H hidden units of length S. In the design, \(h_{k}\) and \(c_{k}\) denote the output (hidden state) and the \"cell state\" at time step \(t_{k}\), respectively. The first LSTM block uses the initial state of the network to calculate the first output (\(h_{1}\)) and the \"updated cell state\" (\(c_{1}\)) at the first time. At the \(t_{k}\), the block uses the previous state ( \(h_{k-1}\), \(c_{k-1}\)) to compute the output (\(h_{k}\)) and the \"updated cell state\" (\(c_{k}\)). A LSTM unit is composed of a cell, an \"input gate\", an \"output gate\" and a \"forget gate\". The basic structure of an LSTM unit is shown in Figure 8 illustrating the flow of data at instant \(t_{k}\). This diagram highlights the three parts: forget gate, input gate and output gate. Figure 8: Basic structure of an Long Short Term Memory Unit. Figure 7: LSTM layered Architecture (source: mathworks/lstm). A LSTM layer has the learnable weights and bias: the input weights \(W_{lstm}\), the recurrent weights \(R_{lstm}\) and the bias \(b_{lstm}\). The matrices \(W_{lstm}\), \(R_{lstm}\), and \(b_{lstm}\) are the union of the input weights, the recurrent weights, and the bias of each part, respectively. These matrices are grouped as follows:  
\[\text{W}_{lstm}=\left[ \begin{array}{c}\text{W}_{\text{f}}\\ \text{W}_{\text{i}}\\ \text{W}_{\text{c}}\\ \text{W}_{\text{o}}\end{array} \right];\,\text{R}_{lstm}=\left[ \begin{array}[] {c}\text{R}_{\text{f}}\\ \text{R}_{\text{i}}\\ \text{R}_{\text{c}}\\ \text{R}_{\text{o}}\end{array} \right];\,\text{b}_{lstm}=\left[ \begin{array}[] {c}\text{b}_{\text{f}}\\ \text{b}_{\text{i}}\\ \text{b}_{\text{c}}\\ \text{b}_{\text{o}}\end{array} \right] \tag{76}\]  
where \(f\), \(i\), \(c\), and \(o\) denote the forget gate, input gate, cell state, and output gate, respectively. In each LSTM block, the objective is the calculation of the output (hidden state) and the cell state through three parts: forget gate, input gate and output gate. As shown in Figure 8, the \"forget gate\" is presented in the first part of the LSTM, which is used to decide what information will be kept in the cell state. The decision is made by a sigmoid layer called \"forget gate layer\". \(h_{k-1}\) and \(x_{k}\) are inputs to the sigmoid function, and the output is a value ranging from 0 to 1 for each number in the cell state \(C_{k-1}\). If the output is \"1\", the information is \"completely kept\" in the cell state. When the output is \"0\", the information is \"completely cleared\". The forget gate's activation vector \(f_{k}\) is written as:  
\[f_{k}=\sigma(W_{f}x_{k}+R_{f}h_{k-1}+b_{f}) \tag{77}\]  
where \(\sigma(. )\) is the sigmoid function, \(h_{k-1}\) is the hidden state at time \(t_{k-1}\), and \(x_{k}\) is the input vector at time \(t_{k}\). The second part is the \"input gate\", which is employed to decide which new information should enter the previous cell state. This gate is composed of two parts: (1) a \"sigmoid\" layer to decide what values will be updated. The output values \(i_{k}\) for this layer ranging from 0 to 1. \"0\" means \"not important\" and \"1\" represents \"important\"; (2) another part is \"tanh\" layer which creates a vector of new candidate values \(\widetilde{C}_{k}\) between \(-1\) and \(1\) to help regulate the network. The input gate's activation vector \(i_{k}\) and the new candidate vector \(\widetilde{C}_{k}\) are calculated as follows:  
\[i_{k}=\sigma(W_{i}x_{k}+R_{i}h_{k-1}+b_{i}) \tag{78}\]  
\[\widetilde{C}_{k}=\tanh\left(W_{c}x_{k}+R_{c}h_{k-1}+b_{c}\right) \tag{79}\]  
where \"\(tanh\)\" is the hyperbolic tangent function. The new cell state \(C_{k}\) is updated from the old cell state \(C_{k-1}\) as follows:  
\[C_{k}=f_{k}*C_{k-1}+i_{k}*\widetilde{C}_{k} \tag{80}\]  
In the last part, the \"output gate\" will decide the output.