## 2 Conceptual Model and Problem Procedure
### Enhanced LSTM Neural Network
Generally speaking, this situation is related to the vanishing gradient and exploding gradient issues mentioned in [37; 43]. More advanced recurrent units, such as enhanced long short-term memory (ELSTM) units and, more recently, gated recurrent units (GRUs), have been proposed to address issues with basic RNNs. In addition to being easier and more effective to train than LSTMs, GRUs have demonstrated performance levels that are comparable to those of LSTM architectures [44]. Formally, a GRU [44] uses the following approximate function to try and implement the hidden layer function \(f_{h}\) in Equation (13):  
\[z_{t}=\sigma(W_{xz}x_{t}+W_{hz}h_{t-1}+b_{z}) \tag{17}\]  
\[r_{t}=\sigma(W_{xx}x_{t}+W_{hr}h_{t-1}+b_{r}) \tag{18}\]  
\[\widetilde{h}_{t}=\tanh(W_{xh}x_{t}+W_{h}(h_{t-1}\odot r_{t})+b_{h}) \tag{19}\]  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]0  
where \(\odot\) stands for multiplication of elements, The weight matrices \(M\times K\), \(M\times K\), \(M\times K\), and \(M\times M\) are represented by \(W_{xz}\), \(W_{xr}\), \(W_{xh}\), and \(W_{h}\); the \(m\times 1\) bias terms are represented by \(b_{z}\), \(b_{r}\), and \(b_{h}\); the \(M\times 1\) (update gate) vector and \(M\times 1\) (rest gate) vector are represented by \(Z_{t}\) and \(r_{t}\), respectively. GRUs can be stacked on top of one another to create a deeper network, much like simple RNN units. Under such circumstances, Equations (17)-(20) are  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]1 An illustration of a basic RNN unit schematic. extended to all layers, much like Equation (15). However, the hidden layer function \(f_{h}\) in Equation (13) is implemented by the following composite function in the proposed LSTM:  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]2  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]3  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]4  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]5  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]6  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]7  
where \(b_{i}\), \(b_{f}\), \(b_{c}\), and \(b_{0}\) denote \(M\times 1\) bias terms; \(i_{t}\), \(f_{t}\), \(c_{t}\), and \(o_{t}\) denote \(M\times 1\) (input gate) vector, \(M\times 1\) (forget gate) vector, \(M\times 1\) (cell state) vector, and \(M\times 1\) (output gate) vector, respectively; \(W_{xi}\), \(W_{xf}\), \(W_{xc}\), and \(W_{xo}\) denote \(M\times K\); \(W_{hi}\), \(W_{hf}\), \(W_{hc}\), and \(W_{ho}\) denote \(M\times M\) weight matrices. The reason the models up to this point have been referred to as unidirectional RNNs is that only observations \(x_{1},x_{2},\ldots,x_{t}\) are utilized in order to predict \(y_{t}\); in other words, information flows from the past to the present and future observations are not utilized in order to predict \(y_{t}\). The bidirectional LSTM neural network that can use data from the \"future\" to predict \(y_{t}\) is proposed in this paper. In this sense, a different set of hidden layers is identified and linked in a different temporal sequence. Formally, the following modification to Equation (13) yields this:  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]8  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]9  
\[y_{t}=W_{hy}h_{t}+b_{y} \tag{14}\]0  
where \(W_{xh}^{f}\), \(W_{hh}^{f}\), \(W_{xh}^{b}\), \(W_{hy}^{b}\), \(b_{h}^{f}\), \(b_{h}^{b}\), and \(b_{y}\) are weight matrices and bias terms that need to be calculated in the right size. In the current study, the model is trained to minimize the error between the predicted output and the actual labels using a suitable loss function, such as binary cross-entropy loss. Backpropagation through time (BPTT) or alternative optimization algorithms like Adam or RMSprop are used to update the network parameters. Where:  
\(y_{t}\): Binary coding vector using (0,1) for various types of faults in power transformers.