# Construction of Power Fault Knowledge Graph Based on Deep Learning
## 3 Construction Model of Power Fault Knowledge Graph Based on Deep Learning  
Entity extraction is an important part of fault recovery knowledge graph construction. Existing entity extraction methods require a large amount of labeled corpus. If part of the labeled corpus is reduced, the extraction accuracy of the method is severely reduced. This section introduces the power system log anomaly detection model based on the bidirectional slice gated cyclic unit and Gated Attention Mechanism (BiSGRU-GA), power fault entity extraction model based on BiGRU-GA-CRF, and power fault relation extraction model based on BiGRU-GA. Based on the above three models, a power fault knowledge graph system is constructed. The method presented in this paper reduces the model's dependence on labeled corpus, which is of significance for improving the power grid fault processing capability. Bidirectional Sliced Gated Recurrent Units with a Gated Attention Mechanism (BiSGRU-GA) Log Anomaly Detection Model  
Log anomaly detection can be attributed to sequence classification or prediction. Because it is very difficult to directly process unstructured log data, the usual method of log anomaly detection is to obtain log templates through log parsing, and then perform anomaly detection on the templates. The template is equivalent to the \"summary\" of the log, and the log can be obtained as a template plus parameters [21]. A log key sequence can be regarded as a template and based on the ideas introduced in DeepLog [22], a new method was designed and implemented; it is a neural network model named Bidirectional Sliced GRU with a Gated Attention Mechanism [23] (BiSGRU-GA). This model has fewer parameters and a faster convergence speed than LSTM models. Whilst the purpose was to reduce the training costs, an additional benefit was that the accuracy was also greatly improved. The model structure is shown in Figure 4. We take the input length of 8 and the minimum subsequence of 2 as an example:  
Figure 3: Backbone network. First, the word2 vec method is used to generate the log key word vector \(x_{m}\) from the system log. Thus, the length of each log key is composed of max key word vectors and \(x_{ij}\) is used to represent \(X_{i}\). The expression of \(X_{i}\) is as shown in Equation (6):  
\[X_{i}=x_{i1}\oplus x_{i2}\oplus\ldots x_{ij} \tag{6}\]  
Then the sequence generated by the word2 vec is input into the GRU layer in both forwards and backwards directions. The specific calculation formula is as shown in the Equations (7) and (8):  
\[\overrightarrow{h_{ij}^{o}}=\overrightarrow{GRU^{0}}(x_{ij},\overrightarrow {h_{i(j-1)}^{0}}) \tag{7}\]  
\[\overleftarrow{h_{IJ}^{o}}=\overleftarrow{GRU^{0}}(x_{ij},\overleftarrow{h _{(I-1)}^{0}}) \tag{8}\]  
where \(x_{ij}\) represents the \(j\)-th input of the \(i\)-th smallest subsequence in the processing direction, \(\overrightarrow{h_{ij}^{o}}\) is a vectorization status after GRU processing in the forwards direction and \(\overleftarrow{h_{IJ}^{o}}\) is a vectorization status after GRU processing in the backwards direction  
The Attention layer is the core part of the network model BiSGRU-GA. This layer assigns weight to the filtered words in each log key. The representation of each minimal subsequence \(s_{i}\) is represented by the weighted sum of vectors so as to realize the focus of some location key features at the log key level. Figure 4: Log anomaly detection BiSGRU-GA.  
Then the output sequence of the attention layer in the second GRU layer is also processed in two directions. The specific calculation formula is as shown in the Equations (9) and (10):  
\[\overrightarrow{h_{t}^{1}}=\overrightarrow{GRU^{1}}(S_{t_{\pi}\frac{p_{0}}{p_{1} }-l_{0}+1}\sim S_{t_{\pi}\frac{p_{0}}{p_{1}}}) \tag{9}\]  
\[\overleftarrow{h_{t}^{1}}=\overleftarrow{GRU^{1}}(S_{t_{\pi}}\frac{p_{0}}{p_ {1}}-l_{0}+1\sim S_{t_{\pi}\frac{p_{0}}{p_{1}}}) \tag{10}\]  
Then in the same way as the second GRU layer, the representation of the nth layer is output.