## 2 Materials and Methods
### Hybrid Model LSTM-MA-AE  
This model efficiently captures the spatial and temporal characteristics of SCADA data, improving fault prediction and diagnosis capabilities. The LSTM-MA-AE model developed in this study processes the input signals to reconstruct the time series and predict faults before they occur. The LSTM-MA-AE model has two main parts: the encoder and the decoder. The structure of the model is presented in Figure 7. In the encoder, a combination of LSTM layers, PE, and MA processes the input sequences. The decoder uses a similar structure to the encoder to reconstruct the time series from the encoded representations. This means that two LSTM layers with 64 hidden units each are employed in the encoder, which are responsible for capturing complex short-term temporal dependencies in the data sequence, embedding patterns relevant to the reconstruction task. These layers are followed by normalization and a dropout of 0.1 to prevent overfitting. At the output of the LSTMs, a PE is integrated for the 8-head MA mechanism to understand the positions of the time series data. These attention features are added to a residual connection from the previous output so that no data information is lost. Each head of the MA allows the model to focus on different aspects of the input sequence simultaneously, enhancing the capability of long-term feature embedding and temporality. Furthermore, in the decoder, we initiate the extraction of the encoder embedded features using an 8-head MA mechanism supplemented with two additional 64-layer hidden LSTM layers. Initially, zero tensors are used as the query in the MA mechanism, whereas the encoder embeddings are employed as a key and value. These are then passed to the LSTM layers, which process the historical values of the dataset along with the attention output, allowing the relevant sequential structure to be extracted and the first value to be predicted through a fully connected layer. This output is subsequently used as feedback in the query the MA mechanism for subsequent predictions. In this way,  
Figure 7: Architecture of the LSTM-MA-AE model. the predictions generated by the decoder are iteratively employed as a query, achieving accurate reconstruction throughout the entire sequence. Finally, we apply normalization and dropout to optimize training and prevent model overfitting. The input to the model is a time series of length \(T\) data with \(d\) characteristics:  
\[\mathbf{X}=[\mathbf{x}_{1},\mathbf{x}_{2},\dots,\mathbf{x}_{T}]\in\mathbb{R}^{T \times d} \tag{15}\]  
The temporal sequence is then encoded through a 2-layer LSTM with _dropout_ in between and a normalization layer _LayerNorm_ at the end of the second layer, to obtain the encoded representation, as represented in the following Equations (16) and (17):  
\[\mathbf{H}_{\text{out}}=\text{LSTM}_{\text{enc}}(\mathbf{X}) \tag{16}\]  
\[\mathbf{H}_{\text{enc}}=\textit{LayerNorm}(\text{LSTM}_{\text{enc}}(\mathbf{ H}_{\text{out}})) \tag{17}\]  
Output series \(\mathbf{H}_{\text{enc}}\) has \(PE\) added to it so that MA can understand the position of the time series, as shown in Equation (18). \[\mathbf{H}_{\text{PE}}=\mathbf{H}_{\text{enc}}+PE \tag{18}\]  
Once PE is added, the MA is calculated to capture complex relationships in the time sequence, as shown in Equations (19)-(22). \[\mathbf{Q}=\mathbf{H}_{\text{PE}}W^{Q} \tag{19}\]  
\[\mathbf{K}=\mathbf{H}_{\text{PE}}W^{K} \tag{20}\]  
\[\mathbf{V}=\mathbf{H}_{\text{PE}}W^{V} \tag{21}\]  
\[\mathbf{att}_{\text{enc}}=\text{Multi-HeadAttention}(\mathbf{Q},\mathbf{K}, \mathbf{V}) \tag{22}\]  
where \(W^{Q}\), \(W^{K}\), \(W^{V}\) are matrices of learnable weights of query \(Q\), key \(K\), and value \(V\). To avoid losing data information, a residual connection is made from the output of \(\mathbf{H}_{\text{PE}}\) to the output of the MA, using Equation (23).