## 2 Materials and Methods
### Hybrid Model LSTM-MA-AE
Early stopping ensures that the model fits with small errors in normal cases. In contrast, in anomalous situations, the model cannot fit the input features, resulting in a significant discrepancy with the normal data. This improves the model's ability to detect anomalies effectively. The hyperparameters with which the model achieved the best performance are shown in Table 3.

### Model Evaluation Metrics

To evaluate the proposed model, we used different error metrics, common in prediction analysis, such as the root mean square error (RMSE), the mean absolute error (MAE), and the coefficient of determination (\(R^{2}\)). Equations (38)-(40) define these metrics. \[\text{RMSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}} \tag{38}\]

\[\text{MAE}=\frac{1}{n}\sum_{i=1}^{n}|y_{i}-\hat{y}_{i}| \tag{39}\]

\[R^{2}=1-\frac{\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}}{\sum_{i=1}^{n}(y_{i}-\bar {y})^{2}} \tag{40}\]

where \(y_{i}\) represents the actual values, \(\hat{y}_{i}\) the predicted values, \(n\) is the number of samples, and \(\bar{y}\) is the mean value of \(y_{i}\). In addition, we use the Precision (\(Pr\)), Recall (\(Re\)), and F1-Score (\(F1\)) metrics, which are commonly employed in anomaly detection [45]. These metrics are defined by Equations (41)-(43). \[Pr=\frac{TP}{TP+FP} \tag{41}\]

\[Re=\frac{TP}{TP+FN} \tag{42}\]

\[F1=\frac{2\cdot Pr\cdot Re}{Pr+Re} \tag{43}\] \begin{table}
\begin{tabular}{l c} \hline \hline
**Parameters** & **Value** \\ \hline Batch size & 256 \\ Sequence length & 20 \\ Dropout & 0.1 \\ Hidden size LSTM & 64 \\ Learning rate & \(1\times 10^{-6}\) \\ Epochs & 75 \\ Regularization factor & \(1\times 10^{-4}\) \\ LSTM layers encoder & 2 \\ LSTM layers decoder & 2 \\ Heads encoder & 8 \\ Heads decoder & 8 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hyperparameters used in LSTM-MA-AE. where True Positive (\(TP\)) represents alarms that detected a fault correctly within a specific time window, False Positive (\(FP\)) corresponds to alarms that do not result in an actual fault, and False Negative (\(FN\)) indicates faults that were not correctly detected. To conclude this section, the experiments were conducted using the Python programming language in the PyTorch library within a Google Colab environment. The environment had an Intel(R) Xeon(R) CPU @ 2.20 GHz, 51.00 GB of RAM, a Tesla K80 accelerator, and 12 GB of GDDR5 VRAM. ## 3 Results

This section presents the results in two subsections. Section 3.1 analyzes the model's ability to generalize the temporal data for each feature. Section 3.2 describes the performance of the model in predicting faults and the advance time until the occurrence of faults. ### Model Evaluation

The results shown in Table 4 were achieved using the hyperparameters outlined in Table 3. We evaluated three model configurations:

1. The LSTM-MA-AE approach. 2. The same approach without the AM (LSTM-AE). 3. A version without PE and residual connection. This enables us to confirm that the improve implemented effectively enhance the performance of the proposed model. The data presented in Table 4 indicates that incorporating MA into the LSTM-MA-AE approach is highly effective for capturing the complexities within temporal data. It shows that the AM enhances model performance. The MA assists the model in comprehend a wide range of sequences, thereby retaining crucial information necessary for effectively reconstructing the time series. In addition, the LSTM-MA-AE approach without positional coding shows inferior performance compared to the approach that includes PE. This is because the addition of PE makes it easier for the model to understand the positional characteristics of each data in the time series, improving its ability to interpret and predict accurately. The \(R^{2}\) coefficient shows the difference between the series; a value of \(R^{2}\) close to 1 reflects a high similarity between the data sets. The MAE measures the loss between two data points, while the RMSE penalizes outliers.