## 2 Materials and Methods
### Hybrid Model LSTM-MA-AE
As can be seen, the best performance was obtained by the LSTM-MA-AE model. During training, we set the maximum number of epochs to 75. However, using early stopping, the model reached its best performance at epoch 25, when the validation loss stopped improving significantly. As shown in Figure 9a, the validation loss decreases rapidly during the first 25 epochs and then stabilizes. Early stopping acted from this point \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Feature**} & \multicolumn{3}{c}{**LSTM-MA-AE**} & \multicolumn{3}{c}{**LSTM-AE**} & \multicolumn{3}{c}{**LSTM-MA-AE without PE**} \\ \cline{2-10}  & **RMSE** & **MAE** & \(\mathbf{R^{2}}\) & **RMSE** & **MAE** & \(\mathbf{R^{2}}\) & **RMSE** & **MAE** & \(\mathbf{R^{2}}\) \\ \hline
3 & 0.44 & 0.33 & **0.91** & 1.50 & 1.22 & \(-\)0.09 & 13.12 & 13.09 & \(-\)80.65 \\
2 & 0.52 & 0.37 & **0.88** & 1.71 & 1.35 & \(-\)0.23 & 20.35 & 20.32 & \(-\)143.71 \\
6 & 0.80 & 0.67 & **0.84** & 2.64 & 2.24 & \(-\)0.69 & 36.92 & 36.18 & \(-\)19.17 \\
8 & 1.55 & 1.17 & **0.82** & 3.34 & 2.68 & 0.18 & 58.62 & 56.84 & \(-\)13.99 \\
9 & 1.75 & 1.38 & **0.94** & 6.99 & 5.86 & 0.11 & 70.15 & 68.15 & \(-\)15.24 \\
7 & 3.63 & 2.60 & **0.93** & 16.28 & 13.15 & \(-\)0.35 & 358.37 & 358.36 & \(-\)9026.75 \\
4 & 4.21 & 3.17 & **0.93** & 22.23 & 16.44 & \(-\)0.88 & 1169.89 & 1052.11 & \(-\)4.21 \\
5 & 125.20 & 96.16 & **0.92** & 326.12 & 260.14 & 0.51 & 1094.50 & 986.58 & \(-\)4.32 \\
1 & 137.11 & 109.85 & **0.91** & 453.77 & 361.64 & \(-\)0.12 & 1169.89 & 1052.11 & \(-\)4.21 \\ \hline \hline \end{tabular} Note: In bold are the best results.

\end{table} 
Table 4: Comparison of variable reconstruction results. to avoid overfitting, which could have occurred if training had continued up to the initially configured epochs. This analysis is crucial to determine the optimal point where the model generalizes best without overfitting, thus ensuring robust performance on unseen data. Figure 9a depicts the behavior of losses during the training epochs, highlighting the importance of early stopping in model optimization. Furthermore, as seen in Figure 9a, a batch size of 256 was selected due to its comparable performance in terms of loss with smaller batch sizes, such as 128, but with a lower computational time, as shown in Figure 10. In Figure 9b, the sequence length is analyzed, and although sequence length 10 presents slightly better performance than sequence length 20, the latter was chosen, as it provides more history to the model, allowing more general learning of the time series. In Figure 9c, it is shown that regularization factors of \(1\times 10^{-4}\) and \(1\times 10^{-6}\) provide similar results, so a value of \(1\times 10^{-4}\) was chosen. Figure 9d shows that increasing the number of heads improves the capture of long-term patterns in the time series. Figure 9e shows that a single layer presents poor performance, while a larger number of layers, such as two or three, reduces the loss but increases the computational cost. Figure 9f shows that a higher dropout affects the model's generalization capability, with a value of 0.1 being the most appropriate in this case. Figure 9g reveals that a hidden layer size of 128 improves the fit to the real values, although it increases the computational cost and time, as shown in Figure 10. Finally, Figure 9h shows that the initial value of the learning rate, together with the scheduler, significantly influences the model fit. We aim to generalize the time series with a good fit, without overfitting, to identify anomalous patterns over time. Although a higher learning rate would be adequate for forecasting tasks, in this case, we need one that allows us to generalize the time series and obtain adequate results, as presented in Table 4.

Table 5 indicates that adding an attention mechanism increases the time required for model training. This is due to the increased computations and the need to learn more complex features.