## 6 Materials and Methods
### Training Phase  
In the initial training phase, several hyperparameters are explored for experimentation with the LSTM and DeepESN models. This in-depth analysis allows us to identify the optimal combination that leads to superior levels of accuracy. The optimal hyperparameters are identified through a targeted experimentation phase in order to maximize the performance of the LSTM and DeepESN models on the specific tasks to which they were applied. In the training phase, the model is adapted to predict the next step (next-step prediction). Training occurs using the training set, allowing the model to learn temporal relationships in the input data. In particular, for LSTM, the mean squared error (MSE) is used as the loss function, and Adam is used as the optimizer, while for DeepESN, the learning function is the one-shot ridge regression learning rule. After training, the next step is the calculation of errors in the training set, measuring the discrepancy between the model predictions and the respective time steps of the training set. These errors are recorded in a buffer, and the standard deviation is generated. This step is crucial for the dynamic calculation of the sigma value, which will subsequently be used to verify anomalies. The standard deviation of the errors of the time steps of the training set (\(\boldsymbol{\sigma}\)) and of the queue (queue, \(\mathbf{q}\)) is not a single value but a vector of standard deviations, each associated with a specific feature (in our case, a composite vector from the 88 features of the time series). This approach allows for more accurate anomaly detection, since the detection process is performed separately for each feature in the time series. A visual representation of vectors is shown in Figure 11.  
Inference Phase  
During this phase, the trained model is combined with the anomaly detector to work in a real case. This phase is described by Algorithm 1. In this scenario, each time a new time step arrives in the stream, it is processed by the deep learning model, which provides the prediction of the next step. The result is passed to the anomaly detector with the real value of the next step, which arrives in the following moment. The anomaly detector generates \(\mathbf{a}(t)\), the vector that contains the predictions of anomalies for each feature at time step \(t\). Fixing a certain threshold \(fd\), a general anomaly can be identified when more features than the threshold \(fd\) are detected as failures. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Model** & **Hyperparameters** & **Value** & **Description** \\ \hline \multirow{8}{*}{DeepESN} & reservoir\_size & 200 & Reservoir and hidden state size \\ \cline{2-3}  & alpha & 0.8 & Leaky rate hyperparameter \\ \cline{2-3}  & input\_ratio & 0.9 & Hyperparam to scale input weights Wx \\ \cline{2-3}  & model\_spectral\_radius & 0.9 & Spectral radius hyperparam \\ \cline{2-3}  & model\_input\_sparsity & 0.5 & Input weights sparsity \\ \cline{2-3}  & reservoir\_sparsity & 0.9 & Reservoir weights sparsity \\ \cline{2-3}  & model\_regularization & 0.01 & Thikonov regularization hyperparam \\ \cline{2-3}  & n\_layers & 2 & Number of reservoir layers \\ \cline{2-3}  & washout & 100 & To cut first hidden states \\ \cline{2-3}  & \(\epsilon\) & 0.8 & \(\epsilon\) constant of anomaly detector (\(ad\)) \\ \cline{2-3}  & \(|Q|\) & 20 & Fixed length of the queue of \(ad\) \\ \hline \multirow{8}{*}{LSTM} & hidden\_state & 200 & Hidden state size \\ \cline{2-3}  & n\_layers & 2 & Number of recurrent layers \\ \cline{2-3}  & dropout & 0 & Dropout regularization hyperparam \\ \cline{2-3}  & lr & 0.01 & Learning rate \\ \cline{2-3}  & b1 & 0.9 & B1 Adam optimizer hyperparam \\ \cline{2-3}  & b2 & 0.99 & B2 Adam optimizer hyperparam \\ \cline{2-3}  & weight\_decay & 0 & L2 regularization hyperparam \\ \cline{2-3}  & \(\epsilon\) & 1.2 & \(\epsilon\) constant of anomaly detector (\(ad\)) \\ \cline{2-3}  & \(|Q|\) & 20 & Fixed length of the queue of \(ad\) \\ \hline \hline \end{tabular}
\end{table}