## 3 Methodology
### Performance Metric  
Performance metrics are fundamental for evaluating algorithms' performance. In the case of supervised learning, common metrics for classification problems, such as confusion matrices, accuracy, F1-score, precision, or recall, were used in some works in the field (e.g., in [31; 38]). However, while treating unsupervised time series or measuring the promptness of detection on streaming data, those metrics do not apply. The Numenta Anomaly Benchmark (NAB) [55] shows a way to evaluate the performance of algorithms with streaming data. To do that, the algorithms are tested in the labeled datasets available to the effect and a group of metrics evaluates the performance of the algorithms in many topics, like if all anomalies were detected or the promptness of detection. Here, it was decided to propose a metric, founded in the NAB principles but more adapted to our unsupervised problem concerns, as we will see below. The proposed performance metric is formulated by combining 2 scores: \(S_{1}\), which is the score that indirectly evaluates the incidence of supposed false positives, and \(S_{2}\), which is the score referring to the promptness classification of anomalies. The final score, \(S_{final}\), combines the previous two and is responsible for assigning the algorithm's performance score for a given test scenario. The score values are in the interval \([0,1]\), with 1 as the best performance. In more detail, \(S_{1}\) is obtained using the equation  
\[S_{1}=\frac{1}{1+\exp\Bigl{(}\frac{n_{\text{var}}-n_{\text{var}}\times k_{1} }{n_{\text{var}}\times k_{2}}\Bigr{)}}, \tag{1}\]  
where \(n_{\text{var}}\) translates a number of unknown points detected as an anomaly by the algorithm (i.e., that do not belong to the anomalies that were introduced), \(n_{\text{up}}\) is the total number of unclassified points in the test sample (number of points in test sample minus  
Figure 5: Architecture of the used VAE. The encoder is represented in orange and the decoder in green, with bi-directional layers of Long Short-Term Memory cells in each direction and the latent space Z modeled according to an isotropic normal distribution (with parameters \(\mu_{z}\) and \(\sigma_{z}\)). the number of anomalies introduced), and \(k_{1}\) and \(k_{2}\) are the values that allow adjusting the associated penalty level. With the score \(S_{1}\), it is possible to evaluate the algorithm in terms of false positives giving tolerance to the classification of uncertain points as anomalies. In our case, \(k_{1}=0.1\) and \(k_{2}=0.01\) were considered. In this case, when the algorithm detects up to 5.0% of false positive anomalies, a small decrease is introduced on the value of \(S_{1}\). When 10% of the readings are marked as false positives, then \(S_{1}\) would be equal to 0.5. As an example, Figure 6 graphically represents the evolution of the \(S_{1}\) score as a function of the number of unknown points detected as an anomaly (\(n_{va}\)) for a total number of unclassified points (\(n_{up}\)) of 8112. This was the scenario studied during the performed tests (Section 6). For example, if the algorithm classifies 300 readings as anomalies, beyond the anomalies introduced (i.e., \(n_{va}=300\)), in a universe of \(n_{up}=8112\) samples, the performance of the algorithm will not be badly evaluated, since 5% of the 8112 is 405.6. However, if the value of \(n_{va}\) increases, a penalty will begin to exist which is reflected in the \(S_{1}\) score. For instance, with \(n_{va}=750\), the value of \(S_{1}\) would decrease to 0.68. Thus, this sigmoidal shape, adjustable by the formula's parameters (\(n_{u}p\), \(k_{1}\), and \(k_{2}\)), allows for being more or less flexible in the identification of anomalies that were not identified as such in the dataset.