# A Real-Time Semi-Supervised Log Anomaly Detection Framework for ALICE O\({}^{2}\) Facilities
## 3 Methodology
### Evaluation  
To evaluate our experiment, five evaluation metrics were used to measure the performance of the proposed framework in detecting anomalous log messages. These metrics  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**No.** & **Parameter** & **Value** \\ \hline
1. & _min\_cluster\_size_ & 5 \\
2. & _min\_samples_ & 5 \\
3. & _cluster\_selection\_epsilon_epsilon_ & 0.0 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The default hyperparameters setting.  
were precision, recall, F1-score, the false positive rate (FPR), and the false negative rate (FNR), all of which can be computed based on values provided in the confusion matrix (i.e., true positives (TPs), false positives (FPs), false negatives (FNs), and true negatives (TN)).  
In particular, for this study, the main focus was on recall, which measures the ability of the framework to detect all anomalous log messages in the dataset. Maximizing recall is critical in high-stakes environments like the ALICE experiment, where detecting every anomaly is essential to minimize the risk of undetected system failures. Recall can be computed using the following formula:  
\[Recall=\frac{True\ Positive}{True\ Positive+False\ Negative}. \tag{2}\]  
Precision is also important, as it evaluates the proportion of correctly identified anomalies among all messages flagged as anomalous. High precision ensures fewer false positives, reducing unnecessary alerts and making the system more practical for real-world monitoring. Precision can be computed as follows:  
\[Precision=\frac{True\ Positive}{True\ Positive+False\ Positive}. \tag{3}\]  
While recall is prioritized, the F1-score is used to balance recall and precision by providing a single measure of overall performance. As the harmonic mean of precision and recall, the F1-score ensures that the model not only captures anomalies effectively but also maintains accuracy in its predictions. This combination of metrics enables a robust evaluation of the framework, ensuring its effectiveness in detecting anomalies and its reliability in operational contexts. The F1-score can be computed as follows:  
\[F1-score=\frac{2\times Precision\times Recall}{Precision+Recall}. \tag{4}\]  
In addition to the above metrics, the anomalies mostly occur rarely and lead to an imbalanced class problem. Precision and recall can be biased towards the majority class as a simple strategy of always predicting the majority class can yield high precision and recall scores, but this is not a desirable outcome, as it fails to capture the model's ability to identify the minority class instances correctly. Therefore, the other two metrics included in this work were the false positive rate (FPR) and the false negative rate (FNR).  
FPR is commonly termed the false alarm rate. It quantifies the proportion of negative instances (e.g., normal cases) that are incorrectly classified as positive (e.g., anomalous cases) through the classification model. A high FPR can be problematic because it leads to an increased number of false alerts, which can pose an inconvenience to system administrators, who receive erroneous alerts generated via the model. FPR is defined as follows:  
\[FPR=\frac{False\ Positive}{True\ Negative+False\ Positive}. \tag{5}\]  
FNR, commonly termed the misdetection rate, quantifies the likelihood that the model will fail to detect or identify a positive instance, resulting in a missed detection or a false negative prediction. A high false negative rate can be problematic because it carries severe implications for administrators, leading to unexpected system failures. FNR is defined as follows:  
\[FNR=\frac{False\ Negative}{True\ Positive+False\ Negative}. \tag{6}\]Incorporating FPR and FNR alongside precision, recall, and F1-score provides valuable insights into the model's ability to identify the minority class instances correctly while minimizing false alarms.