# A Real-Time Semi-Supervised Log Anomaly Detection Framework for ALICE O\({}^{2}\) Facilities
## 3 Methodology
### Proposed Framework
#### 3.1.2 BERTopic  
BERTopic [10] is a topic modeling technique that utilizes transformers for embedding representations and clustering algorithms to discover topics in textual data. It stands as a significant advancement from traditional topic modeling approaches such as LDA and NMF, particularly in handling contextual and semantic information in large-scale text corpora. As illustrated in Figure 1, BERTopic can be divided into the following components:  
* **Text embedding:** BERTopic utilizes pre-trained transformer models, sentence transformer embeddings (SBERTs), to convert textual data into dense vector representations, capturing the contextual relationships between words and sentences. * **Dimensionality reduction:** BERTopic applies dimensionality reduction techniques (i.e., uniform manifold approximation and projection (UMAP) [41]). This reduces the complexity of the embedding space while preserving the essential structure of the data, facilitating efficient and meaningful clustering in subsequent steps. * **Clustering:** The embedded vectors are clustered using Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) [42] as a default algorithm in BERTopic. HDBSCAN excels at identifying clusters of varying densities, handling noise, and automatically determining the optimal number of clusters, making it ideal for complex data. Alternatively, other algorithms can be applied. DBSCAN [43], another density-based method, performs well for uniform-density clusters but requires manual parameter tuning and struggles with varying densities. K-means [44; 45], a centroid-based algorithm, is computationally efficient but assumes spherical clusters and requires the number of clusters to be predefined. * **Topic representation:** Class-based Term Frequency-Inverse Document Frequency (c-TF-IDF) has been introduced in BERTopic for topic representation. It is an adaptation of the traditional TF-IDF technique designed for topic modeling. Unlike TF-IDF, c-TF-IDF calculates term significance within specific clusters or classes (e.g., topics). It aggregates term frequencies within a class and adjusts for their rarity across other classes, highlighting terms that define each topic. This approach is particularly effective in methods such as BERTopic, as it emphasizes the key terms that characterize each topic. This approach provides a meaningful representation of the underlying clusters by highlighting the terms that are used to define the cluster. The c-TF-IDF score for a term \(x\) within topic \(c\) is computed as follows: \[W_{x,c}=tf_{x,c}\cdot log(1+\frac{A}{f_{x}})\] (1)where \(tf_{x,c}\) is the frequency of word \(x\) in class \(c\), \(f_{x}\) is the frequency of word \(x\) across all classes, and \(A\) is the average number of words per class. In anomaly detection within logging systems, BERTopic can be employed to uncover latent topics in log entries, aiding in the identification of anomalous patterns or events. By extracting semantic topics from logs, the model can help distinguish between normal system behavior and potential anomalies, where an anomalous event might correspond with an unusual topic. For instance, logs containing error messages, system failures, or security breaches can be categorized into distinct topics that represent unusual activity, which can then be flagged as anomalies. Figure 1 illustrates the proposed framework, which begins by ingesting raw log messages from the logging system. A preprocessing stage ensures consistency by standardizing log messages through data preprocessing tasks, as mentioned. After preprocessing, preprocessed log messages are transformed into high-dimensional embeddings using SBERT, which encodes their semantic content. These embeddings are then reduced in dimensionality using UMAP, preserving intrinsic structures while reducing computational complexity. Next, the reduced embeddings are clustered using HDBSCAN, which identifies topics and labels outliers as potential anomalies.