# LogEDL: Log Anomaly Detection via Evidential Deep Learning
## 5 Experiment and Analysis  
### Experiment Setting  
#### 5.1.1 Dataset  
We benchmark our log anomaly detection method on three log datasets: HDFS [30], BGL [27], and Thunderbird [31]. These datasets are sourced from the large-scale log dataset LogHub, published by [PERSON] et al. [32]. The statistics of these datasets are shown in Table 1. These datasets have been preprocessed as described earlier, with log data grouped into log sequences. For the HDFS dataset, data are partitioned using Block ID, while the BGL and Thunderbird datasets are partitioned using a time window. The preprocessing results are shown in Table 2.  
The HDFS dataset was generated by running benchmark workloads in a private cloud environment and was manually labeled using hand-crafted rules to identify anomalies. Log data were divided into log sequences based on block IDs. Each log sequence associated  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Log Messages** & **Normal** & **Anomaly** \\ \hline HDFS & 11,175,629 & 10,887,339 & 288,290 \\ BGL & 4,747,963 & 4,399,503 & 348,460 \\ Thunderbird & 20,000,000 & 1,241,438 & 758,562 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Statistics of evaluation datasets.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset** & **Log Sequences** & **Log Keys** & \begin{tabular}{c} **Train Dataset** \\ **Normal** \\ \end{tabular} & \begin{tabular}{c} **Test Dataset** \\ **Normal** \\ \end{tabular} & \
\begin{tabular}{c} **Test Dataset** \\ **Anomaly** \\ \end{tabular} &
\begin{tabular}{c} **Average** \\ **Length** \\ \end{tabular} \\ \hline HDFS & 742,527 & 47 (18) & 167,466 & 558,223 & 16,838 & 19 \\ BGL & 37,315 & 334 (175) & 13,718 & 20,579 & 3018 & 562 \\ Thunderbird & 122,540 & 1165 (866) & 46,293 & 30,862 & 45,385 & 326 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Preprocessing on HDFS, BGL, and Thunderbird datasets.  
with a block ID was assigned a ground truth label of either normal or anomalous. The HDFS dataset contains 11,175,629 log messages, grouped into 742,527 log sequences, with 725,689 normal sequences and 16,838 anomalous sequences. The average sequence length is 19. A total of 47 log keys were extracted from the dataset, with 18 log keys appearing in normal sequences. The model was trained on this dataset for 200 epochs with a masking ratio of 0.65.  
The BGL dataset comes from the BlueGene/L supercomputer system at the Lawrence Livermore National Laboratory (LLNL) in the United States. The logs include both alert and non-alert messages, with alert messages considered anomalies. The BGL dataset contains 4,747,963 log messages, grouped into 37,315 log sequences, with 10,131 normal sequences and 2630 anomalous sequences. For BGL, the data were partitioned using a sliding time window of 5 min to generate log sequences, with an average sequence length of 562. A total of 334 log keys were extracted from the dataset, with 175 log keys appearing in normal sequences. The model was trained on this dataset for 200 epochs with a masking ratio of 0.50.  
The Thunderbird dataset is another large log dataset collected from a supercomputer system, specifically the Thunderbird microprocessor. The logs contain both alert and non-alert messages, identified by alert category labels. In the first column of the logs, \"-\" indicates a non-alert message, while other entries indicate alert messages. This labeling information can be used for alert detection and prediction research. The Thunderbird dataset contains 20,000,000 log messages, grouped into 122,540 log sequences, with 77,155 normal sequences and 45,385 anomalous sequences. For the Thunderbird microprocessor dataset, we used a sliding time window of 1 min to generate log sequences, with an average sequence length of 326. A total of 1165 log keys were extracted from the dataset, with 866 log keys appearing in normal sequences. The model was trained on this dataset for 200 epochs with a masking ratio of 0.50.