## 5 Results and Discussion
### Anomaly Detection Performance  
In the previous section, the log parsers were evaluated in terms of the number of generated templates, accuracy, and F1-measure. These aspects are critical for log parsers. If it is only to obtain the formatted results of log files, judging the advantages and disadvantages of log parsers from these aspects is sufficient. However, structured log messages parsed by log parsers are typically used for subsequent log mining models and are then used to analyze the system for anomaly detection, performance detection, etc. Therefore, the performance of log parsers should also be evaluated in subsequent anomaly detection tasks.  
In this section, parsing performance is evaluated using a published HDFS dataset containing approximately 11 million log messages generated by approximately 570,000 HDFS blocks and 29 log event types. The HDFS blocks include normal and abnormal blocks, as labeled by the original author. We first used different log parsers to parse the HDFS dataset into structured log messages, and we then applied an open-source toolkit [33], taking structured log messages and ground truth as inputs, to obtain the required results. First, an event count matrix is generated from the structured log messages, where each row represents an HDFS block. Each column represents a log event type, and each cell counts the events on an HDFS block. Second, term frequency-inverse document frequency was used to preprocess the event count matrix. Finally, the event count matrix was input to the corresponding exception detection module for exception detection. The open-source toolkit provides six typical anomaly detection methods, including supervised and unsupervised methods. We primarily used the DT and SVM algorithms for evaluation, and the log parsing step used multiple parsers for comparison.  
The HDFS dataset generates different numbers of log events after parsing by different log parsers. In addition, an HDFS block exhibits different log event sequences when different log parsers are applied, which results in different log event count matrices and anomaly detection results. We used five log parsers to perform the parsing task, and then, we input their parsing results into the exception detection module to obtain the results (Table 3).  
\begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**MoLFI**} & \multicolumn{2}{c}{**Drain**} & \multicolumn{2}{c}{**IPLoM**} & \multicolumn{2}{c}{**Spell**} & \multicolumn{2}{c}{**CFTL**} \\ \cline{2-11}  & **Acc** & **F1** & **Acc** & **F1** & **Acc** & **F1** & **Acc** & **F1** & **Acc** & **F1** \\ \hline HDFS & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 & 0.99 \\ Windows & 0.40 & 0.91 & 0.99 & 0.99 & 0.56 & 0.99 & 0.98 & 0.99 & 0.99 & 0.99 \\ Thunderbird & 0.65 & 0.99 & 0.95 & 0.99 & 0.66 & 0.99 & 0.84 & 0.99 & 0.93 & 0.99 \\ Hadoop & 0.94 & 0.99 & 0.94 & 0.99 & 0.95 & 0.99 & 0.77 & 0.92 & 0.96 & 0.99 \\ OpenSSH & 0.48 & 0.99 & 0.78 & 0.99 & 0.80 & 0.99 & 0.55 & 0.91 & 0.80 & 0.99 \\ Apache & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ Android & 0.78 & 0.94 & 0.91 & 0.99 & 0.71 & 0.94 & 0.91 & 0.99 & 0.91 & 0.99 \\ HPC & 0.67 & 0.97 & 0.88 & 0.99 & 0.82 & 0.97 & 0.65 & 0.98 & 0.91 & 0.99 \\ Spark & 0.41 & 0.51 & 0.92 & 0.99 & 0.92 & 0.99 & 0.90 & 0.99 & 0.98 & 0.99 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Accuracy and F1-measure of different approaches.  
The experimental results demonstrate that all log parsers perform well, demonstrating that after a log message passes the parsing process of a log parser, anomaly detection results with high accuracy can be obtained. For Spell and IPLoM, even though their log message templates were overfitted, better test results were obtained. Spell generated too many log templates, which performed particularly complex calculations during anomaly detection, resulting in excessive processing time. In general, generating an appropriate number of log message templates not only improves anomaly detection efficiency but also accuracy.