# LTmatch: A Method to Abstract Pattern from Unstructured Log
## 4 Experiments
### Robustness  
In this section, in order to verify the robustness of the method we proposed, we use the method of LTmatch to conduct accuracy experiments on log datasets of different volumes, so this experiment chooses the labeled logs in dataset 2 as the experiment object. In comparison, the same method as the previous section was selected for the experiment, and the final experimental result is shown in Figure 4.  
First, the comparison between the optimization method we proposed and the original method is analyzed. It can be seen from Figure 4 that the initial results of Lmatch are better on HDFS and GBL data, but after a certain amount of log \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Log Name** & **LenMa** & **Spell** & **AEL** & **IPLoM** & **Drain** & **LTmatch** & **Lmatch** & **Best** \\ \hline Android & 0.8795 & **0.9195 * & 0.6815 & 0.7120 & **0.9110** & **0.9080** & **0.9120** & 0.9195 \\ Apache & **1.0000 * & **1.0000 * & **1.0000 * & **1.0000 * & **1.0000 * & **1.0000 * & **1.0000 * & 1.0000 * & 1.0000 * \\ BGL & 0.8295 & 0.7545 & **0.9570** & **0.9390** & **0.9625** & **0.9325** & 0.8355 & 0.9625 \\ Hadoop & 0.8850 & 0.7840 & **0.9690** & **0.9540** & **0.9475** & **0.9870** & 0.8765 & 0.9870 \\ HDFS & **0.9975** & **1.0000** & **0.9975** & **1.0000** & **0.9975** & **1.0000** & 0.8405 & 1.0000 \\ HealthApp & 0.1740 & 0.6390 & 0.5675 & 0.8215 & 0.7800 & 0.8795 & 0.5565 & 0.8795 \\ HPC & 0.8295 & 0.6540 & **0.9030** & 0.8290 & 0.8870 & **0.9345** & **0.9040** & 0.9345 \\ Linux & 0.7010 * & 0.1505 & 0.6725 & 0.6715 & 0.6900 & 0.6905 & 0.5995 & 0.7010 \\ Mac & 0.6980 & 0.1505 & 0.6725 & 0.6715 & 0.6900 & 0.6905 & 0.5995 & 0.7010 \\ OpenSH & **0.9250 * & 0.5540 & 0.5380 & 0.5400 & 0.7875 & 0.7475 & 0.6780 & 0.9250 \\ OpenStack & 0.7425 * & 0.7535 & 0.7575 & 0.3305 & 0.7325 & 0.8959 * & 0.8065 & 0.8595 \\ Proxifier & 0.5080 & 0.5265 * & 0.4950 & 0.5165 & 0.5265 * & 0.5165 & 0.5265 * & 0.5165 & 0.5265 \\ Spark & 0.8835 & **0.9050** & **0.9050** & **0.9200** & **0.9200** & **0.9975 * & 0.7175 & 0.9975 \\ Thunderbird & **0.9430** & 0.8435 & **0.9410** & 0.6630 & **0.9550 * & **0.9470** & **0.9215** & 0.9550 \\ Windows & 0.5655 & 0.7055 & 0.6895 & 0.5670 & **0.9970 * & **0.9960** & **0.9900** & 0.9970 \\ Zookeeper & 0.8405 & 0.8180 & **0.9210** & **0.9615** & **0.9665** & **0.9885 * & **0.9640** & 0.9885 \\ Average & 0.7751 & 0.7336 & 0.7912 & 0.7562 & 0.8654 & 0.8885 * & 0.8103 & 0.8885 \\ \hline \hline \end{tabular}  
* The best accuracy is highlighted with a asterisk and shown in the column ”Best” for each dataset.  
\end{table} 
Table 2: The average accuracy of the results on different kinds of log. Figure 4: Accuracy of different log parsers over different sizes of BGL/HDFS log data sets. results begins to decrease significantly, which shows that directly using the word matching rate algorithm based on the longest common subsequence has no obvious advantage in parsing log patterns. The accuracy of LTmatch has remained stable when the log capacity changes, indicating that the improved log pattern extraction algorithm has more strong generalization ability than the algorithm before the improvement. When comparing with all other types of log classification algorithms, it can be seen from the Figure 4a on the GBL dataset, the initial accuracy of LenMa algorithm, IPLoM algorithm, and Spell algorithm is relatively high, but they have begun to decline when the number of logs reaches 0.45 million. The remaining Drain, AEL, and the LTmatch algorithm we proposed have maintained stable accuracy. On the HDFS dataset in Figure 4b, except for Spell and the LTmatch algorithm we proposed, the accuracy has been maintained at a high level, and other methods have begun to decline on the number of logs of 0.15 million. It can be seen from the experimental results of the two different log datasets HDFS and GBL on different data volumes that only the LTmatch algorithm can always remain stable when the log type changes.