# LogBD: A Log Anomaly Detection Method Based on Pretrained Models and Domain Adaptation
## 4 Experiment and Analysis
### Results and Analysis
For the BGL-TB scenario, the performance was comparable on the source system and lower than LogBD on the target system. Therefore, LogBD can provide good performance using only normal data when it is difficult to obtain labeled abnormal samples. Unlike previous methods, LogBD uses BERT to extract the semantic information of log messages, and uses semantic vectors to represent log templates, rather than the log templates Word2 Vec and Glove used in previous methods. This paper compared the model performance using four log template representation methods. The detection results generated by different log template representation methods in two scenarios are shown in Tables 7 and 8. We found that the performance was greatly improved when BERT was used for log template representation. This may be because the method based on the log template ID is to number the log template and represent it by number. This method regards the log template as a number; it does not consider the semantic information contained in the log template. Word2 Vec and Glove word vectors are fixed by looking up the dictionary and taking the corresponding word vectors. They cannot dynamically adjust the word vectors according to different context contexts, losing the integrity of the log template semantics. According to the context of the input sentence, BERT returns sentence-level word vectors through model calculation in the model network. Due to the different context of the input context, the returned word vectors are different in order to distinguish polysemous words. Compared with the first three, BERT can learn the deep semantics of sentences and capture the similarity between different log statements. The cross-system log anomaly detection model LogBD proposed in this paper uses the domain adaptation method in transfer learning and achieves excellent performance. In order to prove the effectiveness of the domain adaptation method, a set of comparative experiments was carried out to compare the performance of the model without using the domain adaptation method and using the domain adaptation method. The impact of domain adaptation methods on model performance was evaluated. The detection results using or not using the domain adaptation method in the two scenarios are shown in Tables 9 and 10. \"Without\" indicates that the domain adaptation method was not used, while \"with\" indicates that the domain adaptation method was used. Through observation, it can be seen that the domain adaptive method could greatly \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Source**} & \multicolumn{2}{c}{**Target**} \\ \cline{2-5}  & \(\mathbf{F_{1}}\) & **AUC** & \(\mathbf{F_{1}}\) & **AUC** \\ \hline Log Template ID & 0.725 & 0.733 & 0.784 & 0.776 \\ Word2 Vec & 0.788 & 0.797 & 0.845 & 0.909 \\ Glove & 0.832 & 0.857 & 0.897 & 0.918 \\ BERT (ours) & **0.880** & **0.880** & **0.938** & **0.973** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Detection results generated by different log template representation methods in the case of TB-BGL. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Source**} & \multicolumn{2}{c}{**Target**} \\ \cline{2-5}  & \(\mathbf{F_{1}}\) & **AUC** & \(\mathbf{F_{1}}\) & **AUC** \\ \hline Log Template ID & 0.781 & 0.799 & 0.679 & 0.691 \\ Word2 Vec & 0.813 & 0.854 & 0.728 & 0.774 \\ Glove & 0.885 & 0.862 & 0.792 & 0.828 \\ BERT (ours) & **0.933** & **0.978** & **0.841** & **0.854** \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Detection results generated by different log template representation methods in the case of BGL-TB. improve the performance of the anomaly detection model, because the anomaly detection model could learn the similarities between the two system log data, thereby detecting the anomalies of the two systems. This experiment also further evaluated the performance of LogBD when training with different numbers of target system normal log sequences.