# LogBD: A Log Anomaly Detection Method Based on Pretrained Models and Domain Adaptation
## 3 Methodology
### Anomaly Detection Based on Domain Adaptation
#### 3.4.2 Adversarial Learning  
Although LogBD uses a TCN to map log sequences into the hypersphere space, log sequence mappings from different systems can still be located in different regions of the hypersphere space, rather than clustered around a common hypersphere sphere center c, as shown in Figure 7, where \(\mathbb{R}^{128}\) represents a 128-dimensional space.  
Figure 6: Feature extractor processing log sequence representation.  
Therefore, this paper proposes a cross-system data mapping method for adversarial training, which brings log sequence mappings from different systems close to each other. A domain discriminator \(\mathrm{G_{d}}\) is set to distinguish whether the log sequence mapping comes from the source or the target system, and the domain discriminator uses a multilayer perceptron containing two 64-dimensional fully connected neural networks. A logistic function is used for prediction, as follows:  
\[G_{d}(v^{\epsilon})=\sigma\Big{(}w^{\mathrm{T}}v^{\epsilon}+b\Big{)}, \tag{4}\]  
where \(\sigma()\) denotes the logistic function, and w and b are trainable parameters.  
The loss function of the domain discriminator is calculated by the cross-entropy loss function, which is as follows:  
\[loss\Big{(}\hat{d},\hat{d}\Big{)}=-\hat{d}[d]+\log\Bigg{(}\sum_{i}\exp\Big{(} \hat{d}[i]\Big{)}\Bigg{)}, \tag{5}\]  
where d is the true domain label of the log, and \(\hat{d}\) is the domain prediction label of the domain discriminator output.  
The TCN is trained as a feature extractor, i.e., \(\mathrm{v^{\epsilon}=G_{f}(X^{\epsilon})}\), so that it gains the ability to extract features common to both the source and the target systems, making the discriminator unable to distinguish the source system data from the target system data. The adversarial training of the feature extractor and the domain discriminator is achieved by the gradient reversal layer (GRL) [34], where the GRL multiplies the error passed back to the feature extractor by a negative number (\(-\lambda\)) from the domain discriminator to achieve the opposite of the training objective and achieve the adversarial effect. The objective function of adversarial training is as follows:  
\[\mathcal{L}_{adv}=\underset{G_{f}}{\text{minimax}}\left(\frac{\mathbb{E}_{X^{ \mathrm{S}}\sim P_{target}}\Big{[}log\mathrm{G}_{d}\Big{(}G_{f}\big{(}X^{ \mathrm{S}}\big{)}\Big{)}\Big{]}+}{\mathbb{E}_{X^{\mathrm{T}}\sim P_{target}} \Big{[}log\Big{(}1-G_{d}\Big{(}G_{f}\big{(}X^{\mathrm{T}}\big{)}\Big{)}\Big{)} \Big{]}}\right), \tag{6}\]  
where \(X^{\mathrm{S}}\) and \(X^{\mathrm{T}}\) denote the source log sequence and the target log sequence, respectively.  
By minmax optimization training, the log sequence mapping generated by the TCN is able to mislead the domain discriminator, i.e., the distributions of the source and target log sequences are confused. Thus, the objective function training framework can be expressed as the following equation:  
\[\mathcal{L}=\mathcal{L}_{ex}+\lambda\mathcal{L}_{ndc}, \tag{7}\]  
where \(\lambda\) is a hyperparameter that is used to balance these two components. Its value is set to 0.1.  
During the LogBD training process, an optimizer is needed to update the model parameters. In this paper, we use the Adam optimizer to optimize the model, which maintains the average of the gradient of each parameter, along with the average of the  
Figure 7: Different systems data clustered in different regions.  
gradient squared, and then uses these estimates to calculate the adaptive learning rate for each parameter.