# MDFULog: Multi-Feature Deep Fusion of Unstable Log Anomaly Detection Model
## 3 Methods
### Log Vectorization  
The authors found that multiple anomalies could not be detected by log semantics alone. Therefore, time information needs to be introduced as another log feature to detect faults. After log parsing, the session is built through a sliding window. The log sequence is converted into the semantic feature \(F\) and time feature \(T\). Next, the semantic and temporal features are encoded. **Semantic features:** When representing a log, the normal log should be close enough, and the semantic vectors should be as evenly distributed on the hypersphere as possible. The reason is that the uniform distribution has the highest information entropy. The more uniform the distribution is, the more information remains. The semantic vector obtained by contrastive learning can meet the above requirements. Therefore, the [PERSON] model with contrastive learning embedded can improve the noise caused by the update of log statements and enhance the robustness of the model. The cross-entropy function of contrast learning was used as a loss function to adjust the parameters of [PERSON]'s pre-trained language model. The cross-entropy function of contrast learning can make the model learn rich and accurate semantic features. The formula is  
\[\text{loss}_{i}=-\left(\log\frac{e^{\frac{\cos\left(f_{i}j_{i}^{+}\right)}{ \rho}}}{e^{\frac{\cos\left(f_{i}j_{i}^{+}\right)}{\rho}}+\sum_{j=1}^{a}e^{ \frac{\cos\left(f_{i}j_{i}^{+}\right)}{\rho}}}+\log\frac{e^{\frac{\cos\left(f_{ i}j_{i}^{+}\right)}{\rho}}}{e^{\frac{\cos\left(f_{i}j_{i}^{+}\right)}{\rho}}+\sum_{j=1}^{a}e^{ \frac{\cos\left(f_{i}j_{i}^{+}\right)}{\rho}}}\right), \tag{3}\]  
\[\text{loss}_{i}^{*}=-\log\frac{e^{\frac{\cos\left(f_{i}^{+}j_{i}^{+}\right)}{ \rho}}}{e^{\frac{\cos\left(f_{i}^{+}j_{i}^{-}\right)}{\rho}}+\sum_{j=1}^{a}e^{ \frac{\cos\left(f_{i}^{+}j_{i}^{-}\right)}{\rho}}}, \tag{4}\]  
where \(f_{i}\) denotes the semantic vector of the common log template \(lt_{i}\), \(f_{i}^{+}\) is the semantic vector of the common log template \(lt_{i}^{+}\) that has a context with \(lt_{i}\), \(f_{i}^{*}\) is the semantic vector of the noisy log template \(lt_{i}^{*}\), \(f_{a}\) is the semantic vector of the abnormal log template \(lt_{a}\), \(a\) is the number of abnormal log templates, and \(\rho\) is an adjustable parameter. The Bert structure is shown in Figure 2. \(e_{1},e_{2},\ldots,e_{n}\) is the input sequence of the Bert model. TM is the Encoder model of the Transformer, and \(x_{1},x_{2},\ldots,x_{n}\) is [PERSON]'s output word vector sequence. [PERSON]'s input sequence is shown in Figure 3, using [CLS] to mark the starting position of the log message and sending each word (token) in the log template to the token embedding layer to transform each word into a vector form. The token embedding layer transforms each word into a fixed-dimension vector. In [PERSON], each word is transformed into a 768-dimensional vector representation. Segment embedding is a vector representation that distinguishes two sentences when the input sequence is a sentence pair. Position embedding is the position information of each word, which is used to solve the problem that token embedding cannot encode the position information of the input sequence, i.e., to assign different semantics to the same word at different positions. Then, the embedded vector \(e_{1},e_{2},\ldots,e_{n}\) is input into the transformer encoder (TM) as the model input, and finally, the log semantic vector \(F\) is obtained. **Time characteristics:** It is worth paying attention to the time interval between two logs if an abnormalexception appears at runtime. Potential threats may cause the interval. Each log message contains a timestamp, which can be used to calculate the time interval between two log messages. Consider time interval sequence \(\Delta T=\left\{\Delta t_{1},\Delta t_{2},\ldots\Delta t_{i}\ldots\right\}\), and let \(\Delta t_{1}\) to \(-1\), \(\Delta t_{i}\) be the timestamp of \(l_{i}\) minus the timestamp of \(l_{i-1}\).