## 3 Proposed Models
### Log Parsing and Preprocessing
#### 3.2.2 Log Sequence Feature Matrix Generation  
* **Log sequence vector:*
* Each dataset has a unique identifier. In the HDFS dataset, the block_id is set as the identifier for a specific sequence of operations. Log events are grouped on the strength of these identifiers, or log items generated by concurrent processes are broken into separate and single-threaded sequential sequences. Each row in Figure 4 refers to a log execution sequence, and each number (ID) in the execution sequence indicates a log. All logs in each log execution sequence have the same block_id. Logs with identical block_ids are arranged together in order to form a log sequence vector. All of these efforts help detect exceptions when the log execution sequence is incorrect.
* **Word vector:*
* Word2 Vec is an effective method of embedding words that takes the context of the words into account during training to establish a language model f(x) = y between the goal word (x) and its context (y). In this way, the targeted word is obtained according to the context word or vice versa. Word2 Vec is performed in LTAnomaly to effectively process semantic information in log messages. Word2 Vec requires words as input, so we need to first split the log template content. The log can be separated by spaces or other methods. For instance, \"PacketResponder <*> for block <*> terminating\" is divided into four words: \"PacketResponder\", \"for\", \"block\", and \"terminating.\" Then, we use Word2 Vec to predict targeted words from context words. As shown in Figure 5, one-hot encoding was performed on the top \(n\) words and the bottom \(n\) words of the target word. Next, the hidden and softmax layers are used to predict the targeted word, obtaining a word vector for each word. The word vectors of all words form a word vector matrix \(V\in\mathbb{R}^{a\times b}\) according to their order of appearance in the log sequence vector.
* **Word sequence weight matrix**: TF-IDF is a popular weighting technique implemented in both information search and data mining. The importance of words in a sentence can be measurable by TF-IDF, which meets the demand for high resolution. A word's weight tends to grow according to its quantity in the log sequence. However, if a word occurs frequently, the weight of the word decreases. For instance, if the word \"Block\" occurs regularly in a log event, then the word may better describe the log event. In contrast, if \"Block\" occurs in whole log events, then it is not possible to distinguish log events based on this word, so its weight should be reduced. Therefore, we denote significance with the term frequency (TF) and measure the occurrence of log events by the inverse document frequency (IDF). These are calculated as \[TF(word_{i})=\frac{n_{i}}{\sum_{k}n_{k}},\] (1) and \[IDF(word_{i})=log\bigg{(}\frac{s_{i}}{s+1}\bigg{)},\] (2) where \(n_{i}\) is the number of occurrences of \(word_{i}\), \(\sum_{k}n_{k}\) is summed over words in whole log sequences, and \(s\) and \(s_{i}\), respectively, denote the sum of log sequences and the sum of log sequences with \(word_{i}\). The TF-IDF weight W of each word is computed as  
Figure 4: Log sequence vector example.  
* **Log sequence feature matrix**: To produce the log sequence feature matrix, we multiply the word vector matrix V by the word sequence weight matrix W to obtain the log sequence feature vector \(Ew\in\mathbb{R}^{c\times b}\), i.e., \[Ew=W\times V.\] (3) Therefore, the log sequence characteristic matrix can pay attention not only to exceptions that occur in the log execution order but also to the semantic information of the log. This method represents log messages more accurately than existing log representations.