# A Malware Detection Framework Based on Semantic Information of Behavioral Features
## 3 Methodology
### Feature Extraction and Generation  
#### 3.2.1 Sequence-Based Embedding  
Word embedding [49] is the representation of a word in an n-dimensional vector space. Within the context of malware detection, the sequence of calls to API in malware is not arbitrarily present. It reflects some contextual pattern of malicious activity. These patterns have similar manifestations across different malware. We can identify the contextual relationships among API function sequences by learning these patterns from a mount of malicious sequences.  
Since the possible contextual relationships between API calls cannot be captured by statistical methods alone, inspired by word embeddings used in NLP, we initialize the embedding matrix with pre-trained word embeddings rather than random initialization to give semantic meaning to the word vectors. There are various models used in Word Embeddings, one of them is the Word2 Vec model, which represents words as vectors based on several features they have, such as window size and vector size. Word2 Vec captures the similarity values between words from the training of a large corpus. The resulting similarity values are computed from the word vector values then using the cosine similarity formula. Thus, similar words tend to have the same vector values and are grouped into the same block. Word2 Vec is considered to be one of the most commonly used forms of word embeddings ([PERSON] et al. [50]; [PERSON] et al. [51]; [PERSON] al. [52]), which contains both CBOW and Skip-Gram models. The core idea of the CBOW model is to use  
Figure 1: Overview of proposed model.  
the context before and after a keyword to predict the probability of the occurrence of that keyword, as shown in Equation (1). Skip-Gram model is just the opposite; it is the use of a keyword to predict the probability of the words that appear before and after the keyword, and the model's target formula is shown in Equation (2). Unlike the CBOW model, the Skip-Gram will output multiple probability distributions from the hidden layer to the output layer.  
\[P(\omega_{c}|\omega_{o_{1}},\dots,\omega_{o_{2}m})=\frac{exp\Big{(}\frac{1}{2m}u _{c}^{\top}(\
u_{o_{1}}+\dots+\
u_{o_{2}m})\Big{)}}{\sum_{i\in V}exp\Big{(} \frac{1}{2m}u_{i}^{\top}(\
u_{o_{1}}+\dots+\
u_{o_{2}m})\Big{)}} \tag{1}\]  
\[P(\omega_{o}|\omega_{c})=\frac{exp\big{(}u_{o}^{\top}v_{c}\big{)}}{\sum_{i\in V }exp\big{(}u_{i}^{\top}v_{c}\big{)}} \tag{2}\]  
where \(\omega_{o},\omega_{c}\) represent context words and center words, respectively, and \(u_{o},v_{c}\) represent context word vectors and center vectors. Here, the window size is \(m\) and \(V\) is the whole lexicon.  
In this paper, we use the Skip-Gram model (as shown in Figure 2), a shallow single-layer neural network, to capture the connection of hidden contexts between features and to efficiently model API sequence patterns. In this study, the pre-processed API call sequences are used as a corpus for training, which is trained by predicting the \(C\) words of the word's context by the target word to obtain the encoding of all APIs.  
First, each API call is vectorized using the One-Hot algorithm, and each vector's length \(V\) is the amount of unique APIs counted. Row number of the element corresponding to the index position of a word in the vocabulary is 1, and other elements are 0. Then, word embedding is performed. According to the index mapping, each word is mapped to the N-dimensional space so that all words can be mapped to the matrix \(W\in R^{V\times N}\), where every column in the matrix matches a word. Finally, the output is multiplied with the weight matrix \(W^{\prime}\in R^{N\times V}\), to acquire a probability distribution. The ultimate goal of model training is to obtain a weight matrix \(W\), in which each row represents a lower dimensional vector of different APIs. In this way, the sequence \(r\) in which each API is expressed as a lower dimensional vector \(\Phi(V_{i})\). And for out-of-vocabulary (OOV) words, we use '0' as padding.