## 4 The Proposed Method
### Malware Detection Model  
The malware detection model uses the ViT model proposed by [PERSON] et al. (2018). The ViT model consists of multiple layers of Transformer Encoders. The following Figure 2 shows the structure of an encoder block used in the ViT. The model consists of \(L\) encoder layers. For each layer, the input is the previous layer's output, and the input of the first layer is the model's input. The 'Embedded Patches' at the bottom of Figure 2 are the input of the ViT model. The encoder block of a layer consists of a normalized multi-head self-attention (MSA) and a normalized multi-layer perceptron (MLP). Between MSA and MLP, the residual connection exists. ViT uses a number of transformer encoder blocks to learn relationships between image patches and uses MLP, an output layer, to make the model's output. The DEX image created in Section 4.2 is used as input to the model. The image shape is \((H,W,C)\), where \(H\) is the height of the image, \(W\) is the width of the image and \(C\) is the number of channels. The image is divided into patches that are used as model input. The image is divided into patches of size \((P,P)\), where \(P\) is the size of the patch. The number of patches, \(N\), is calculated by Equation (2) ([PERSON] et al., 2018). \[N=\frac{HW}{P^{2}} \tag{2}\]  
The generated patch is \(x_{p}^{i}\), \((1\leq i\leq N)\). The shape of \(x_{p}^{i}\) is \(x_{p}^{i}\in\mathbb{R}^{P^{2}C}\). To enter the patches into the model, embedding is applied, as shown in Equation (3). \[z_{0}=[x_{cls};x_{p}^{1}E;x_{p}^{2}E;\cdot\cdot\cdot;x_{p}^{N}E]+E_{pos},\;E\in \mathbb{R}^{(P^{2}\cdot C)\times D},\;E_{pos}\in\mathbb{R}^{(N+1)\times D} \tag{3}\]  
where \(z_{0}\) is the initial input to the model. To embed patches, we use patch projection with \(E\). \(E\) is a matrix. \(D\) is the embedding dimension, so patches map to \(D\) dimensions. The position embedding, \(E_{pos}\), is added. The class token \(x_{cls}\) is added to the first position of the input. Equation (4) is a set of layer normalization (LN) and the MSA in an encoder block. First, layer normalization is performed on the encoder's input, as shown in the equation. LN normalizes each sample's features. For more information about LN, refer to [PERSON] et al. (2018), [PERSON] et al. (2018). Then, MSA is performed. For more information about MSA, refer to [PERSON] et al. (2018). The input and output of a set of LN and MSA are connected by a residual connection. The result of the residual connection \(z_{l-1}^{{}^{\prime}}\) goes into the input of a set of LN and MLP in Equation (5). \[z_{l-1}^{{}^{\prime}}=MSA(LN(z_{l-1}))+z_{l-1},\;l=1\cdot\cdot L \tag{4}\]  
Figure 2: The Transformer Encoder used in Vision Transformer ([PERSON] et al., 2018). where \(L\) is the total number of encoder layers in the model. Equation (5) is a set of LN and MLP in the encoder block. The input and output of a set of LN and MLP are connected by a residual connection. The result of the residual connection \(z_{l}\) is an output of the encoder block. \[z_{l}=MLP(LN(z^{\prime}_{l-1}))+z^{\prime}_{l-1},\;l=1\cdots L \tag{5}\]  
where \(L\) is the total number of encoder layers in the model. The output of each Transformer Encoder is \(z_{l}\). The \(z_{l}\) becomes the input of the next layer. An attention weight \(A\) is generated in the MSA from Equation (6) as follows. \[A=softmax(\frac{QK^{T}}{\sqrt{D_{h}}}) \tag{6}\]  
where \(Q\) is the query and \(K\) is the key. \(D_{h}\) is \(D/k\) and \(k\) are the number of heads [22]. The attention weight \(A\) is used to create the attention map in Section 4.4.2. The following Equation (7) finally generates a detection result in ViT. \[\widehat{y}=MLP(z_{L}^{0}) \tag{7}\]  
where \(z_{L}^{0}\) is the first row of \(z_{L}\). After iterating over the Transformer Encoders \(L\) times, the final output is \(z_{L}\in\mathbb{R}^{(N+1)\times D}\). Since the value of the model input \(z_{0}\) is a class token in Equation (3), \(z_{L}^{0}\) in \(z_{L}\) is the class token.