## 4 Experiments and Discussion
### Comparison with Previous Methods  
The comparison on Dataset One between our methods and the previous studies is given in Tables 4 and 5. In Table 4, we chose two different kinds of methods to report the results. The first five methods are classic methods [14; 44; 45; 46; 47] to do the malware family classification, and we report the results from their papers. The following five methods [16; 20; 21; 23; 48] are the latest effective work on the classification based on API calls, so we reproduce the methods and offer a convincing comparison result. The [21] method adopts a two-way feature extraction architecture for API calls, but the core module is a multi-layer CNN, and the correlation analysis is performed through Bi-LSTM. Our architecture is unified based on Transformer and attention mechanisms, and so by comparing with this method, it can reflect the role of the backbone network. The study [48] further adopted a pre-training mechanism and integrated multiple Transformer architecture models through Random Forest, which has similarities with our backbone network. Moreover, it uses the process of integrating Random Forest, which is like Step 2. However, we implemented this process through the Normalization Vector. So, comparing this work with others can reflect the role of our mechanisms and strategies. We chose these two works as the latest and most effective API call classification model, which can also give a similarity comparison with our model details. In Table 5, we also compare these five methods with a new method [49] since we can reproduce this work with our cuckoo analysis results.  
From Table 4, as can be observed, the first five methods give a baseline for classification tasks, and models based on malware images or traditional NLP methods can achieve accuracy around 0.90. The model of [21] achieved an accuracy of 0.90, the model of [48], achieved an accuracy of 0.93, and our model had an accuracy of 0.96. From a baseline perspective, all three models go beyond other basic or advanced methods and achieve a better result, demonstrating the effectiveness of the API calls and Encoder/Transformer architecture models. From the perspective of SOTA, our model performs much better than the two currently optimal models, indicating the superiority of our model.  
\begin{table}
\begin{tabular}{l l l} \hline
**Category** & **Subcategory** & **Ratio (\%)** \\ \hline Backdoor & & 3.37 \\ \hline \multirow{4}{*}{Worm} & Worm & 3.32 \\  & Email-Worm & 0.55 \\  & Net-Worm & 0.79 \\  & P2P-Worm & 0.30 \\ \hline Packed & & 5.57 \\ \hline \multirow{4}{*}{PUP} & Adware & 13.63 \\  & Downloader & 2.94 \\  & WebToolbar & 1.22 \\ \hline \multirow{8}{*}{Trojan} & Trojan (Generic) & 29.3 \\  & Trojan-Banker & 0.14 \\  & Trojan-Clicker & 0.12 \\  & Trojan-Downloader & 2.29 \\  & Trojan-Dropper & 1.91 \\  & Trojan-FakeAV & 18.8 \\  & Trojan-GameThief & 0.63 \\  & Trojan-PSW & 3.79 \\  & Trojan-Ransom & 2.58 \\  & Trojan-Spy & 3.12 \\ \hline Misc. & & 5.52 \\ \hline \end{tabular}
\end{table}
Table 2: Dataset One description.  
\begin{table}
\begin{tabular}{l l l} \hline
**Category** & **Subcategory** & **Ratio (\%)** \\ \hline Backdoor & & 27.30 \\ \hline \multirow{4}{*}{Worm} & Email-Worm & 1.71 \\  & Net-Worm & 1.00 \\ \hline \multirow{4}{*}{Trojan} & Trojan (Generic) & 29.3 \\  & Trojan-Banker & 1.61 \\  & Trojan-Clicker & 1.90 \\  & Trojan-Downloader & 18.46 \\  & Trojan-Dropper & 7.18 \\  & Trojan-GameThief & 18.11 \\  & Trojan-PSW & 8.47 \\  & Trojan-Proxy & 1.23 \\  & Trojan-Spy & 8.40 \\ \hline Virus & & 1.77 \\ \hline Exploit & & 1.07 \\ \hline Rootkit & & 0.50 \\ \hline HackTool & & 1.28 \\ \hline \end{tabular}
\end{table}
Table 3: Dataset Two description.  
In Table 5, we further compared the performance of the latest seven methods on the second dataset to demonstrate the robustness and broad performance of the model. From Table 5, the performance of all three models has decreased on the second dataset, but our proposed method still performs the best.