# Android Malware Detection Based on Hypergraph Neural Networks
## 5 Evaluation
### Evaluation Results  
_Malnet-Tiny:_ Since the samples of different categories in Malnet-Tiny are balanced, the accuracy rate is chosen as a metric to measure the strengths and weaknesses of the methods in this paper. In this task, the baseline methods selected in this paper include mapping-based methods, SLaq-VNGE [36] and SLaq-LSD [36]; feature-based methods, Feather [36], NoG [36], and LDP [36]; and graph neural network-based methods, GCN [36] and GIN [36]. In the study [36], GCN is implemented based on the PYG framework. For a fair comparison, this paper reproduces the GCN and GIN methods in the above hypergraph graph classification framework, in line with the study [36], setting five-layer GCN and GIN with a learning rate of 0.0001, and the implicit layer dimension is set to 64. The GCN and GIN methods reproduced in this paper have accuracy of 75% and 83%, respectively. For the hypergraph-based methods, this paper evaluates five-layer HGNN, HyperGCN, UniGNN, and HGNNP as hypergraph convolution modules to extract hypergraph features, and the pooling layer was uniformly chosen with an average pooling operator. The learning rate is set to 0.001, and the hidden layer dimension is 64. As shown in Table 3, HGNNP obtains the best classification performance of 91.10%, which outperforms all the feature-based, graph-spectral-based, and GNN-based methods. Compared to the GNN-based methods GCN and GIN, all hypergraph-based methods achieve a superior classification performance. HGNN, HyperGCN, and HGNNP are spectral-based convolutional neural networks in the hypergraph approaches, with improvements of 12.2%, 3.9%, and 15.7%, respectively, compared to the spectral-based GCN. The relatively poor results of HyperGCN among the hypergraph-based methods may stem from the high number of isolated vertices in the constructed hypergraph dataset, as HyperGCN needs to discard isolated vertices [26]. For UniGNN, we choose UniGIN to compare with the GNN-based GIN. The results show that UniGIN gained a 5.9% performance improvement over GIN. It can therefore be concluded that modeling function call relations using hypergraphs can bring gains for android malicious detection tasks, reflecting the advantages of hypergraph modeling of higher-order function call relations. _Drebin:_ Due to the unbalanced sample of categories in Drebin, in this task, in addition to accuracy, we add precision, recall, and F1 score as metrics for comparison. As mentioned in Section 5.1, the top 20 categories in terms of sample size were selected for malicious family classification, with a total of 4664 malicious samples. This experiment was able to fully extract the centrality features of the function as well as the functional features, and the construction of two forms of hyperedges. In this experiment, graph neural network-based methods are used as baseline algorithms in this paper, including GCN, GIN, and Graph-SAGE. Specifically, the following settings are adopted for the above graph-based methods: the number of convolutional layers is 3, the learning rate is 0.001, the hidden layer dimension is 300, and the batch size is 64. For the hypergraph-based methods, HyperGCN, UniGNN, HGNN, and HGNNP, with specific settings of 3 convolutional layers, the learning \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Family** & **Samples** & **Family** & **Samples** \\ \hline Fakelnstaller & 925 & Adrd & 91 \\ DroidKungFu & 667 & DroidDream & 81 \\ Plankton & 625 & LinuxLotoor & 70 \\ Opfake & 613 & GoldDream & 69 \\ GingerMaster & 339 & MobileTx & 69 \\ BaseBridge & 330 & FakeRun & 61 \\ Iconosys & 152 & SendPay & 59 \\ Kmin & 147 & Gappusin & 58 \\ FakeDoc & 132 & Imlog & 43 \\ Geinimi & 92 & SMSreg & 41 \\ \hline Total & - & - & 4664 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Top 20 malicious families by the number of samples of Drebin. rate of 0.0001, the hidden layer dimension of 300, and the batch size of 64. All methods use the Adam optimizer as well as the average pooling layer.