# Phishing Node Detection in Ethereum Transaction Network Using Graph Convolutional Networks
## 5 Evaluation
### Findings  
In our evaluations, we followed the guideline in [28] to set our bagging parameters, which were \(T=100\) and \(K=1165\). The parameters of our GCN were as follows: the number of hidden layers was 3, and the units of hidden layers 1, 2, and 3 were 16, 16, and 8, respectively. The maximum epoch number per bag was 20, the learning rate was 0.01, and the dropout rate was set to 0.5. We selected the value of the NP ratio among the following: 5, 10, 20, 50, 100, 200, 500, and \"All\", where \"All\" means that we used all nodes in the experiment (the NP ratio was 2,972,324:1165). **Evaluation of [PERSON] et al. 's method:** Figure 5 shows the performance of [PERSON] et al. 's approach [8] for various NP ratios and graph scales. For each scale, we constructed three test graphs following the approach, and we used their average AUC when evaluating the performance. The average node and edge numbers are provided in Table 1. The following two limitations can be observed in their scheme. 1. As the NP ratio increases, the performance of their scheme decreases consistently. Specifically, the average classification AUC value of Trans2 vec decreased from 0.886 to 0.732 when the NP ratio increased from 1 to 25. In other words, Trans2 vec is not capable of dealing with data imbalances. 2. As the network scales, the performance of their model decreases gradually. In other words, the scale of the network impacts the node representation capabilities of their scheme and degrades the classification performance (i.e., Trans2 vec is not inadequate for large-scale transaction graphs). **Effect of different features:** We evaluated the effect of different feature combinations on the proposed BM-GCN model using two NP ratios, i.e., 50 and \"All\". Table 2 displays the aggregate AUC of the GCN with different feature combinations. We observe that the classification performance is most significantly improved by the feature \(G_{i}\) out of the three analyzed. For multiple feature combinations, we found that the combination of \(G_{i}\), \(G_{f}\), and \(G_{v}\) worked best. \(G_{i}\) is an indicator that describes the total number of transactions between nodes, which evidently reflects the closeness of the relationships between nodes. It outperforms the other two. \(G_{f}\) and \(G_{v}\) describe the time properties of transactions from the perspective of transaction frequency and changes in transaction time. This combination effectively improves the AUC value as they complement each other. The combination of all three features allows us to achieve the best classification performance. This implies that \begin{table}
\begin{tabular}{c c c} \hline \hline
**Scale** & **Node Number** & **Edge Number** \\ \hline
1 & 32,582 & 70,082 \\
3 & 39,606 & 95,154 \\
5 & 45,397 & 134,552 \\
10 & 59,250 & 188,875 \\
15 & 76,344 & 241,639 \\
20 & 90,388 & 283,260 \\
30 & 119,368 & 375,159 \\
50 & 162,388 & 535,819 \\ All & 2,973,489 & 13,551,303 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Average scale of different test graphs  
Figure 5: Curves of average AUC of [PERSON] et al.’s model [8], with varying NP ratios. Each curve represents a network scale, and it refers to the number after “G” in the legend. It indicates the proportion of positive and negative examples when the network is initialized. the features reflect the topological characteristics of the nodes to a certain extent, and our transaction feature extraction scheme is effective. **Bagging vs. no bagging:** In this section, we maintain the NP ratio to evaluate the impact of removing the bagging strategy on the classification performance of the model. As shown in Table 3, in the case of no bagging, the classification performance of the model decreases rapidly as the NP ratio increases. Even when the NP ratio is equal to 50, the AUC of the model drops below 0.5.