# Phishing Node Detection in Ethereum Transaction Network Using Graph Convolutional Networks
## 5 Evaluation
### Findings
The findings show that if the bagging strategy is not used in the model's training process, the original GCN solution will not be able to cope with extreme data imbalances in the Ethereum transaction network and detect phishing nodes effectively. **Evaluation of the PU problem:** Next, we utilized the spy technique [29] to set false negative examples to evaluate the robustness of our model with respect to the PU problem. In the evaluation, we selected 15% positive examples and set them as negative examples, and we placed them in the training set to simulate the PU problem in the training set. Then, we checked whether these examples that were intentionally marked as negative examples could be detected by the model. Specifically, we evaluated the classification performance of the BM-GCN model with 173 spy nodes at NP ratios of 5, 20, 50, and \"All\". Table 4 shows the model's capability of recovering spy nodes' labels. It also indicates that the AUC value of classification increases as the NP ratio increases, which intuitively reflects the negative impact of unlabeled data. For small datasets, such as when the NP ratio equals 5, there are only 4141 negative examples. Thus, the introduction of 173 unlabeled nodes confuses the model significantly, resulting in the degradation of its performance. However, even in the worst case, 97.6 of the 173 spy nodes are restored successfully by the model. Thus, our model effectively avoids the adverse impact of the unlabeled nodes on the results of the classification. In addition, it illustrates that our model can deal with the PU problem in the Ethereum transaction data. \begin{table}
\begin{tabular}{c c c} \hline
**Features** & **NP-ratio@50** & **NP-ratio@All** \\ \hline \(G_{f}\) & 0.852591 & 0.851575 \\ \(G_{v}\) & 0.863451 & 0.848624 \\ \(G_{i}\) & 0.872630 & 0.867106 \\ \(G_{f}+G_{v}\) & 0.861069 & 0.867045 \\ \(G_{f}+G_{i}\) & 0.867851 & 0.855019 \\ \(G_{v}+G_{i}\) & 0.869612 & 0.875120 \\ \(G_{i}+G_{f}+G_{v}\) & **0.883325** & **0.875443** \\ \hline \end{tabular}
\end{table} 
Table 2: AUC with different features \begin{table}
\begin{tabular}{c c c} \hline
**NP Ratio** & **No Bagging** & **Bagging** \\ \hline
5 & 0.855752 & 0.870561 \\
10 & 0.798114 & 0.880036 \\
20 & 0.745765 & 0.877302 \\
50 & 0.495823 & 0.883325 \\ \hline \end{tabular}
\end{table} 
Table 3: AUC without bagging strategy  
**Baseline evaluation:** Figure 6 shows the aggregate AUC of our approach and all of the baselines with varying NP ratios. We can see that the model (GCN with \(G_{i}+G_{f}+G_{v}\)) outperforms all of the baseline systems. First, on the entire range of NP ratios, our model achieves a higher AUC than all of the baselines. Second, the BM-GCN model achieves an average AUC of 0.877, whereas the Deepwalk-LR only achieves an average AUC of 0.661. Thus, we conclude that our BM-GCN model uses more transaction information than other models. Moreover, our model is more robust than all of the baseline models. For example, Figure 6 shows that the performance of all the baselines decreased rapidly as the NP ratio increased, but our scheme remained stable. This implies the potential in using our BM-GCN model for larger datasets. **Comparison with other methods:** Table 5 shows the comparison between the proposed method and other methods in terms of the AUC metric. All the methods use transaction data for phishing node detection. However, compared to other methods that directly use raw transaction data or relevant statistical features, BM-GCN extracts the global structural features and preprocesses the raw transaction information into three types of interactive information, i.e., node interaction, time variance, and transaction frequency. As shown in Table 5, our method achieves the best results. \begin{table}
\begin{tabular}{c c c} \hline \hline
**NP Ratio** & **Restored Nodes** & **AUC** \\ \hline
5 & 97.6 & 0.819079 \\
20 & 115.8 & 0.852759 \\
50 & 123.0 & 0.858245 \\ All & 133.6 & 0.870821 \\ \hline \hline \end{tabular}
\end{table}