# Detecting Rug-Pull: Analyzing Smart Contract Backdoor Codes in Ethereum
## 4 Evaluation and Discussion
### Evaluation of Backdoor Code Detection Accuracy  
Through this experiment, a total of 989 EVM codes were inspected for each of the six attack types defined in Section 2.2. This total includes 200 EVM codes prepared for the evaluation of each attack type and an additional 189 actual backdoor EVM code samples extracted directly from real-world data registered on the Ethereum blockchain. The accuracy of the model was evaluated by comparing the inspection results with the presence of actual backdoor codes. For the experimental results, a confusion matrix was used, as shown in Table 4. The confusion matrix consists of the following four elements. The frequency of accurately detecting the actual backdoor code type (TP: true positives). The frequency of correctly detecting normal code without backdoor code (TN: true negatives). The frequency of incorrectly detecting normal code as backdoor code (FP: false positives). The frequency of incorrectly detecting backdoor code as normal code (FN: false negatives). Based on these values, accuracy, precision, and recall for each type were calculated using Equations (1)-(3).  
\[\text{Accuracy}=(\text{TP}+\text{TN})/(\text{TP}+\text{FP}+\text{FN}+\text{ TN}) \tag{1}\]  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Item** & **Specification or Version** \\ \hline  & Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40 GHz (Santa Clara, \\ CPU & CA, USA) \\ RAM & 32 GB \\ OS & Rocky Linux 9.2 \\ Python & Python 3.9.16 \\ Decompiler & Panoramix Decompiler 0.6.1 \\ Ethereum Signature Database & Access date: 20 October 2024 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Server Specifications and Software Versions.  
Precision = TP/(TP + FP) (2)  
Recall = TP/(TP + FN) (3)  
As shown in Table 4, the proposed model achieved high accuracy across all backdoor code types, ranging from 97.4% to 99.4%. The precision values varied between 0.95 and 0.98, while recall values ranged from 0.95 to 1.0. These metrics indicate that the backdoor codes were accurately detected with minimal false positives and false negatives. Notably, the model achieved a recall of 1.0 for the proxy backdoor code type, meaning it correctly identified all instances of this backdoor without missing any. This indicates that the proposed model can effectively identify backdoor codes in smart contracts. In addition, it accurately detected all backdoor codes in smart contracts where multiple backdoor codes existed simultaneously. These results demonstrate that the model can effectively respond even when an attacker performs a complex attack by combining multiple backdoors.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Backdoor Code Type** & **TP** & **FP** & **FN** & **TN** & **Accuracy** & **Precision** & **Recall** \\ \hline Token generation & 228 & 5 & 6 & 750 & 98.8\% & 0.97 & 0.97 \\ Destroy token & 226 & 9 & 3 & 751 & 98.7\% & 0.96 & 0.98 \\ Transaction limitation & 309 & 12 & 13 & 655 & 97.4\% & 0.96 & 0.95 \\ Funds manipulation & 202 & 9 & 1 & 778 & 98.9\% & 0.95 & 0.99 \\ Fee & 200 & 4 & 2 & 783 & 99.3\% & 0.98 & 0.99 \\ Proxy & 200 & 5 & - & 784 & 99.4\% & 0.97 & 1.0 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Confusion Matrix and Accuracy for Backdoor Code Detection.  
Figure 12: The operation results of the Balance-Tracking-Based Backdoor Detection Model. (a) Operation log. (b) Contents of the inspection_results.csv file.