# Structure-Enhanced Prompt Learning for Graph-Based Code Vulnerability Detection
## 4 Experiments
### Result Analysis
#### 4.3.3 RQ3: Structure-Enhanced Prompt Effectiveness  
To evaluate the impact of incorporating structural prompts, we compared our architecture (SE Prompt) with a plain-text baseline (Text Prompt) lacking structural information. To ensure experimental rigor and maintain consistency with RQ2, we fixed all hyperparameter settings and conducted 20 independent runs for each configuration.  
In the experimental results shown in Table 3, the Text Prompt method helps the model better understand and process the semantic information in the input data by introducing specific textual prompts. This approach leads to notable performance improvements on the FFmpeg\(+\)Qemu dataset, where it achieves an accuracy of 61.32%, precision of 60.41%, recall of 59.71% and F1 score of 59.53%.  
In contrast, SE Prompt, which combines structured information such as graph structures or domain knowledge, provides richer contextual information, further enhancing the model's performance. On the FFmpeg\(+\)Qemu dataset, the four evaluation metrics of SE Prompt are significantly better than text prompts. This result indicates that structured enhancement of prompts offers substantial advantages in improving accuracy and consistency.  
The observed performance trends in the SVulD and Reveal datasets align closely with those in the FFmpeg\(+\)Qemu dataset, thereby validating the effectiveness and generalizability of our method across diverse vulnerability detection tasks.  
Our results were validated through statistical tests, using the \(p\)-value (Mann-Whitney U) and effect size metrics. As shown in Table 4, all \(p\)-values are less than 0.001, indicating that  
\begin{table}
\begin{tabular}{c l l l l l} \hline \hline
**Dataset** & **Method** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline \multirow{3}{*}{FFmpeg\(+\)Qemu} & GCN & 57.14 \(\pm\) 0.18 & 55.69 \(\pm\) 0.18 & 54.60 \(\pm\) 0.04 & 53.52 \(\pm\) 0.21 \\  & CNN-GCN & 58.05 \(\pm\) 1.12 & 56.95 \(\pm\) 1.36 & 58.05 \(\pm\) 1.12 & 55.47 \(\pm\) 1.44 \\  & Text Prompt & 61.32 \(\pm\) 0.34 & 60.41 \(\pm\) 0.35 & 59.71 \(\pm\) 0.89 & 59.53 \(\pm\) 0.67 \\  & SE Prompt & 64.40 \(\pm\) 0.50 & 63.59 \(\pm\) 0.47 & 63.33 \(\pm\) 0.48 & 63.41 \(\pm\) 0.49 \\ \hline \multirow{3}{*}{SVulD} & GCN & 82.36 \(\pm\) 0.89 & 71.96 \(\pm\) 0.72 & 57.56 \(\pm\) 0.38 & 59.31 \(\pm\) 0.68 \\  & CNN-GCN & 82.52 \(\pm\) 0.68 & 74.65 \(\pm\) 1.45 & 57.66 \(\pm\) 0.97 & 59.39 \(\pm\) 0.92 \\  & Text Prompt & 82.69 \(\pm\) 0.15 & 75.59 \(\pm\) 0.29 & 65.60 \(\pm\) 0.96 & 68.37 \(\pm\) 0.84 \\  & SE Prompt & 83.44 \(\pm\) 0.18 & 75.89 \(\pm\) 0.53 & 67.80 \(\pm\) 0.69 & 70.69 \(\pm\) 0.60 \\ \hline \multirow{3}{*}{Reveal} & GCN & 88.90 \(\pm\) 0.15 & 44.61 \(\pm\) 0.14 & 50.00 \(\pm\) 0.15 & 47.15 \(\pm\) 0.02 \\  & CNN-GCN & 89.21 \(\pm\) 0.35 & 55.35 \(\pm\) 1.62 & 50.08 \(\pm\) 0.89 & 50.42 \(\pm\) 1.69 \\ \cline{1-1}  & Text Prompt & 89.69 \(\pm\) 0.06 & 61.68 \(\pm\) 1.16 & 51.76 \(\pm\) 0.58 & 52.01 \(\pm\) 1.25 \\ \cline{1-1}  & SE Prompt & 90.69 \(\pm\) 0.62 & 56.63 \(\pm\) 0.51 & 55.14 \(\pm\) 0.55 & 56.11 \(\pm\) 0.66 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Vulnerability detection results (%) on three datasets (mean \(\pm\) std over 20 runs).  
our method significantly outperforms the three baseline models across all datasets and metrics. The consistently large effect sizes (reaching 1.00) further underscore the practical and theoretical superiority of our approach.  
To further validate the effectiveness of SE Prompt, we present the ROC curves of all models across the three datasets in Figure 11. SE Prompt consistently produces curves closer to the top-left corner, indicating higher true-positive rates across a wide range of thresholds. This effect is particularly evident on the FFmpeg\(+\)Qemu and SVuID datasets, suggesting that incorporating structural information significantly enhances the model's discriminative power. Furthermore, SE Prompt achieves consistently higher AUC values across all datasets, reinforcing its robustness and strong generalization capability.