## 5 Experiment Results and Analysis
### Experiment for Generalization in Low-Resource Scenarios (Answer to RQ3)
Table 6 quantifies the fluctuations (degree of dispersion) of all the curves in Figure 5 using SD. The analysis is based on the relative performance improvement formula outlined in Section 4.3.3. LePB-SA4 RE achieved a relative improvement of 56% in accuracy and 72% in F1-Score on the BFRU dataset, and a relative improvement of 14% in accuracy and 30% in F1-Score on the Dabrowski dataset. The CI (the formula outlined in Section 4.3.2) is calculated using the 99% critical value based on the t-distribution to further verify the statistical significance of the results and minimize the impact of random errors. The results in Figure 6 indicate that the PT-based BERT model consistently performs better and demonstrates greater robustness compared to the SOTA method across different training set scales, untrained test datasets, and evaluation metrics. To further investigate the differences between Hard-prompt and Soft-prompt approaches, particularly regarding their generalization performance on unseen data, we conduct a comparative experiment. The AWARE dataset is used for training, while BFRU and Dabrowski are selected as zero-shot test sets to evaluate the models' cross-domain generalization capabilities. Since the Soft-prompt technique introduces additional learnable parameters (prompt embeddings), the learning rate is increased to 2 \(\times\) 10\({}^{-4}\) to enable the model to fully optimize these continuous vector representations and find an optimal parameter configuration. As shown in Figure 7, the Soft-prompt model achieves performance comparable to LePB-SA4 RE on the AWARE training set, with minor differences in accuracy (0.67%) and F1-Score (0.68%). This suggests that in supervised learning scenarios, the Soft-prompt model can potentially learn semantic representation spaces analogous to Hard-prompt models through parameter optimization. However, divergent performance characteristics between the two methods become more apparent in zero-shot generalization tests. On the BFRU test set, the Soft-prompt model exhibits lower accuracy and F1-Score. This discrepancy reflects the absence of explicit semantic constraints in Soft-prompting, and the highly flexible prompt embeddings may lead to domain drift. Specifically, the model optimizes for minimal loss in the source domain (AWARE) while deviating from optimal solutions in the target domain (zero-shot test sets). Therefore, the model struggles to accurately capture semantic relationships in new data, resulting in degraded generalization and stability. In contrast, the Hard-prompt model maintains minimal performance degradation, as its static template enforces explicit task guidance and preserves consistent semantic information, thereby sustaining robust cross-domain reasoning capabilities amid data distribution shifts. This advantage becomes more evident on the smaller-scale Dabrowski test set, where the Soft-prompt model demonstrates a steeper performance decline (particularly, a 4.64% F1-Score reduction), highlighting its diminished adaptability to low-resource settings. This limitation may arise from the increased model complexity associated with Soft-prompt, where limited test data fail to sufficiently activate pre-trained prompt vectors. Additionally, small-scale test sets are inherently susceptible to sample distribution sparsity and representation imbalance, challenges that synergize with the Soft-prompt's input sensitivity to amplify information loss during inference. Moreover, the Soft-prompt model displays greater performance instability in parallel experiments. Its F1-Score on test sets plummets to 47% at lower learning rates (2 \(\times\) 10\({}^{-5}\)), emphasizing heightened sensitivity to hyperparameter selection. In summary, the Hard-prompt model demonstrates advantages in stability and generalization capabilities, particularly when handling distribution shifts and low-resource app review settings.