## 5 Experiment Results and Analysis
### Empirical Exploration (Answer to RQ5)
Secondly, the standard deviations of large models are generally higher than base models. This is primarily because larger models, with more parameters and deeper network structures, are better at capturing complex patterns and relationships in training data. However, this adaptability also increases the likelihood of learning incidental patterns (noise), negatively impacting the model's understanding ability to generalize to underlying data structures. As a result, large models are generally less stable than base models when applied to new datasets. Lastly, large models utilizing whole-word masking (such as LUC-W and LC-W) demonstrate superior generalization on the BFRU dataset. Specifically, LUC-W achieves the highest F1-Score, surpassing that of LUC, and outperforms all models except our proposed framework. Whole-word masking treats entire words as single units rather than splitting them into subword units, which enables more effective learning of complete word semantics. This training strategy helps mitigate semantic misunderstandings caused by improper word segmentation in new or complex data. As a result, whole-word masking improves the both generalization and robustness to diverse inputs. Furthermore, the figures consistently demonstrate that the BERT-Large model within the LePB-SA4 RE framework achieves superior overall performance. However, the combination of Large model and Bayesian optimization introduces substantial computational overhead. The training of the BERT-Large model requires massive computational resources, while Bayesian optimization demands multiple iterations to converge to optimal hyperparameter configurations, rendering the entire process both time-consuming and computationally intensive. This may become a primary bottleneck for practical deployment, especially in large-scale industrial applications that necessitate frequent model updates and retraining. Therefore, finding an optimal balance between computational costs and deployment feasibility emerges as a critical issue. To address the scalability challenges, several promising theoretical and technical improvements are discussed. Primarily, leveraging model compression theory, knowledge distillation [52] can be employed to transfer the discriminative capabilities of the BERT-Large model to smaller architectures with fewer parameters. Specifically, it operates by minimizing the relative entropy between the output distributions of the teacher model (Large model) and the student model (e.g., DistilBERT or ALBERT), while introducing consistency constraints in the hidden feature space. Theoretically, this method can effectively extract and retain the most critical feature representations for specific tasks, thereby improving the model's parameter optimization, training efficiency, and real-time processing capabilities in resource-constrained scenarios. Secondarily, based on surrogate model theory, more efficient alternatives to Gaussian Processes, such as Random Forests, can be employed as surrogate models to provide faster approximations while reducing computational overhead and memory consumption during optimization. Finally, with the proliferation of distributed computing and cloud platforms, hardware acceleration represents another crucial direction for improving the scalability of large-scale models. Industrial applications often require multi-objective optimization (e.g., balancing performance, stability, and training time). Different hyperparameter combinations can be evaluated simultaneously across multiple compute nodes through parallelization and distributed Bayesian optimization, significantly reducing optimization time costs. In summary, this empirical exploration highlights the nuanced variations of performance, robustness, and generalization among different historical BERT checkpoints in evolving requirements elicitation from app reviews. The findings further validate the effectiveness of the proposed lexicon-enhanced hybrid framework based on the PT paradigm.