## 5 Experiment Results and Analysis
### Empirical Exploration (Answer to RQ5)  
This study conducts a primary empirical study on the performance of diverse BERT scale variants based on PT in ABSA for requirements elicitation. Google has released a total of 15 model checkpoints on Hugging Face ([[https://huggingface.co/google-bert](https://huggingface.co/google-bert)]([https://huggingface.co/google-bert](https://huggingface.co/google-bert)), (accessed on 23 December 2024)). Since AWARE is an English-language dataset, checkpoints trained on German and Chinese data were excluded. Tables 8 and 9 summarize the features of the selected 11 checkpoints. Additionally, we integrate the innovative strategies of the LePB-SA4 RE model into the original uncased versions of the base and large models, which are included in this empirical study. The final results for app review mining-based evolving requirements elicitation are shown in Figures 10 and 11. To ensure the reliability of the experiment, the setup for the empirical experiment is as follows:  
* **Datasets**: AWARE is used as the benchmark dataset, with an 8:1:1 ratio for the training, validation, and test sets. The BFRU dataset is used to evaluate the generalization performance of models. * **Prompt template**: The prompt template is the Formula (5) mentioned in Section 3.2.2. * **Results and metrics**: The final results are averaged over ten parallel tests, with model hyperparameters consistent with the description in Section 4.2. The F1-Score and SD from parallel tests are used as evaluation metrics for performance and robustness. \begin{table}
\begin{tabular}{l l c c c c} \hline \hline \multirow{2}{*}{**Index**} & \multirow{2}{*}{**Model Name**} & \multicolumn{4}{c}{**Features**} \\ \cline{3-6}  & & **Case** & **Language** & **Masking** & **Fine-Tuning** \\  & & **Sensitivity** & **Language** & **Strategy** & **Task** \\ \hline BUC & Bert-base-uncased & uncased & English & Standard & - \\ BC & Bert-base-cased & cased & English & Standard & - \\ BUC-M & Bert-base-multilingual-uncased & uncased & multilingual & Standard & - \\ BC-M & Bert-base-multilingual-cased & cased & multilingual & Standard & - \\ BC-F & Bert-base-cased-finetuned-mrpc & cased & English & Standard & MRPC \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Base model experiments. Figure 9: Confidence interval bands at 95% critical value on BFRU dataset. The two figures demonstrate that large models consistently outperform base models. The proposed lexicon-enhanced dynamic approach (_BUC_LePB_ and _LUC_LePB_) significantly improves the performance and robustness of BERT models. Specifically, compared to the original models with similar characteristics, the F1-Score increased by 1.2% to 2.01%, while the SD decreased by approximately 32% to 56% on the AWARE dataset. On the BFRU dataset, the F1-Score increased by 1.3% to 1.8% and the SD decreased by approximately 29% to 52%. These results indicate that the strategy of LePB-SA4 RE effectively improves the generalization of the model. From Figure 10, it is evident that the uncased models generally achieve higher performance and robustness on the AWARE dataset. This improvement is attributed to the uncased training strategy, which effectively reduces vocabulary complexity and noise inter  
Figure 11: F1-Score and standard deviation of historical BERT checkpoints on BFRU dataset.