# Explainable Security Requirements Classification Through Transformer Models
## 4 Results
### Few-Shot Learning Results  
Given the size of the dataset, we used a technique called few-shot learning, described in Section 3.3.2, that enables models to learn how to learn and adapt quickly to new tasks based on a small number of labeled examples. As a result, the models become more flexible and adaptable. Based on the results described in Section 4.1, we selected the model with the best performance considering the average metrics. Specifically, considering the Table 5, for the normal dataset, we selected the RoBERTa-base model; regarding the dataset augmented using contextual word embeddings and the other augmented using the back translation technique, we used BERT-base-uncased. Lastly, regarding the dataset augmented using the paraphrasing technique, we used DistilBERT-base-uncased. After this phase, we trained the models using transfer learning-based methods that adapt the selected pre-trained models to learn from a few data to classify the requirements. We trained each model for the specified dataset for two epochs, reaching the results shown in Table 6. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline \multicolumn{5}{c}{**Normal dataset**} \\ \hline RoBERTa-base & 0.92 & 0.94 & 0.81 & 0.97 \\ \hline \multicolumn{5}{c}{**Contextual word augmented dataset**} \\ \hline BERT-based-uncased & 0.94 & 0.9 & 0.9 & 0.9 \\ \hline \multicolumn{5}{c}{**Back translation augmented dataset**} \\ \hline BERT-based-uncased & 0.9 & 0.86 & 0.86 & 0.86 \\ \hline \multicolumn{5}{c}{**Paraphrasing augmented dataset**} \\ \hline DistilBERT-base-uncased & 0.9 & 0.89 & 0.81 & 0.85 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of the few-shot learning process regarding accuracy, precision, recall, and F1-score on the four datasets. Figure 6: Confusion matrix for the models evaluated on the test set, providing insights into the model’s classification accuracy. (**a**) Confusion matrix obtained for BERT-base-uncased on the standard dataset. (**b**) Confusion matrix obtained for BERT-base-uncased on the contextual word augmentation dataset. Based on its strong performance in the previous testing, RoBERTa was chosen for the normal dataset. Its performance in the few-shot scenario, where it attained an exceptional F1-score of 0.97, accuracy of 0.92, and precision of 0.94, confirms this selection. In particular, the strong F1-score highlights RoBERTa's ability to handle unaugmented data, indicating that its contextual handling fits in well with the original dataset structure and enables it to perform very well in correctly categorizing security requirements. BERT was selected for the contextual word embeddings augmented dataset because previous studies consistently demonstrated its ability to navigate contextually modified data. BERT leads the few-shot outcomes with an accuracy of 0.94 and consistent scores for precision, recall, and F1-score at 0.90. These findings demonstrate that BERT's design can adjust to subtle rephrasings while preserving semantic integrity, successfully capturing the little but significant lexical changes from contextual embeddings. BERT manages the various rephrasings brought about by back translation with only a minor memory loss, as seen by its accuracy of 0.90 and F1-score of 0.86. This could result from some sensitivity to the more extensive syntactic alterations in the translated text. However, BERT's ability to maintain excellent performance in back-translated augmentations and contextual embeddings indicates that it can adjust to subtle and more noticeable rephrasings. Finally, DistilBERT performed well in few-shot learning, achieving an accuracy of 0.90, precision of 0.89, and F1-score of 0.85, despite the greater lexical diversity of paraphrased lines. DistilBERT may overlook some pertinent examples in this situation, as evidenced by its poorer recall compared to precision, which points to a more selective performance that prioritizes categorization accuracy.