# Explainable Security Requirements Classification Through Transformer Models
## 6 Conclusions and Future Work  
This research presented a method to classify security and non-security requirements. This identification enables development teams to allocate resources and concentrate their efforts efficiently. A clear separation makes it easier to guarantee that these crucial areas are sufficiently addressed, as security needs frequently call for specialist expertise and techniques. Additionally, organizations may better analyze and reduce risks by establishing security needs often tied to possible vulnerabilities. This is especially crucial for adhering to industry norms and laws, which frequently call for specialized security precautions. Finally, security requirements can evolve due to changes in threats or regulations. Organizations may more easily monitor and adjust these criteria throughout the software lifetime and ensure they stay applicable and efficient by keeping a distinct categorization. This categorization improves software systems' dependability and security. In order to carry out this work, we used an original labeled dataset of security and non-security requirements to fine-tune four different transformers that are BERT, DistilBERT, RoBERTa, and XLNET, achieving a maximum accuracy of 0.9, precision of 0.86, recall of 086 and f1-score of 0.86 with the RoBERTa model. Since the dataset classes are imbalanced, we used a data augmentation technique that generates contextual word embeddings to insert, substitute, or remove a word in the original requirement to generate alternatives and fine-tune the presented models. Additionally, we relied on the well-known back translation to generate alternatives to a requirement. Lastly, we introduce another technique to handle class imbalance using an augmentation technique consisting of three fine-tuned LLMs for paraphrasing tasks to generate coherent paraphrases of the minority class of the requirements. Using these techniques, we perform best by fine-tuning the models on the augmented dataset using contextual word embeddings, obtaining an accuracy value of 0.94, precision of 0.9, recall of 0.9, and f1-score of 0.9. In addition, we relied on SetFit, a framework that aims to improve few-shot learning for text classification tasks. Based on the models' performances previously described, we select for each created dataset the best model and use for the few-shot task, obtaining amaximum value of accuracy by using BERT on the augmented dataset with contextual word embeddings, obtaining an accuracy value of 0.94, precision of 0.9, recall of 0.9, and f1-score of 0.9. The results highlight how crucial model selection is for particular dataset attributes. When data are left unchanged, RoBERTa's robust contextual representations are demonstrated by its improved performance on the normal dataset. BERT is perfect for jobs with augmented inputs because of its performance with word augmentation and back translation data, demonstrating its resilience to word and sentence changes. The success of DistilBERT with the paraphrased data indicates that, although smaller models can manage rephrasing with SetFit's help, they could have a little trouble remembering things when there are significant semantic changes. Also, augmentation and SetFit together show promise to improve model robustness in jobs involving requirement classification. While sentence-level augmentations are proper but add complexity, word-level augmentations provide the most reliable generalization gains, especially for BERT. Finally, by random sampling a security requirement from the dataset, we use explainability to gather explanations for the decisions generated by fine-tuned models from the four different datasets. In order to achieve these results, we created heatmaps based on the attention mechanisms used by the models to understand which words or subwords the models pay more attention to in classifying the requirements. The results show that various augmentation methods produce unique attention patterns in each model, which reflects their interpretative approaches and constraints.