# Explainable Security Requirements Classification Through Transformer Models
## 2 Related Work
### LLMs for Text Classification  
In the work by [PERSON] et al. [10], the authors proposed CARP (Clue And Reasoning Prompting) to address these issues by prompting LLMs to find superficial clues and induce a diagnostic reasoning process for final decisions. They describe the standard prompt-based ICL (In-Context Learning) paradigm, transforming text classification into generating pre-defined textual responses based on prompts. They also explore strategies for sampling demonstrations, such as random sampling, kNN sampling using SimCSE, and fine-tuned models. The authors introduce a progressive reasoning strategy involving clue collection, reasoning, and decision-making and discuss the preparation of clues and reasoning for all training examples in advance for few-shot learning. They evaluate CARP's performance on five widely used datasets, showing that CARP outperforms existing methods and achieves new state-of-the-art results, demonstrating the approach's effectiveness in low-resource scenarios and achieving comparable performance with significantly fewer training examples.  
In the work [11], authors have fine-tuned various models (BERT family, GPT, ELECTRA, and XLNet) to classify human-written and AI-generated sentences. The models are tested on two datasets: one with 25,000 sentences and another with 22,929 abstracts. The proposed workflow includes training, validation, and an inference phase where sentences generated by ChatGPT and Wikipedia are classified. An explainability phase using integrated gradients and token importance techniques is also conducted. The fine-tuned models achieved high accuracy, precision, recall, and F1-score for the sentence dataset, indicating they effectively learned patterns to distinguish between human and AI-generated sentences. The research underscores the ethical implications of distinguishing AI-generated content from human-generated content. As AI systems become more sophisticated, it is crucial to promote transparency in AI-generated content and establish clear guidelines for attribution. This is essential to maintain trust in information sources and foster discussions about the ethical use of AI.  
The study by [PERSON] et al. [12] discusses training mLSTM and Transformer language models on a large 40 GB text dataset and their application to text classification problems, precisely binary sentiment, and multidimensional emotion classification. For the emotion classification, they used a dataset of tweets labeled with eight categories plus three additional emotions for training and evaluations. They also created a dataset of relevant tweets to evaluate the models' performance on domain-specific tasks. The tweets are labeled using human raters and employ an active learning technique to balance class distribution. The study demonstrates that unsupervised pretraining combined with fine-tuning provides a practical framework for complex text classification tasks. The Transformer model, in particular, shows strong performance when fine-tuned for specific tasks with noisy labels and specialized context. This approach offers a practical solution for niche text classification problems, making it accessible for academics and small organizations.