## 3 Methodology
It primarily uses the BERT's initial version architecture as a base, replacing heavier architectures with more parameters with a light version of the same architecture with fewer parameters. Hence, it reduces the number of layers by half in the BERT-based model, eliminating identifier embeddings and poolers to yield a significantly faster and smaller version of BERT for widespread use [44]. The model also applies dynamic masking and ignores next-sentence predictions. DistilBERT aims to generate a faster-running version of BERT [42]. #### 3.5.4. RoBERTa

With an almost similar architecture to BERT and built on the same language masking strategy, RoBERTa (Robustly Optimized BERT Pretraining Approach) is an optimized method for pre-training a self-supervised NLP system [45]. The main difference between them is that BERT uses static masking while RoBERTa uses dynamic masking [44]. RoBERTa allows for better performance by changing the basic hyperparameters in the BERT model. ### Evaluation and Statistical Validation Metrics

This study employed frequently used performance evaluation metrics--F-score and AUC--and statistical validation metrics--MCC and Kappa--to evaluate the performance results of models developed with artificial intelligence approaches. #### 3.6.1. Performance Metrics

A set of metrics is required to compare and evaluate the results of algorithms developed for a classification problem. Confusion matrix (CM) is the primary instrument to acquire the essential metrics for the binary classification of software requirements as functional and non-functional. True-positive (TP) and true-negative (TN) depicted in Figure 6 are regarded as correct predictions, whereas false-negative (FN) and false-positive (FP) are considered incorrect predictions [43]. Figure 5: BERT language model architecture. Finding the precision and recall values is necessary to calculate the F-score metric. The precision value defines the number of FR-predicted requirements for the actual FR class presented in Figure 6, and it is calculated with the formula in Equation (1). In contrast, the recall value expresses the number of FR-predicted requirements of all FR requirements in the dataset, and it is calculated with the formula in Equation (2). Using precision and recall values, the F-score [46] is calculated by the formula in Equation (3). \[\text{Presicion}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{1}\]

\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{2}\]

\[\text{F-score}=2*\frac{\text{Presicion}*\text{Recall}}{\text{Presicion}+\text{ Recall}} \tag{3}\]

The maximum appropriate value for an F-score is 1.0, indicative of perfect precision and recall, while the minimum achievable value is 0, when either precision or recall equals zero. Therefore, for a classification problem, the algorithms are more successful as their F-scores approach '1'. Since the definition of the F-score is the harmonic average of precision and recall values, it averts outliers from affecting performance in unbalanced datasets. Additionally, the dataset in the current study was partly unbalanced; as a result, the study identified the F-score as the evaluation metric. AUC, or 'area under the ROC curve', is the measure of the area under the ROC curve where the AUC takes values between 0 and 1. Generally, an AUC of 0.5 suggests no discrimination, while values between 0.7 to 0.8 are considered acceptable, and 0.8 to 0.9 is regarded as excellent for a classification problem. #### 3.6.2 Statistical Validation Metrics

Matthews Correlation Coefficient (MCC) and Kappa statistics-based metrics are ideal for evaluating algorithms developed for prediction problems. All values in the complexity matrix are used in the calculation to assess the performance of a model with the Matthews Correlation Coefficient metric [47]. Considering the correlation between the actual data and the predicted data, Equation (4) is employed to calculate this matrix.