## 2 Materials and Methods
### Evaluation Metrics  
Various metrics were collected or computed to evaluate the ML model's performance. They were all average values computed from the 100 samples measured from the RFs induced for each cost value assigned. The fundamental metrics collected were those from the confusion matrix. The **true positive (TP)** is the number of defective software modules correctly classified as defective by the ML model. Thus, they correctly inform the software testing team about the modules that must be considered in the software testing scope because they are defective, using the available resources appropriately. The **true negative (TN)** is the number of non-defective software modules correctly classified as non-defective by the ML model. Thus, they correctly inform the software testing team about the modules that could be left outside the software testing scope since they are not defective, saving the available resources appropriately. The **false positive (FP)** is the number of non-defective software modules incorrectly classified as defective by the ML model. Thus, they wrongly induce the software testing team to consider them inside the testing scope, although they are not defective, wasting resources, which reduces their efficiency. The **false negative (FN)** is the number of defective software modules incorrectly classified as non-defective by the ML model. Thus, they wrongly induce the software testing team to leave those defective modules outside the testing scope, reducing their efficacy. Therefore, FNs are dangerous and must be avoided since those defective modules can cause severe consequences when the software operates in production. When managers design the software testing scope informed by the ML classifier, they include all the modules classified as defective (TP + FP). Thus, the metric **number of modules tested (MT)** is defined by Equation (1) [40],  
\[MT=TP+FP. \tag{1}\]  
Therefore, using a decision-making process informed by the ML classifier, managers will exclude from the software testing scope the modules indicated as non-defective (TN + FN). Thus, Equation (2) defines the metric **number of modules not tested (MNT)**,  
\[MNT=TN+FN. \tag{2}\]  
When the ML model supports decision-making, the result is a reduction in software testing scope, according to [40]. Equation (3) defines the metric **scope reduction (SR)**[40],  
\[SR=\frac{MNT}{MT+MNT} \tag{3}\]  
On the other hand, the fraction of the total number of modules suggested by the ML classifier as the proper software testing scope is the **relative test scope (RTS)**[40], defined in Equation (4),\[RTS=1-SR. \tag{4}\]  
Cost-sensitive training is influenced by the relationship between the costs assigned to FN (\(C_{FN}\)) and FP (\(C_{FP}\)). Therefore, the approach core strategy is increasing the **relative cost (RC)**[40], defined by Equation (5), between the cost assigned to FN and FP and evaluating the average performance of the ML models. \[RC=\ \frac{C_{FN}}{C_{FP}}, \tag{5}\]  
where, in the present research, \(C_{FP}=1\), thus,  
\[RC=C_{FN}. \tag{6}\]  
Since \(C_{FN}\subset\{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,30,40,50,100\}\), RC \(\subset\{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,20,30,40,50,100\}\). As defined by [40], **Precision (P)**[80], in this research domain, translates the **efficiency (Eff)** of the test effort, because it represents the total number of defective modules detected from the total number of modules tested. Ideally, software testing effort should be spent only on defective modules indicated by a 100% efficient ML model. Equation (7) indicates the expression used to compute the Model's Eff [40],  
\[Eff=P=\frac{TP}{(TP+FP)}. \tag{7}\]  
Analogously, according to [40], **Recall (R)**[80] can be referred to as **efficacy (Ef)**, since it indicates how effective the test effort can be following, considering exactly the software testing scope suggested by the ML model.