## 2 Materials and Methods
### Evaluation Metrics
Since the software testing goal is to discover 100% of the defective modules in the system, R measures the fraction of the goal achieved by the test effort informed by the ML model. A software testing scope delineated by a 100% effective ML model would discover all the defective modules. Equation (8) indicates the expression used to compute the Model's Ef [40]. \[Eff=R=\frac{TP}{(TP+FN)}. \tag{8}\]  
Furthermore, the ML model **Accuracy (Acc)**[80; 81] indicates the ratio of software modules correctly classified (TP + TN) from the total number of modules (TP + TN + FP + FN). A 100% accurate ML model would result in no misclassification, that is, nor FP or FN. Although that seems highly desirable, paradoxically, an ML model with 100% accuracy usually has an overfit, indicating compromised generalizability. That is highly undesirable since it reduces its practical application. Equation (9) indicates the expression used to compute the Model's Acc [80; 81]. \[ACC_{RC}\ =\frac{(TP_{RC}+TN_{RC})}{(TP_{RC}+TN_{RC}+FP_{RC}+FN_{RC})}. \tag{9}\]  
As used by [40], a benchmark based on the unitary cost ML Model was used to evaluate how software testing efforts using the scope suggested by the induced ML models with higher RC values performed (Eff and Ef) compared to those with \(RC=1\). Thus, the **relative efficiency to the unitary cost ML model (RefFU)** and the **relative efficacy to the unitary cost ML model (RefFU)** were computed for each RC \(>1\) to support those comparisons using Equations (10) and (11), respectively. \[REFU_{RC}=\frac{Eff_{RC}}{Eff_{RC=1}}, \tag{10}\]\[MNT=TN+FN. \tag{2}\]0  
As suggested by [40], another benchmark was used to evaluate how software testing efforts using the scope suggested by the induced ML models performed (Eff and Ef) compared to similar software testing efforts with identical scope sizes but based on the random selection of modules, representing a decision-making not informed by the ML models. The **efficiency of the random selection (EffR)** was computed using Equation (12), which \(\mathbf{p_{TP}}\) is the **probability of a defective module being selected randomly**, which is 7.4% for the assembled dataset used in this study and is not affected by RC values. The **efficacy of the random selection (EfR)** was computed for each RC to support those comparisons using Equation (13). \[MNT=TN+FN. \tag{2}\]1  
\[MNT=TN+FN. \tag{2}\]2  
Aiming to compare Eff to EffR and Ef to EfR, some additional ratios were computed. The **relative efficiency to the random selection (REFR)** was computed using Equation (14). The **relative efficacy to the random selection (REFR)** was computed for each RC to support those comparisons using Equation (15). \[MNT=TN+FN. \tag{2}\]3  
\[MNT=TN+FN. \tag{2}\]4  
As in the original study [40], other performance comparisons were performed using the metric **Relative Percent Correct (RPC)**, which represents the ratio of the number of modules classified correctly by each benchmark. The **Relative Percent Correct relative to the Unitary cost ML model benchmark (RPCU)** was computed using Equation (16), while the **Relative Percent Correct relative to the Random selection of modules (RPCR)** was computed using Equation (17). \[MNT=TN+FN. \tag{2}\]5  
\[MNT=TN+FN. \tag{2}\]6  
The metric **Misclassified Defective Modules (MDM)** indicates the ratio of the number of defective modules misclassified as non-defective by the ML model to the total number of existing defective modules in the system (3196 in the assembled dataset, and k is the number of folds). The metric **Misclassified Non-defective Modules (MNDM)** indicates the ratio of the number of non-defective modules misclassified as defective by the ML model to the total number of existing non-defective modules in the system (39827 in the assembled dataset, and k is the number of folds). Those metrics were computed for each RC value using Equations (18) and (19), respectively.