## 3 Results  
The protocol described in the previous section was executed entirely, providing a dataset of results with the metrics used to support the analysis presented here. In all tables with results, \"*\" indicates the statistical significance at a 5% level of the paired \(t\)-test with correction (compared to the benchmark, \(RC=1\)). Table 2 shows the models' accuracies for each relative cost. A paired \(t\)-test (with correction) was used to compare the accuracies' averages with the significance test performed at a 5% level. The statistical significance of the \(t\)-test is indicated by \"*\" where the p-value was lower than 5% (compared to the benchmark, \(RC=1\)). Like in the reference study [40], as RC increases, the accuracy decreases. However, here, the reduction was only 2.2% (93.27% to 91.19%), much smaller than that observed in the reference study, where it was reduced to almost half (52%) when the RC was 10 times higher. Notably, an average accuracy of 92.27% cannot be considered a good result for a model trained with an imbalanced binary dataset, with 92.6% of the instances belonging to the most representative class (non-defective module). Finally, all values of ACC (RC > 1) were statistically significant compared to its baseline value (RC = 1) at a 5% level. Table 2 also shows the information from the confusion matrix (TP, TN, FP, FN), indicating how its distribution and RC change. All values of TP, FP, TN, FN, and MT for RC > 1 were statistically significant compared to their baseline values (RC = 1) at a 5% level. As \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**RC** & **ACC** & **TP** & **FP** & **TN** & **FN** & **MT** & **MNT** & **SR** \\ \hline
**1** & 93.3\% & 84.04 & 54.11 & 3928.59 & 235.56 & 138.15 & 4164.15 & 96.8\% \\  
& 74.4\% * \\ \hline \hline \end{tabular} Note: A paired \(t\)-test (with correction) was used to compare the accuracies’ averages at a 5% significance level. “*” indicates \(p\)-values lower than 5% (compared to the benchmark, RC = 1).  
\end{table} 
Table 2: TP, TN, FP, FN, MT, MNT, SR \(\times\) RC. found in the reference study, increasing RC results in increasing TP and decreasing FN, which is positive for using ML models to support test effort allocation. However, while the TP almost doubled from \(RC=1\) to 10, it increased to over seven times in the reference study using a different classifier on a single project and language dataset. Naturally, the increase in TP and the decrease in FN could only happen with an increase in FP (\(\sim\)4\(\times\)) and a slight decrease in TN (4%). Consequently, as shown in Table 2, the number the classifier indicates to be tested (MT) grows as RC increases. That growth (2.7\(\times\)) is lower than observed in the reference study (18.4). Moreover, while SR was reduced from 96% to 29% in the benchmark research, the reduction was much more moderate (96.8% to 91.3%) for the protocol run. Figure 3 illustrates the behavior of TP, TN, FP, FN, MT, and MNT over the distinct costs. Like all charts with results in the present study, the lower chart represents the full range of RC evaluated, and the upper one zooms in on RC [1,15] to better observe the initial behavior in a range compatible with the benchmark study. It is worth noting that the findings reveal a pattern of decreasing marginal returns leading to a saturation of earnings with the technique, demonstrating an asymptotic behavior that persists from the initial points onward. Finally, all values of MT, MNT, and SR (RC \(>\) 1) were statistically significant compared to their baseline values (RC = 1) at a 5% level. Table 3 shows the classifiers' test efficiency and efficacy metrics and a theoretical benchmark obtained with the expected results from a random selection of modules to be tested with the same scope size for each RC. Except for EffR, all other metrics in Table 3 were statistically significant for RC \(>\) 1 compared to their baseline values (RC = 1) at a 5% level.