## 3 Results
Comparing the effect sizes of RC on efficiency (\(-0.0196\)) and efficacy (\(0.0580\)), each decrease in the relative test efficiency (caused by the increase of the RC) results in an average rise of relative test efficacy that is \(3\times\) higher than the efficiency decrease. Thus, each RC unit increment returns an improvement in relative test efficacy \(3\times\) higher (on average) than the decline observed in relative test efficacy on average, corroborating the benchmark study's finding. Another benchmark comparison shown in Table 3 was related to using the random benchmark. The test efficiency and efficacy reached by each ML model induced with distinct RC values were compared against the baseline value of non-informed software testing based on random module selection. Chart (c) in Figure 5 shows the plot of the relative efficiency, efficacy, and RPC, with the random benchmark as the baseline. The results also corroborate the benchmark study's findings, where the relative efficacy drops faster than the relative efficiency as RC is increased. Still, those values are always higher than \(100\%\), demonstrating that the ML-based approach outperforms the non-informed selection of modules for testing. However, since an increase in RC implicates an increase in test scope, it is natural to expect that as the scope increases, it weakens the ML-based approach advantages since, ultimately, when \(100\%\) of the modules are tested, an ML-based approach offers no additional value when compared to the random selection of modules to be tested. Finally, charts (b) and (d) of Figure 5 show the same analyses for a more extensive RC range encompassing higher values. Figure 5: [REffR, RefR, RPC] \(\times\) RC. The last analysis was to understand how RC increases affect MDM, MNDM, and UT. Table 4 shows the values of each of those metrics for each RC value. All those metrics were statistically significant for RC \(>\) 1 compared to their baseline values (RC = 1) at a 5% level. The experiments were conducted using a 10-fold cross-validation approach. Thus, each fold contained, on average, 320 defective modules (used for MDM calculation) and 3983 non-defective modules (used for MNDM calculation) with \(k=10\). Figure 6 shows those metrics plotted for each RC value. The upper chart of Figure 6 is focused on a narrow range of RC, while the bottom one shows a chart encompassing the full range of RC. The behavior of MDM, MNDM, and UT versus RC was compared using linear models obtained with linear regressions. The regressions expressions are shown in Equations (25)-(27),  
\[UT=(0.0119\cdot RC+0.4509)\times 100\%;R^{2}=0.8284, \tag{25}\]  
\[Eff=(0.0151\cdot RC+0.3348)\times 100\%;\ R^{2}=0.8394, \tag{21}\]0  
\[Eff=(0.0151\cdot RC+0.3348)\times 100\%;\ R^{2}=0.8394, \tag{21}\]1  
Those models reached values \(R^{2}\) greater than 80%, indicating they could adequately explain the variance of those metrics using the independent variable RC. The linear model expressions are expressed in the chart. The MDM, MNDM, and UT effect sizes in the models were \(-0.0151\), \(0.0033\), and \(0.0119\), respectively. Those coefficients indicate that each increase in RC causes a reduction in MDM that is 26.9% higher than the increase it causes in UT. Moreover, they suggest that each rise in RC causes a decrease of 457.6% in MDM, which is higher than the increase it causes in MNDM. MDMs are dangerous, especially in safety-critical systems, because they divert the software testing effort to evaluate properly those misclassified defective classes, increasing the risks of failure during customer use. Thus, they are highly undesired. Using the proposed approach, the MDM is reduced in a ratio much higher than it increases the MNDM. Although MNDMs are undesired, their negative outcome is to induce the software testing effort in testing a non-defective software module, which wastes resources. Still, they do not cause dangerous situations that could cause more severe losses, such as jeopardizing life or property.