## 4 Experimental Results
### Related Work
The number of samples for the former model was equal to 100, while for the latter, it was ten, corresponding to only one repetition of 10-fold cross-validation. The difference in their average predictive accuracies was not big enough to be statistically significant (\(p\)-value = 0.88). Then, for the same models, we conducted a Welch \(t\)-test with respect to the average number of rules. The difference between the values provided by RF-DIMLP and RuleFit was big enough to be statistically significant (\(p\)-value = \(1.0\times 10^{-5}\)). Statistical comparisons between random sampling evaluations and cross-validations were not performed, since they were not fully comparable. The results on the \"Ionosphere\" classification problem are compared in Table 14. Rulefit gave the highest average predictive accuracy (93%), with the ensembles of DIMLP networks providing slightly lower values (92.3%), but with the lowest number of extracted rules (18.3 versus 25), on average. Note also that the rule extractors that produced rules without multiplicative coefficients were NH, RF-DHC, and G-Rex. They provided average predictive accuracies equal to 89%, 89%, and 91.4%, which were below that yielded by the DIMLPs. We performed a Welch \(t\)-test for the average predictive accuracies provided by the DIMLP ensembles and RuleFit. The difference in their average predictive accuracies was not big enough to be statistically significant (\(p\)-value = 0.68). Then, for the same models, we carried out a Welch \(t\)-test with respect to the average number of rules. The difference \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Dataset** & **Model** & **Nb. R.** & **Nb. R. (SR)** & \(p\)**-Value** \\ \hline Australian Credit Appr. & GB (1,100) & **13.2** (0.7) & 19.3 (0.8) & \(7.1\times 10^{-13}\) \\ Breast Cancer & RF (150) & **24.3** (0.5) & 31.2 (1.7) & \(1.4\times 10^{-7}\) \\ Divorce Prediction & GB (3,50) & **3.8** (0.0) & 13.9 (1.0) & \(1.4\times 10^{-10}\) \\ Heart Disease & DIMLP (50) & **20.4** (0.5) & 23.2 (0.9) & \(5.6\times 10^{-7}\) \\ Ionosphere & DIMLP (50) & 18.3 (0.6) & **14.1** (0.9) & \(1.9\times 10^{-9}\) \\ Mammographic Mass & DIMLP (25) & **12.0** (0.4) & 14.2 (1.1) & \(8.6\times 10^{-5}\) \\ Student Perf. (Math) & GB (1,25) & **2.0** (0.0) & 13.9 (0.8) & \(4.4\times 10^{-12}\) \\ Voting Records & GB (3,50) & **13.2** (0.5) & 17.5 (0.9) & \(2.5\times 10^{-9}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Summary of the average number of rules extracted from the most accurate rulesets produced by DIMLPs or DTs. A comparison with SR was carried out with a Welch \(t\)-test, with \(p\)-values in the last column. A bold number represents a significantly lower number of rules. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Evaluation** & **Acc. R.** & **Nb. R.** & **Nb. Ant.** \\ \hline ET-FBT [24] & 10 \(\times\) RS & 83.8 (2.1) & — & — \\ RF-FBT [24] & 10 \(\times\) RS & 83.6 (2.5) & — & — \\ AFBT [24] & 10 \(\times\) RS & 83.5 (2.0) & — & — \\ InTrees [25] & 100 \(\times\) RS & 84.3 (—) & — & — \\ \hline G-REX [16] & \(1\times 10\)-fold-CV & 85.9 (—) & — & — \\ GB-DIMLP (1,100) & \(10\times 10\)-fold-CV & **86.6** (0.5) & 13.2 (0.7) & 2.8 (0.1) \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Other results on the “Australian” dataset. A bold number represents a highest average accuracy of the rules. between the values of the DIMLPs and RuleFit was big enough to be statistically significant (\(p\)-value \(=3.9\times 10^{-5}\)). For the \"Mammographic\" dataset, Table 15 depicts some results obtained by other rule extractors from ensembles of DTs. The DIMLP ensembles obtained the highest average accuracy, with that of AFBT being a bit lower (84.6% versus 83.4%). Nevertheless, the evaluation protocol was different. Hence, these results are rather indicative.