## 4 Experimental Results
### Related Work  
We compare our results with those of other state-of-the-art approaches. For the \"Australian\" classification problem, Table 12 illustrates the results provided by several rule extraction techniques applied to ensembles of DTs (ET-FBT, RF-FBT, AFBT, and InTrees). Moreover, G-REX is a rule extraction method that is applied to ensembles of MLPs. Finally, the last row indicates our results with respect to Gradient-Boosted Trees (GB-DIMLP). The second column describes the evaluation method. Specifically, \"10 \(\times\) RS\" signifies that the random sampling of the training and testing sets was applied ten times, while \"\(1\times 10\)-fold-CV\" indicates one repetition of ten-fold cross-validation. Here, the highest average predictive accuracy was given by our rule extraction method. However, a strict comparison of results is difficult because not all evaluation procedures are completely the same. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Dataset** & **Model** & **Acc. R.** & **Acc. R. (SR)** & \(p\)**-Value** \\ \hline Australian Credit Appr. & GB (1,100) & **86.6** (0.5) & 85.5 (0.3) & \(2.8\times 10^{-5}\) \\ Breast Cancer & RF (150) & **96.9** (0.2) & 94.4 (0.2) & \(2.2\times 10^{-16}\) \\ Divorce Prediction & GB (3,50) & 97.3 (1.0) & 96.5 (0.7) & \(5.5\times 10^{-2}\) \\ Heart Disease & DIMLP (50) & **84.4** (1.1) & 77.1 (1.4) & \(2.9\times 10^{-10}\) \\ Ionosphere & DIMLP (50) & **92.3** (1.1) & 88.4 (0.8) & \(8.4\times 10^{-8}\) \\ Mammographic Mass & DIMLP (25) & **84.6** (0.2) & 83.1 (0.5) & \(1.6\times 10^{-6}\) \\ Student Perf. (Math) & GB (1,25) & **92.0** (0.2) & 91.0 (0.4) & \(7.6\times 10^{-6}\) \\ Voting Records & GB (3,50) & **96.2** (0.4) & 94.9 (0.5) & \(6.0\times 10^{-6}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Summary of the best average accuracies of rulesets produced by ensembles of DIMLPs or DTs. A comparison with SR (fourth column) was achieved through a Welch \(t\)-test; \(p\)-values are illustrated in the last column. A bold number represents a significantly better average predictive accuracy. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **Acc.** & **Fid.** & **Acc. R. (a)** & **Acc. R. (b)** & **Nb. R.** & **Nb. Ant.** \\ \hline GB (2,50) & 95.8 (0.3) & 99.4 (0.4) & 95.6 (0.5) & 96.0 (0.3) & 9.0 (0.3) & 2.4 (0.1) \\ GB (2,100) & 96.4 (0.4) & 98.7 (0.3) & 96.1 (0.5) & 96.8 (0.4) & 14.1 (0.3) & 3.0 (0.0) \\ GB (2,150) & **96.6** (0.5) & 98.9 (0.4) & **96.2** (0.4) & 96.9 (0.4) & 16.0 (0.5) & 3.1 (0.0) \\ \hline GB (3,25) & 95.7 (0.3) & 99.3 (0.4) & 95.4 (0.4) & 95.9 (0.4) & 10.2 (0.4) & 2.5 (0.0) \\ GB (3,50) & 96.4 (0.4) & 98.5 (0.4) & **96.2** (0.4) & **97.0** (0.4) & 13.2 (0.5) & 2.9 (0.0) \\ GB (3,100) & 96.3 (0.6) & 98.7 (0.6) & 95.8 (0.6) & 96.6 (0.7) & 19.2 (0.6) & 3.2 (0.0) \\ GB (3,150) & 95.9 (0.6) & 98.4 (0.5) & 95.3 (0.6) & 96.4 (0.6) & 22.7 (0.8) & 3.4 (0.0) \\ \hline SR (25) & — & — & 94.9 (0.5) & — & 17.5 (0.9) & 3.0 (0.0) \\ SR (50) & — & — & 94.7 (0.5) & — & 23.9 (1.6) & 3.0 (0.0) \\ SR (100) & — & — & 94.5 (0.4) & — & 31.7 (0.9) & 3.0 (0.0) \\ SR (150) & — & — & 94.3 (0.6) & — & 37.7 (1.6) & 3.0 (0.0) \\ \hline \hline \end{tabular}
\end{table} 
Table 9: \(Cont.\)Table 13 depicts the results for the \"Breast Cancer\" classification problem. The first five rows show a number of rule extraction techniques applied to an ensemble of DTs (NH, RuleFit, RF-DHC, RF-SGL, and RF-mSGL). The last row indicates our results obtained with ensembles of RFs transformed into ensembles of DIMLPs. The highest average predictive accuracies were provided by RuleFit and RF-DIMLP (97% and 96.9%, respectively), with fewer rules generated by RF-DIMLP, on average (24.3 versus 38). It is also important to keep in mind that each rule generated by RuleFit, RF-SGL, or RF-mSGL has a coefficient that makes rule interpretability more difficult, which is not the case for RF-DIMLP. We carried out a Welch \(t\)-test on the average predictive accuracies provided by RF-DIMLP and RuleFit.