## 4 Experimental Results
### Cross-Validation Experiments  
We conducted experiments based on ten repetitions of stratified 10-fold cross-validation trials. Table 2 illustrates the results for the \"Australian\" dataset; between brackets are shown the standard deviations. For the DIMLP ensembles, we observed that the numbers in each column were quite stable, although the number of predictors in an ensemble increased from 25 to 150. This was also true for RFs to a lesser extent. For GB and SR, the average complexity of the rules increased with an increasing number of predictors (last two columns). Note also that the highest average fidelity was obtained with the simplest ensembles; e.g., decision stumps trained by gradient boosting with 25 and 50 predictors. RF ensembles with 50 predictors provided the highest average predictive accuracy (87.2%), but the generated rulesets were among the most complex (76.8 rules on average). Interestingly, the average predictive accuracy obtained by the decision stumps was very close, with an average of 87.1% (see GB (1, 100)), but with less complex rulesets (13.2, on average). As a general observation, the average predictive accuracy of the ensembles tended to be a bit higher than that obtained by the extracted rules. Nevertheless, the average predictive accuracy of the extracted rules, when rules and ensembles agreed (fifth column) tended to be a bit higher than that obtained by the models (first column). \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Acc.** & **Fid.** & **Acc. R. (a)** & **Acc. R. (b)** & **Nb. R.** & **Nb. Ant.** \\ \hline DIMLP (25) & 86.8 (0.6) & 98.1 (0.4) & 86.5 (0.7) & 87.4 (0.6) & 22.2 (0.4) & 3.6 (0.1) \\ DIMLP (50) & 86.9 (0.4) & 98.2 (0.4) & 86.5 (0.6) & 87.4 (0.4) & 22.8 (0.4) & 3.7 (0.0) \\ DIMLP (100) & 86.8 (0.3) & 97.9 (0.5) & 86.4 (0.3) & 87.4 (0.2) & 22.5 (0.6) & 3.7 (0.1) \\ DIMLP (150) & 86.9 (0.3) & 98.0 (0.3) & 86.5 (0.4) & 87.4 (0.4) & 22.4 (1.0) & 3.7 (0.1) \\ \hline RF (25) & 86.9 (0.4) & 95.1 (0.6) & 85.1 (0.6) & 87.9 (0.4) & 76.6 (0.9) & 4.6 (0.1) \\ RF (50) & **87.2** (0.6) & 95.5 (1.0) & 85.9 (0.5) & **88.3** (0.4) & 76.8 (0.8) & 4.6 (0.0) \\ RF (100) & 87.1 (0.4) & 96.0 (0.4) & 85.7 (0.7) & 87.9 (0.4) & 77.1 (0.9) & 4.7 (0.0) \\ RF (150) & 87.1 (0.4) & 95.3 (0.5) & 85.8 (0.9) & **88.3** (0.5) & 76.9 (0.9) & 4.6 (0.0) \\ \hline RF-3 (25) & 85.9 (0.8) & 98.7 (0.3) & 85.8 (0.7) & 86.3 (0.7) & 17.2 (1.1) & 3.4 (0.1) \\ RF-3 (50) & 86.1 (0.4) & 98.5 (0.5) & 86.1 (0.5) & 86.7 (0.4) & 18.1 (1.1) & 3.5 (0.1) \\ RF-3 (100) & 86.4 (0.6) & 98.7 (0.4) & 86.1 (0.6) & 86.7 (0.6) & 17.6 (0.7) & 3.6 (0.1) \\ RF-3 (150) & 86.1 (0.4) & 98.4 (0.4) & 86.0 (0.3) & 86.6 (0.3) & 17.7 (0.6) & 3.6 (0.1) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average results obtained on the “Australian” dataset. From left to right are presented the average results on predictive accuracy, fidelity on the testing sets, predictive accuracy of the rules, predictive accuracy of the rules when ensembles and rules agreed, number of rules, and number of antecedents per rule. Standard deviations are given between brackets. For DIMLP, RF, RF-3, and RS, the number of predictors is given between brackets (first column). For GB, the first number in brackets is the depth of the trees and the second is the number of predictors. For each column, the highest average accuracy or average fidelity is represented in bold, along with the lowest average number of rules or average number of antecedents. Table 3 presents the results for the \"Breast Cancer\" classification problem. As in the previous table, the results provided by DIMLPs and RFs were quite stable with the increasing number of predictors in an ensemble. For GB and SR, the complexity of the extracted rulesets tended to be augmented as the number of predictors increased, but to a lesser extent with the decision stumps (trees with only one node). Again, the highest average predictive accuracy of the model and the extracted rules was provided by RF (97.4% and 96.9%).