## 4 Experimental Results
### Learning Parameters and Assessment Measures  
Training sets were normalized through Gaussian normalization. The following models were trained with the eight classification problems used in this work: \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Dataset** & **\#Samp.** & **\#Attr.** & **Attr. Types** & **Maj. Class (\%)** & **Ref.** \\ \hline Australian Credit Appr. & 690 & 14 & bool., cat., int., real & 55.5 & [32] \\ Breast Cancer & 683 & 9 & int. & 65.0 & [33] \\ Divorce Prediction & 170 & 54 & bool. & 50.6 & [34] \\ Heart Disease & 270 & 13 & bool, cat., int., real & 55.6 & [31] \\ Ionosphere & 351 & 34 & int., real & 64.1 & [35] \\ Mammographic Mass & 830 & 5 & int., cat. & 51.4 & [36] \\ Student Perf. (Math) & 649 & 32 & bool., cat., int. & 67.1 & [37] \\ Voting Records & 435 & 16 & bool. & 61.4 & [38] \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Datasets used in the experiments. From left to right, the columns designate: number of samples; number of input features; types of features (Boolean, categorical, integer, real); proportion of samples in the majority class; references. * DIMLP ensembles trained by bagging (DIMLP);
* Random Forests (RF);
* Shallow decision trees trained by Gradient Boosting (GB);
* Skope-Rules (SR) [9]. All DIMLP ensembles were trained through back-propagation with the default learning parameters:  
* Learning parameter = 0.1;
* Momentum = 0.6;
* Flat spot elimination = 0.01;
* Number of stairs in the staircase activation function = 50. The DIMLP architectures were defined with two hidden layers. The default number of neurons in the first hidden layer was equal to the number of input neurons. For the second hidden layer, this number was empirically defined in order to obtain a number of connections less than or similar to the number of training samples. For each classification problem, the number of neurons in the second hidden layer was:  
* Australian Credit Appr. : 20;
* Breast Cancer: 20;
* Divorce Prediction: 2;
* Heart Disease: 10;
* Ionosphere: 10;
* Mammographic Mass: 50;
* Student Performance (Math): 6;
* Voting Records: 10. Finally, out-of-bag samples were used to avoid the overtraining phenomenon by applying an early-stopping technique. Specifically, the out-of-bag set constituted a subset of the training dataset that was not used to fit the weight values of the neural networks. Ensembles of DTs were trained with the Scikit Python Library [39]. For GB, the depth of the trees was varied from one to three, while for RF, the depth parameter was unconstrained or fixed to three (RF-3). Furthermore, for all of the random forests, the number of attributes that were taken into account at each induction step was, by default, the square root of the total number of attributes. For comparison purposes, the \"Skope-Rules\" rule extractor [9] was also included in this work. Specifically, its propositional rules were generated from ensembles of RFs. Three important parameters were used:  
* Minimal recall of rules;
* Minimal precision of rules;
* Maximal depth of the trees. The minimal recall parameter influences the number of rules generated. Specifically, recall is defined as \(TP/(TP+FN)\), with \(TP\) designating the number of true positives and \(FN\) denoting the number of false negatives. Precision is defined as \(TP/(TP+FP)\), with \(FP\) representing the number of false positives. In order to approach the performance of the other models used in this work, we carried out several preliminary tests. We noticed that when the maximum tree depth was not limited, the average number of antecedents per rule was too high. We found that a maximal depth of three was a good value. Furthermore, with a low value of the minimal rule accuracy, the predictive accuracy tended to be too weak. After several preliminary experiments, we determined that 95% was an appropriate value for the eight selected datasets. Finally, too high of a value of the minimal recall parameter meant that many samples were not covered by the rules, and too low of a value generated a high number of rules.