# Confidence-Based Voting for the Design of Interpretable Ensembles with Fuzzy Systems +
## 3 Results  
The efficiency of the proposed CBV approach was tested on a set of databases from the KEEL repository [27]. The properties of the datasets are presented in Table 1.  
The used datasets are taken from different areas, have large number of instances or classes, some of them are imbalanced, which make them difficult to solve for many \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Number of Instances** & **Number of Features** & **Number of Classes** \\ \hline Australian credit & 690 & 14 & 2 \\ German credit & 1000 & 24 & 2 \\ Magic & 19/020 & 10 & 2 \\ Page-blocks & 5472 & 10 & 5 \\ Penbased & 10.992 & 16 & 10 \\ Phoneme & 5404 & 5 & 2 \\ Ring & 7400 & 20 & 2 \\ Satimage & 6435 & 36 & 6 \\ Segment & 2310 & 19 & 7 \\ Texture & 5500 & 40 & 11 \\ Twonorm & 7400 & 20 & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Datasets used in the experiments. Figure 2: Flowchart of the Confidence-Based Voting. The HEFCA parameters were set as follows: population size 100, number of generations 5000, maximum rule base size 40, all genetic operators, including Michigan part were self-configured. The instance selection approach used 30% of the original sample to create the training subsample, and the adaptation period was 50 generations long. To estimate the classification quality, the 10-fold cross-validation was used. The accuracy, as well as other measures were used, including F-measure for 2-class problems, and the confusion matrix, averaged over 10 folds. The classification qualities were estimated for all classifiers separately, and then the CBV method was applied. In Table 2 the results of combining HEFCA rule bases with neural networks trained by Keras are presented. In Table 2 the cases where the confidence-based voting led to an improvement in accuracy compared to the HEFCA results are highlighted. For seven datasets out of 11 there was an improvement observed, and for four other cases there was no improvement, because the neural network had worse results. In Tables 2, 4 and 5 in addition to the accuracies the results of the Mann-Whitney rank sum statistical test with tie-braking and normal approximation are provided. The test indicates the performance improvement (1), deterioration (\(-1\)) or equal performance (0). Furthermore, the standard score Z-values are provided for every performed test. The \(p\) level was set to 0.05, i.e., the threshold \(Z\) values were set to \(-1.96\) and \(1.96\) (\(2\sigma\)). Table 3 shows the values of the F-measure for the datasets with two classes. The neural network had better values in most cases, but only for the first class, compared to the fuzzy rule bases generated by HEFCA. This is probably due to the fact that for imbalanced datasets the first class is the majority class. Comparing the F-measures of HEFCA, NN and CBV, almost in all cases there was an improvement for the first class, except the synthetic Ring problem, where the neural network had worse results. For the second class, the F-measures were also better, compared to the results of the neural network, however these results are still worse than those of the HEFCA generated rule base. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **HEFCA** & **NN (Keras)** & **CBV** & **MW Test** & **Z Value** \\ \hline Australian credit & 0.8537 & 0.8695 & **0.8551** & 0 & \(-0.60\) \\ German credit & 0.7280 & 0.7610 & **0.7310** & 0 & 2.66 \\ Magic & 0.8415 & 0.8284 & 0.8362 & \(-1\) & \(-2.49\) \\ Page-blocks & 0.9406 & 0.9589 & **0.9461** & 0 & 0.75 \\ Penbased & 0.9395 & 0.9874 & **0.9790** & 1 & 3.77 \\ Phoneme & 0.8260 & 0.7972 & 0.8225 & 0 & \(-0.75\) \\ Ring & 0.9236 & 0.8259 & 0.9008 & \(-1\) & \(-3.25\) \\ Satimage & 0.8636 & 0.7640 & 0.8244 & \(-1\) & \(-3.77\) \\ Segment & 0.9100 & 0.9498 & **0.9429** & 1 & 3.33 \\ Texture & 0.9122 & 0.9653 & **0.9356** & 1 & 2.64 \\ Twonorm & 0.9043 & 0.9805 & **0.9305** & 1 & 3.56 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Accuracy of HEFCA, neural network and CBV.