# Confidence-Based Voting for the Design of Interpretable Ensembles with Fuzzy Systems +
## 3 Results
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset** & \multicolumn{2}{c}{**HEFCA**} & \multicolumn{2}{c}{**NN (Keras)**} & \multicolumn{2}{c}{**CBV**} \\ \hline  & Class 1 & Class 2 & Class 1 & Class 2 & Class 1 & Class 2 \\ Australian credit & 0.567 & 0.686 & 0.585 & 0.680 & 0.571 & 0.684 \\ German credit & 0.793 & 0.304 & 0.811 & 0.247 & 0.800 & 0.272 \\ Magic & 0.773 & 0.430 & 0.776 & 0.404 & 0.777 & 0.409 \\ Phoneme & 0.802 & 0.394 & 0.808 & 0.334 & 0.805 & 0.379 \\ Ring & 0.645 & 0.653 & 0.614 & 0.631 & 0.636 & 0.649 \\ Twonorm & 0.646 & 0.642 & 0.663 & 0.662 & 0.652 & 0.649 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: F-measures of HEFCA, neural network and CBV. For a better understanding of the CBV performance when combining different classifiers, the following series of experiments have been performed with Decision Tree classifier. The results are presented in Table 4.  
Again, in the same way as for neural network, if the Decision Tree is better than HEFCA, then the CBV always improves the accuracy, in this case for seven datasets out of 11. Table 5 contains the results of combining HEFCA with Random Forest classifier. The used Random Forest classifier has shown high efficiency for most of the used datasets, so that the CBV was able to improve the accuracy for 10 datasets out of 11. The only failure was for the German credit dataset, where the difference between HEFCA and RF was small, and the decrease of overall accuracy was observed, probably due to the difficulty of this classification problem. As was mentioned above, the CBV procedure was tuned so that 75% of the test set was classified by the HEFCA algorithm. For this purpose the membership values were estimated for the whole dataset, and then 25% of instances with smallest confidence were classified by other methods. Figure 3 shows the distribution of membership values returned by fuzzy classifier for the Texture problem, on one of the folds. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **HEFCA** & **Decision Tree** & **CBV** & **MW Test** & **Z Value** \\ \hline Australian credit & 0.8537 & 0.8158 & **0.8493** & 0 & \(-\)0.30 \\ German credit & 0.728 & 0.697 & 0.723 & 0 & \(-\)0.15 \\ Magic & 0.8415 & 0.8162 & 0.8359 & 0 & \(-\)1.43 \\ Page-blocks & 0.9406 & 0.9627 & **0.9475** & 0 & 1.58 \\ Penbased & 0.9395 & 0.9616 & **0.9646** & 1 & 3.77 \\ Phoneme & 0.826 & 0.8701 & **0.8481** & 1 & 2.79 \\ Ring & 0.9237 & 0.8785 & 0.9101 & \(-\)1 & \(-\)2.78 \\ Satimage & 0.8636 & 0.8609 & **0.8712** & 0 & 1.21 \\ Segment & 0.91 & 0.9654 & **0.9537** & 1 & 3.92 \\ Texture & 0.9122 & 0.9298 & **0.9235** & 0 & 1.43 \\ Twonorm & 0.9043 & 0.847 & 0.8907 & \(-\)1 & \(-\)3.02 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Accuracy of HEFCA, Decision Tree and CBV. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **HEFCA** & **Random Forest** & **CBV** & **MW Test** & **Z Value** \\ \hline Australian credit & 0.8537 & 0.8594 & **0.8551** & 0 & \(-\)0.30 \\ German credit & 0.728 & 0.731 & 0.72 & 0 & 0.23 \\ Magic & 0.8415 & 0.8662 & **0.8533** & 1 & 3.02 \\ Page-blocks & 0.9406 & 0.9704 & **0.9507** & 1 & 2.04 \\ Penbased & 0.9395 & 0.9871 & **0.9783** & 1 & 3.77 \\ Phoneme & 0.826 & 0.9034 & **0.8593** & 1 & 3.40 \\ Ring & 0.9237 & 0.9359 & **0.9299** & 0 & 1.43 \\ Satimage & 0.8636 & 0.9038 & **0.8921** & 1 & 3.77 \\ Segment & 0.91 & 0.9797 & **0.961** & 1 & 3.79 \\ Texture & 0.9122 & 0.9696 & **0.9349** & 1 & 3.10 \\ Twonorm & 0.9043 & 0.9477 & **0.9188** & 1 & 2.42 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Accuracy of HEFCA, Random Forest and CBV. The texture dataset was sorted by class numbers. From Figure 3 it could be seen that the HEFCA classifier is relatively confident about most classes, but has low confidence in other classes. For example, the first two classes, around 100 instances, have membership values mostly larger then 0.9, while for the third class the membership values drop down to 0.7 or even lower.