# Confidence-Based Voting for the Design of Interpretable Ensembles with Fuzzy Systems +
## 2 Materials and Methods
### Classifying with Fuzzy Systems
To assign the class number to the rule, the class label having the largest confidence values is chosen. The rule weight was estimated as follows:  
\[CF_{q}=2*Conf(A_{q}\rightarrow\text{ Class }k)-1, \tag{3}\]  
so that the rules which have the confidence equal to 1 receive the weight 1, and the confidence of 0.5 is mapped to zero weight. The rules having confidence lower than 0.5 are considered as empty, and if the rule base has more than 50% empty rules, it is generated again. In this study the fitness assignment scheme was modified compared to the version of HEFCA presented in [23]. Here the loss value was added in fitness estimation, so that the fitness value is calculated as follows:  
\[Fitness_{i}=\frac{5000*Error_{i}}{N}+\frac{5000*Loss_{i}}{N}+NR_{i}+Len_{i}, \tag{4}\]  
where \(Error_{i}\) is the total amount of misclassified points, \(NR_{i}\) is the number of non-empty fuzzy rules. Here \(Len_{i}\) is the total number of non-empty predicates in all rules, \(i=1,\dots,NP\), where \(NP\) is the population size. Besides, \(N\) is the sample size, and \(Loss_{i}\) is calculated as:  
\[Loss_{i}=\sum_{i=0}^{N}1-\mu_{A_{w}}(x_{i})CF_{w}, \tag{5}\]  
where \(w\) is the winner rule index. The winner rule is determined as the one which has the largest weight value. The loss value added into the fitness function makes the algorithm sensible not only to the number of misclassified instances, but also to the confidence of the classifier about its final decisions. The Michigan part contain the following steps:  
* Define each rule in a rule base as an individual and calculate its fitness;
* Remove or add new rules to the rule base with genetic or heuristic approach;
* Return the modified rule base to the population. The Michigan-style part is applied to every rule base after the mutation operator. Firstly, the fitness values are calculated for every rule, and the rule fitness is equal to the number of correctly classified instances by this particular rule. If there are two identical rules, only one of them gets the non-zero fitness. There were three types of the Michigan part: adding new rules, deleting rules and replacing rules, i.e., first deleting, then adding. In the case of deleting rules, the number of rules \(k\) to be deleted is defined as \(5k-5<S<5k\), where \(S\) is the size of the rule base. The rules with the lowestfitness values are deleted first. In the case of adding new rules, the number of rules to be added is defined in the same way as for deleting, but if the number of rules exceeds the maximum number of rules then no rule is added. New rules are added with the use of two different methods, heuristic and genetic ones. The heuristic method uses incorrectly classified instances to generate new rules using the same procedure as at the initialization step. The genetic method uses rules from the rule base to produce new rules with the tournament selection, uniform crossover and average mutation as in the genetic algorithm. In addition to this the HEFCA, algorithm applied the instance selection mechanism, presented in [23]. The instance selection technique idea with balancing strategy is to generate a subsample from the original sample, where the number of instances from each label are balanced, if it is possible. This prevents the biased learning of the classifier due to existing imbalance of many datasets and also makes the search process faster, as smaller sample size is used. The probability values were assigned to each training instance, and this probability depended on the number of successful classifications of this particular instance during the adaptation period. Instances which were successfully classified many times before, are chosen less often, while complex cases are chosen with larger probabilities. The detailed procedure of instance selection is given in the HEFCA algorithm description in [23].