# Confidence-Based Voting for the Design of Interpretable Ensembles with Fuzzy Systems +
## 2 Materials and Methods  
### Classifying with Fuzzy Systems  
The fuzzy logic systems originated from the fuzzy set theory, and today are applied in many areas of engineering, including data mining and machine learning. The most popular way of automated fuzzy systems design is to use evolutionary fuzzy systems (EFS) [15]. Among EFS, the fuzzy systems which are composed of a set of rules are called fuzzy rule-based systems (FRBS). The most efficient group of techniques used for FRBS design is the evolutionary algorithms, or more precisely an implementation of specific genetic algorithm (GA) [16] with modified crossover and mutation operators. These algorithms are referred to as genetic fuzzy systems (GFS) [17]. The genetic algorithms are applied because of their ability to perform search in multidimensional spaces with high efficiency. Most of the GFS algorithms try to incorporate knowledge extracted from the dataset or some kind of a priori information about the search space or problem structure. In general, the biology-inspired and evolutionary algorithms find many applications for automated design of classification and regression problems [18; 19; 20; 21]. One of the main reasons of fuzzy systems popularity is their flexibility. Although it allows FRBS to be interpretable, there are two main directions of research: accurate fuzzy systems and interpretable fuzzy systems. The first trend is focused on generating large and complex fuzzy systems, which are usually difficult to interpret, but their accuracy is compared to the accuracy of neural networks. The second trend aims at finding relatively simple fuzzy rules, which are understandable by the experts in the subject area, but the accuracy of such systems is smaller. Their main advantage is that they are capable of extracting non-trivial knowledge available for further use. In different areas, the first or the second type of fuzzy systems are usually applied, but obviously a tradeoff between accuracy and interpretability is the main goal of most research projects. This study uses the Hybrid Evolutionary Fuzzy Classification Algorithm (HEFCA) presented in [22]. The HEFCA algorithm have shown to be efficient for classification problems from different areas. It generates fuzzy rules, presented as follows:  
\[R_{q}:\text{IF }x_{1}\text{ is }A_{q1}\text{ and }...\text{ and }x_{n}\text{ is }A_{qn}\text{ then Class }C_{q}\text{ with }CF_{q} \tag{1}\]  
where \(R_{q}\) is the \(q\)-th rule, \(x=(x_{p1},\dots,x_{pn})\) is the set of \(m\) training instances in \(n\)-dimensional space, \(A_{qi}\) is the fuzzy set for the \(i\)-th input variable, \(C_{q}\) is the class label, and \(CF_{q}\) is the weight of the rule. The training set is used to normalize the sample, so that all input variables are in range [0, 1], the test set is normalized according to normalization parameters used for the training set. To calculate the resulting membership value of the rule, the product operation is used. Fuzzy classifiers usually rely on a single granulation of the fuzzy variables, however this approach is not always efficient. One of the ways of improving the accuracy, proposed in [22], is to use several fuzzy granulation levels, for example into 2-5 fixed triangular terms. This allows keeping the interpretability while improving the accuracy. The used granulation is presented in Figure 1. The algorithm also used the \"Don't Care\" condition to simplify the rules. The algorithm initialized the set of rule bases using heuristic generation procedure, which uses instances from the training sample. The main loop proceeds as described in Algorithm 1:  
Figure 1: Fuzzy granulation into 2, 3, 4 and 5 terms. The quality of each generated rule was estimated using the confidence value:  
\[Conf(A_{q}\rightarrow\text{ Class }k)=\frac{\sum_{x_{p}\in\text{ Class }k}\mu_{A_{q}}(x_{p})}{\sum_{p=1}^{m}\mu_{A_{q}}(x_{p})}, \tag{2}\]  
where \(A_{q}\) is the \(q\)-th rule left part, \(k\) is the class number, \(\mu_{A_{q}}(x_{p})\) is the membership value for the value from the sample \(x_{p}\).