## 3 Materials and Methods
### Addressing Imbalanced Data Classification
#### 3.3.1 Rationale for the Choice of Ensemble Classifiers  
According to [PERSON] [48], the key to the good behavior of classifier ensembles is the diversity provided by different training sets. The ideal classifier should be induced from a training set randomly generated from the distribution of the entire space of hypothesis. However, only one training set \(Z=\left\{z_{1}\ldots z_{N}\right\}\) is usually available and does not have an appropriate size to be split into mutually exclusive subsets with a considerable number of instances. In these cases, the bootstrap sampling can be used to generate \(L\) training sets. When training sets are generated from Bootstrap sampling, significant improvements are achieved mainly if the base classifier is unstable, that is, small changes in the training set should lead to large changes in the classifier output. Although instability depends on the learning algorithm, it is also influenced by the dataset characteristics, such as data imbalance. On the other hand, majority vote properties usually ensure the improvement of the single classifier results if the outputs are independent and the individual accuracies of the base classifiers have a normal distribution. This fact was evidenced in [48] by means of the rationale described below. Given a set of labels \(\Omega=\left\{\omega_{1},\ldots,\omega_{c}\right\}\), a set of classifiers \(D=\left\{D_{1},\ldots,D_{L}\right\}\) and objects \(x=\left[x_{1},\ldots,x_{n}\right]^{T}\in\mathcal{R}^{n}\) to be classified, each classifier \(D_{i}\) in the ensemble provides a label \(s_{i}\in\Omega,i=1,\ldots,L\). Thus, for any object \(x\in\mathcal{R}^{n}\) to be classified, the \(L\) classifier outputs define a vector \(s=\left[s_{1},\ldots,s_{L}\right]^{T}\in\Omega^{L}\). The labels given by the classifiers can be represented as binary vectors \(\left[d_{i,1},\ldots,d_{i,c}\right]^{T}\in\left\{0,1\right\}^{c},i=1,\ldots,L\), where \(d_{i,j}=1\) if the classifier \(D_{i}\) assigns to \(\mathbf{x}\) the label \(\omega_{j}\), and \(0\) otherwise. The final choice of the class is obtained by majority vote. This means that the following rule must be complied with:  
\[\sum_{i=1}^{L}d_{i,k}=max_{j=1}^{c}\sum_{i=1}^{n}d_{i,j} \tag{7}\]  
We must take into account that the number of classifiers, \(L\), must be odd for binary classifications. Let\({}^{\prime}\)s consider that the probability of predicting the right class for any \(x\in\mathcal{R}^{n}\) for each classifier is \(p\). If the classifier outputs are independent, then the joint probability for any subset of classifiers \(A\subseteq D,A=\left\{D_{i1},\ldots,D_{ik}\right\}\) can be decomposed as \(P(D_{i1}=s_{i1}\ldots,D_{ik}=s_{ik})=P(D_{i1}=s_{i1})\times\ldots\times P(D_{ ik}=s_{ik})\), where \(s_{ij}\) is de label given by the classifier \(D_{ij}\). When using majority vote, the ensemble output will be correct if at least [L/2] + 1 classifiers give the correct label. When the individual probabilities \(p\) are the same, the accuracy of the ensemble is:  
\[P=\sum_{m=\left[\frac{1}{2}\right]+1}^{L}\left( \begin{array}{c}L\\ m\end{array} \right)^{m}(1-p)^{L-m} \tag{8}\]  
For unequal probabilities, when the distribution of the individual probabilities \(p_{i}\) is normal, the value of \(p\) can be substituted for the mean value of \(p_{i}\) in the equation. According to the Condorcet theorem, if the individual accuracies \(p>0.5\) then \(P\to 1\)_as_\(L\rightarrow\infty\). Therefore, in many problems, the use of majority vote-based ensembles, such as bagging and random forest, is expected to lead to an improvement in individual accuracy even if the base classifiers are not completely independent. In addition to accuracy, there are other aspects that need to be studied when using ensemble classifiers, one of which is overfitting. This problem, associated with data oversampling, causes loss of generalization capacity in classifiers when they are very adapted to the training set and show poor predictive performance for classifying other examples.