## 3 Class Imbalance Problem
### Classifier Ensembles  
Combining classifiers is a developed research area in pattern recognition to develop highly accurate and reliable systems. In most of the pattern recognition applications, a single algorithm cannot uniquely represent the problem in an optimal way. Moreover, the generalization attributes of each classification algorithm are very sensitive to the model and parameter selection steps. Instead of keeping only one classification algorithm, if we combine several classification algorithms, we can bring more reliability [30] and accuracy [31] in the results [32]. While approaching towards an efficient classifier ensemble, it is necessary that the complexity of testing process should not increase in a way that isn't appropriate for a real time application. So this could be achieved through one of the following ensemble design strategies [32]:  
* Combine different classifiers or learning algorithms instead of keeping only one classifier which is giving the lowest estimation value of the expected risk. * For a given learning algorithm, apply various initialization procedures or model selection strategies to obtain different candidate solutions. After obtaining candidate solutions, combine them. * Generate subsets of training set or feature sets and train several classifiers on these subsets. Some classifiers can be more efficient on local subspaces and they will act like experts on their local domain. [PERSON] [33] introduced the concept of bagging that is also known as Bootstrapped Aggregation. Bagging combined voting with a vote generation process to obtain multiple classifiers. The idea behind bagging is to allow each base classifier to be trained on different random subsets of training examples, so that the base classifiers can become more diverse [34]. Bringing diversity in classifiers is a historically accepted area of research in the domain of ensemble learning, that increases the classification performance. It is commonly admitted that a large diversity amongst the classifiers in a team is preferred. However, depending on the context, diversity can be understood differently. For example one can view it as complementarity, or even orthogonality, between classifiers, or it can be considered as a measure of dependence [33]. Diversity is often used implicitly in the ensemble growing techniques, where the ensembles become progressively diverse. Another use of diversity is its application as optimization criterion for ensemble training. Finally, diversity controls the ensemble pertinence by checking if it is diverse enough [32]. An example of diversity method is bagging that generates several diverse base classifiers. The bagging makes use of original training set to generate multiple bootstrap training sets from it, so that a classifier can be generated to become a part of the ensemble. From a training set having \(N\) training examples, a bootstrap training set can be generated by performing \(N\) multinomial trials in such a way that one example is drawn out from the training set in each trial. In each trial every example present in the training set has a probability \(1/N\) of being drawn out. Bootstrap sampling does exactly this \(N\) times, and selects a number r in the range of 1 to \(N\). The algorithm selects the \(r\)-th training example and add it to the bootstrap training set \(S\).Which means that, some examples will be chosen more then one time to become the part of training set, while there will be some examples present in the original training set, that will never be able to become the part of bootstraped training set. In bagging, several classifiers are generated by creating \(M\) such bootstrap training sets. The bagging algorithm returns a function \(h(x)\) which is used to classify new examples on the basis of voting through the base models \(h_{1},h_{2},\ldots,h_{M}\). The corresponding class \(y\) is considered as one which gets the maximum number of votes. In general, bagged ensembles work better if the learning models are unstable.