## 2 AdaBoost Algorithm  
Boosting is a general term for a system performance enhancement. The principle of boosting is to effectively combine simple decision rules (so called weak classifiers) into strong ensembles of classifiers. This allows for the capturing of even quite complex decision boundaries. The Adaboost algorithm [27] (from adaptive boosting) is a powerful learning method with a built-in feature selection mechanism. The main concept of the AdaBoost learning algorithm is to create a strong classifier that has high detecting performance by joining weak classifiers. The AdaBoost algorithm learns by repeated calculations, and has two functions: choosing the feature and learning the classifier. Through reiteration of calculation, the simple classifying ability is reinforced, because weak classifiers that are an index of performance are added to the strong classifier in iteration [2]. After the initial step of learning, it updates a new weight to the previous classifier by using an emphasizing error. After learning, a strong classifier that forms the 2 nd layer perceptron is acquired [28]. In the AdaBoost algorithm proved by [PERSON] and [PERSON], errors come close to zero when the number of iterations is increased exponentially [29]. Suppose we take a set of training data (\(x_{1}\), \(c_{1}\)), (\(x_{2}\), \(c_{2}\)),..., (\(x_{n}\), \(c_{n}\)), where the input (predictor variable) \(x_{i}\in\mathbb{R}^{p}\), and the output (response variable) \(c_{i}\) is qualitative and assumes values in a finite set, e.g., 1, 2,..., K where \(K\) is the number of classes. The aim is to find a classification rule \(C(x)\) from the training data, so that when given a new input \(x\), one can give it a class label \(c\) from 1, 2,..., \(K\) problems [1]. What is the best possible classification rule? A commonly used definition of the best classification rule is to achieve the bottommost misclassification error rate. Usually, it is supposed that the training data are independently and identically distributed samples from an unknown probability distribution Prob(X,C). Then, the misclassification error rate for \(C(x)\)[1] is:  
\[\mathrm{E}_{X,C}\mathbb{I}_{C(X)\
eq C}=\mathrm{E}_{X}\mathrm{Prob}(C(X)\
eq C |X) \tag{1}\]  
\[\mathrm{E}_{X,C}\mathbb{I}_{C(X)\
eq C}=1-\mathrm{E}_{X}\mathrm{Prob}(C(X)=C|X) \tag{2}\]  
\[\mathrm{E}_{X,C}\mathbb{I}_{C(X)\
eq C}=1-\sum_{k=1}^{K}\mathrm{E}_{X}[ \mathbb{I}_{C(X)=k}\mathrm{Prob}(C=k|X)] \tag{3}\]  
It is clear that \(C^{*}(x)=\mathrm{arg}\underset{k}{\max}\mathrm{Prob}(C=k|X=x)\) will reduce this quantity with the misclassification error rate equal to \(1-\mathrm{E}_{X}\mathrm{max}_{k}\mathrm{Prob}(C=k|X)\). This classifier is named the Bayes classifier, and the error rate it reaches is the Bayes error rate. The AdaBoost algorithm is an iterative process that combines many weak classifiers to approximate the Bayes classifier \(C^{*}(x)\). Beginning with the unweighted training sample, the AdaBoost builds a classifier, for instance, a classification tree, which produces class labels. If a training data point is misclassified, the weight of that training data point is enlarged (boosted). A second classifier is built using the new weights, which are no longer equal. Again, misclassified training data have their weights boosted and the process is repeated [1]. A description of the AdaBoost algorithm in the form of a flowchart is shown in Figure 1.  
The following algorithm, Algorithm 1, shows the AdaBoost technique used to solve a classification problem of two classes. It is first used in case of two classes only and then it is developed to cover the multi-class problems as shown in Algorithm 2.