## 3 Materials and Methods
### Model Training  
In this step, feature sets generated in the feature extraction phase were used to construct four popular individual machine learning classifiers. These classifiers were linear discriminate analysis (LDA), K-nearest neighbor (KNN), support vector machine (SVM), and decision tree (DT) classifiers. Distance metric used for the KNN is Euclidian distance and the number of neighbors (\(K\)) is equal to 1. Linear kernel function is employed for SVM. As for DT, the split criterion used is gini index and the number of splits is 100.  
Ensemble classifier is a technique that combines prediction of several individual classifiers by a fusion rule. Ensemble combines the strength of each classifier in the pool of classifiers which commonly leads to an enhancement in the performance of individual classifiers. Also, it can lower chances of poor results that can be produced from a certain unsuitably individual model. Ensemble process is similar to the case in which clinical diagnosis is made by taking diagnosis from a number of clinicians to end up to a more confident final decision.  
Ensemble has two topologies; cascaded and parallel. In the former topology, classifiers of the pool are connected in cascaded manner. This topology includes the Adaboosting. In Adaboosting, every classifier in the pool of classifiers tries to enhance the performance of the previous weaker classifier. Adaboosting employs a class weighting resampling technique to learn the subsequent model in the ensemble. Examples that are misclassified using the initial classifier in the ensemble are given higher weights and then these resampled instances go into the next classifier. This is reiterated until all models are processed. In the later structure, classifiers of the ensemble were connected in parallel and their predictions are combined using majority voting, maximum probability, minimum probability, or averaging methods. This type includes Bagging. Bagging ensemble depends on the Bootstrap resampling technique which generates several random data subsets from the original data. These subsets are used to construct the pool of classifiers available in the ensemble [26].  
Bagging and Adaboosting were employed in this paper to build ensembles of decision tree classifiers. Both individual and ensemble classifiers are constructed and validated using 10 cross validation method.  
Our experiments were implemented on Intel(r) core I7 and NVIDIA GEFORCE, Windows 10, 64 bit with 4 GB RAMS. MATLAB R2016 b software was used for the implementation of the proposed framework.  
In order to evaluate the performance of proposed framework, several metrics were adopted such as; classification accuracy (CA), sensitivity, specificity, and area under receiver operation characteristic (AUC) were used. Equations (1)-(3) describe these metrics.  
\[\mathrm{CA}=\frac{\mathrm{T_{P}+T_{N}}}{\mathrm{T_{N}+F_{P}+F_{N}+T_{P}}} \tag{1}\]  
\[\mathrm{sensitivity}=\frac{\mathrm{T_{P}}}{\mathrm{T_{P}+F_{N}}} \tag{2}\]  
\[\mathrm{specificity}=\frac{\mathrm{T_{P}+T_{N}}}{\mathrm{T_{N}+F_{P}+F_{N}+T_{P }}}, \tag{3}\]  
where; \(T_{P}\)--True Positive, \(T_{N}\)--true negative, \(F_{P}\)--false positive, and \(F_{N}\)--false negative.  
A receiving operating characteristics (ROC) is a curve demonstrating the sensitivity as a function of the 1- specificity. The AUC describes the area under the ROC curve it value ranges between '0' and '1'. A higher AUC value gives a better classification performance.