# Two Majority Voting Classifiers Applied to Heart Disease Prediction
## 3 Methodology
### Model
The OP scores can be seen in Tables 2 and 3, where CFARS-AR FL stands for the algorithm proposed in [2], and CFARS-AR NB, CFARS-AR SVM, CFARS-AR ANN stand for NB, SVM, and ANN backed with the feature selection process proposed in [2]. The accuracy, sensitivity, and specificity of CFARS-AR NB, CFARS-AR SVM, and CFARS-AR ANN are obtained from [2] (they have the default configurations of WEKA [56]). In Tables 2 and 3, 'Proposed Method-I' and 'Proposed Method-II' stand for GKMVB and MKMVB, respectively. We have also tested the scenarios where the preprocessing (Robust Scaling + Quantile Transformation + RFECV Feature Selection) remains the same while the base estimator is one of the state-of-the-art classifiers NB, SVM, ANN, or other ones such as DT, Perceptron, Passive Aggressive Classifier (PAC), Linear Discriminant Analysis (LDA) and Gaussian Process Classifier (GPC). The results are in Tables 4 and 5. ## 5 Discussion

The proposed Majority Voting Algorithms (GKMVB and MKMVB) have several advantages. The performance comparison given in Table 2 shows that the proposed methods outperform CFARS-AR SVM, CFARS-AR ANN, CFARS-AR NB, and CFARS-AR FL in terms of the optimized precision of the classification on both datasets. An additional point is that our method is more successful on unbalanced datasets with respect to the balanced dataset, which is quite rare in the medical field. The success of MKMVB on Spectf can be explained roughly by the capability of the Maxwell-Boltzmann distribution to capture the spatial characteristics of the SPECT images, which needs further investigation. Moreover, the 'quasi'-density function given in Equation (2) also needs further investigation. From an empirical point of view, the logic behind the general separation capability of this function can be explained by its emphasis on the variance. Another point that separates our work from the others is the maximal usage of ensemble learning: first, in the base estimator level where the majority voting is applied; Second, in the bagging phase, where the subsampling is applied; Lastly, in the stacking phase where the classifiers are fused. This three-fold ensemble structure makes the model robust. One disadvantage of the proposed method is the dependence on the random nature of Bagging Classifiers. Although it almost took two or three trials to get the optimal random state, a Bagging Classifier having a higher average OP (and possibly accuracy) than CFARS-AR FL is, of course, a better option. One could claim that the methods are too 'handcrafted' due to the statistical computations of the attributes, which can be seen as 'against the spirit of Machine Learning'. While this critique is partly true, we think that making a statistical analysis and grounding the work on the output of this analysis is suited to the framework of 'Statistical Learning Theory' as long as the analysis is automatic. Albeit there is a lack of a specific theoretical assessment, such as [40], we think that assisting the majority vote sum classifiers by characteristics of the distribution of the random variable does no harm. Moreover, since the two datasets have distinct feature characteristics, good optimized precision results imply the 'generic classifier' potential of the proposed methods. For future work, we can note that a density estimation other than a Maxwell-Boltzmann or Gaussian, which would be more accurate and novel than the one at hand, can be developed. Regression variants can be plugged into a Logitboost framework, or more sophisticated new base classifiers can be designed. Classical probability distributions can be evaluated to find the most suitable one for a majority voting scheme. Normality tests other than kurtosis measures can be used or engineered. This work is modular in that all of its methods for 'density estimation', 'normality', and 'voting' can be replaced by more efficient ones.