# Two Majority Voting Classifiers Applied to Heart Disease Prediction
## 3 Methodology
### Model  
In the data pre-processing stage, we have applied a Quantile Transformer [43] and a Robust Scaler [44], where the Interquartile Range (IQR) of the data is considered. Then, we have selected important features via Cross-Validated Recursive Feature Elimination (RFECV) [45], which is a cross-validated variant of [46]. An SVM estimator with a linear kernel is chosen to calculate the importance of each selected feature. Afterward, a Bagging Classifier which has a Logistic Regression [47] base estimator is stacked [48] with a Bagging Classifier which has a custom written base estimator. Our final estimator (i.e.,'meta-classifier') is also a Logistic Regression instance, combining the outputs of the bagged Logistic Regression and the bagged Majority Vote. The overall model can be seen in Figure 1.  
Our contribution relies on employing a majority voting scheme through attribute weighting calculations. For the first variant, \(\exp(-|\kappa|)\), where \(\kappa\) is the kurtosis, both it and the KS test results are plugged into the classifier by adding their values to the overall sum associated with the class label. The Maxwell-Boltzmann distribution is fitted to each attribute individually in the second variant. The KS test results of the fit are added to the overall sum of the winner class for the considered attribute. The proposed Gaussian Distribution and Kurtosis-based Majority Voting Base Classifier Algorithm (GKMVB) is employed by a Bagging Classifier for heart disease prediction (Figure 2). The method is a binary classification scheme since Heart Disease Prediction has two classes. Details of the algorithms can be seen from Algorithm 1. The base estimator comprises two functions: FIT() & PREDICT(). The FIT() function is used during training to calculate the statistical measures. In particular, the class-based mean, variance, and kurtosis values are calculated for each feature \(i\). In the prediction phase, PREDICT() is employed, where we use the means and variances to calculate the Gaussian probability densities of the attributes, and majority voting is applied to determine the winning class. Additionally, we add the kurtosis and the KS statistic of the component to the overall value of the class vote. The pseudo-code of the proposed algorithm is as follows:  
Figure 1: Prediction Model. ```
1:procedurefit(\(X,y\))\(\triangleright\) Dataset, class labels
2:\(means\_0\leftarrow\mathit{calc\_means}(X,y,0)\)\(\triangleright\) Means of features for class 0
3:\(means\_1\leftarrow\mathit{calc\_means}(X,y,1)\)\(\triangleright\) Means of features for class 1
4:\(vars\_0\leftarrow\mathit{calc\_variances}(X,y,0)\)\(\triangleright\) Variances of features for class 0
5:\(vars\_1\leftarrow\mathit{calc\_variances}(X,y,1)\)\(\triangleright\) Variances of features for class 1
6:\(kurtosis\leftarrow\mathit{calc\_kurtosis}(X)\)\(\triangleright\) Kurtosis of each feature
7:\(ks\leftarrow\mathit{ks\_test}(X)\)\(\triangleright\) KS test result of each feature
8:return\((means\_0,means\_1,vars\_0,vars\_1,kurtosis,ks)\)
9:endprocedureprocedurepredict(\(x,c\_0,c\_1,means\_0,means\_1,vars\_0,vars\_1,kurtosis,ks\))\(\triangleright\) means\_0,\(means\_1,vars\_0,vars\_1,kurtosis,\) and \(ks\) are results of the FIT procedure. \(c\_0\) and \(c\_1\) are method parameters. 2:\(s\_0\gets 0\)\(\triangleright\) Initialize votes for class 0. \(s\_1\gets 0\)\(\triangleright\) Initialize votes for class 1.