# Two Majority Voting Classifiers Applied to Heart Disease Prediction
## 3 Methodology
### Model
4:\(i\gets 0\)\(D\gets dim(x)\)\(\triangleright\) Number of dimensions
6:while\(i<D\)do\(val\_0\leftarrow\mathit{calc\_dens}(x[i],means\_0[i],vars\_0[i])\)\(\triangleright\) Given mean and variance, calculate density according to Equation (2)
8:\(val\_1\leftarrow\mathit{calc\_dens}(x[i],means\_1[i],vars\_1[i])\)\(\triangleright\) Repeat for class 1
9:if\(val\_0>val\_1\)then\(\triangleright\) Feature class based probabilities are compared
10:\(s\_0\gets c\_0+exp(-|kurtosis[i]|)+ks[i]\)\(\triangleright\) Kurtosis and KS-statistic added
11:else
12:\(s\_1\gets c\_1+exp(-|kurtosis[i]|)+ks[i]\)
13:endif
14:\(i\gets i+1\)
15:endwhile
16:\(y\gets 0\)\(\triangleright\) Class of \(x\)
17:if\(s\_0>s\_1\)then\(\triangleright\) Class of \(x\) to 0. 18:\(y\gets 0\)\(\triangleright\) Class of \(x\) to 0. 19:else
20:\(y\gets 1\)\(\triangleright\) Class of \(x\) to 1. 21:endif
22:return\(y\)
23:endprocedure
```  
**Algorithm 1** Proposed Base Estimator Method I: Gaussian Distribution based Majority Voting Classifier  
For GKMVB, \(X\) refers to the overall training data, whereas \(x\) refers to an observation from the test set. PREDICT() is supposed to be executed for each observation in the test dataset. \(val\_0\) and \(val\_1\) are the Gaussian-based probability densities of the sample vector \(x\), given class 0 and 1, respectively. \(s_{0}\) and \(s_{1}\) stand for the majority vote sums for each class. \(\exp(-|kurtosis[i]|)\) hack is used to obtain a scalar inversely proportional to the kurtosis of feature \(i\) (\(|kurtosis[i]|\)). \(ks[i]\) is directly used since its value is between 0 and 1. \(c_{0}\) and \(c_{1}\) are set according to the class priorities, which are \(c_{0}=2\) and \(c_{1}=1\) for the Spectf dataset and \(c_{0}=c_{1}=1\) for for the Statlog dataset. In the MKMVB variant, a Maxwell-Boltzmann random variable is fitted to each attribute's class components; that is, a fit is calculated for the samples belonging to class 0, and a second fit is calculated for the remaining samples (the samples of class 1). For each attribute, these two fits are stored to form a feature-level decision function. During the prediction stage, these feature-level decision functions are fused to determine the final class label via the majority vote sum rule (in the sum, the KS test values and class weights are used). Another point is that our density calculation is a bit modified version of the normal pdf. For a normal distribution of location \(\mu\) and scale \(\sigma\), the probability density function is [49]  
\[f(x)=\frac{1}{\sqrt{2\pi\sigma}}\exp(\frac{-(x-\mu)^{2}}{2\sigma^{2}}) \tag{1}\]  
We instead used  
\[g(x)=\frac{1}{c\sigma^{4}}\exp(\frac{-(x-\mu)^{2}}{2\sigma^{4}}) \tag{2}\]  
to emphasize the effect of the variance on the classification, and we have seen that this gives better results in combination with kurtosis and KS statistic weighting. Despite the fact that there are robust estimation routines for kurtosis [50], we have used the sample kurtosis [51]  
\[\kappa=\frac{\hat{\mu}_{4}}{\hat{\sigma}^{2}}-3 \tag{3}\]  
where \(\hat{\mu}_{4}\) is the sample moment defined by  
\[\hat{\mu}_{r}=n^{-1}\sum_{i=1}^{n}{(x_{i}-\bar{x})^{r}} \tag{4}\]  
and \(\hat{\sigma}^{2}\) is the sample variance. Our second method, MKMVB (details can be seen from Algorithm 2), differs from GKMVB in that the probability density function is used as given in [52] (\(x,\theta>0\))  
\[f(x)=\frac{4}{\sqrt{\pi}}\frac{1}{\theta^{3/2}}x^{2}\exp(-x^{2}/\theta) \tag{5}\]  
and its CDF is  
\[F(x)=\frac{1}{\Gamma(3/2)}\Gamma(\frac{3}{2},\frac{x^{2}}{\theta}) \tag{6}\]  
where \(\Gamma\) is the incomplete gamma function:  
\[\Gamma(a,x)=\int_{0}^{x}{u^{a-1}e^{-u}\,du} \tag{7}\]  
On the other hand, for MKMVB, the weighting is done by summing the separate KS statistic measures on two random variable fits.