# Heart Disease Risk Prediction Using Machine Learning Classifiers with Attribute Evaluators
## 3 Materials and Methods
### Machine Learning Models for Classification  
Researchers have applied multiple supervised machine learning algorithms on a single dataset to identify the best classifier for disease prediction. This section discusses the various classifiers that were used in this work to predict heart disease risk.  
Naive Bayes (NB) is based on the Bayes theorem, which assumes that the training observations are samples from a set of statistical distributions. Each response class has its distribution. Each distribution in the model provides a probability that a new data point would be found at its location. For the normal distribution, the parameters are the mean and standard deviation [34].  
Figure 2: Visualization of attributes of the Cleveland heart dataset.  
Logistic regression (LR) is an equation where each predictor is multiplied by a coefficient and summed together. This sum becomes the argument for the logistic function to predict the class [35]. For a single observation x with n features the response y is given by  
\[\text{y}=\frac{1}{1+e^{-(\beta_{0}+\beta_{1}x_{1}+\cdots+\beta_{n}x_{n})}} \tag{1}\]  
Sequential minimal optimization (SMO) is an algorithm that is used to solve very large quadratic programming (QP) optimization problems quickly without any extra storage requirement while training a support vector machine (SVM) [23]. SMO selects two Lagrange multipliers and analytically finds the optimal values for these multipliers to solve the SVM QP problem [36].  
The instance-based classifier IBk, also known as k-nearest neighbors, determines the class of observation by comparing it to nearby observations from the training data set. The distance measure that is used to determine the neighbors can be selected from a range of options. The model uses majority voting from the K nearest data points and assigns a class to the unknown observation [37].  
Bagging, also known as the bootstrap aggregation technique, is a simple and powerful ensemble technique that is used to decrease the variance of the decision tree classifier [37]. It provides the learning algorithm with a training set consisting of a random sample of m training examples that are selected from the initial training set of m items on each run.  
Boosting is an example of an ensemble technique that creates a robust classifier from several weak classifiers. Adaptive boosting (AdaBoostM1) is a successful boosting algorithm that was developed for binary classification and is used to boost the performance of any machine learning algorithm. The decision tree with one level or one decision for classification, called the decision stump, is the most suitable algorithm to work with AdaBoost [38].  
JRip is a rule-based classifier that utilizes repeated incremental pruning to produce error reduction (RIPPER). It is a bottom-up approach for learning rules that treats specific judgments of examples in the training data as a class and finds a set of rules that covers all members of the class [39].  
Random forest constructs a forest of random trees by creating a set of decision trees from a random sample of the training set to minimize the variance at the expense of a small increase in bias (controlling over-fitting) and results in a final prediction model that should be more accurate and reliable. While growing the trees, the random forest adds more randomness to the algorithms by using random thresholds for each attribute [40].