Applying Machine Learning Models to Predict The Presence of Heart Disease in Humans
By: Arvin Seifipour
Tags: Logistic Regression, Machine Learning, AI, Data Science, K Nearest Neighbours

Artificial Intelligence(A.I) is gradually taking over many industries and automating tasks with high efficiency and accuracy. Machine Learning is an application of A.I, where systems learn from data and improve without explicit programming. One of the applications of Machine Learning in the medical field is predicting diagnosis for different diseases and conditions.

It’s important to have a model with high accuracy since the predictions are concerned with the health of human beings, so it is necessary to test different models and see which one provides a better result.

I’m going to explore two Machine Learning models, Logistic Regression and K Nearest Neighbor, and implement them to predict diagnosis for the presence of Heart Disease in Humans. The dataset I’m gonna be using comes from UCI and can be found on kaggle(link on the bottom of this article).

I’m gonna use Jupyter Notebook as my environment, Python3 as the programming language, Seaborn and matplotlib for data visualization, and SKLearn library for Machine Learning models and metrics.

Let’s import the libraries first:

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

import seaborn as sns

from sklearn.linear_model import LogisticRegression

Let’s import data and visualize the row and columns, as well as the amount of different labels:

The column with the name ‘target’ will include our labels(0 for no heart disease and 1 for heart disease) and the rest of the columns will be used for our training dataset. Before feeding the columns as training features, we need to make sure they are not highly correlated so we can sustain accuracy.

Add this code and you should get a polished heatmap visualization with the darkness indicating the amount of correlation:

#see if features are correlated

plt.figure(figsize = (20,20))

matrix = np.triu(df.corr())

sns.heatmap(df.corr(),annot=True,mask=matrix)

Now we need to split our dataset into testing and training sets, we can use sklearn to accelerate this task. After splitting the data, we’re gonna scale it, so values that are not on the same scale don’t cause miscalculations:

from sklearn.model_selection import train_test_split

from sklearn import preprocessing

#split data into training and testing sets target = df['target']

x = df.drop(columns=['target'])

y = target

X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=0) #preprocess and scale

scaler = preprocessing.StandardScaler()

X_train = pd.DataFrame(scaler.fit_transform(X_train))

X_test=pd.DataFrame(scaler.transform(X_test))

Now we can instantiate our Logistic Regression model and train our data. Logistic Regression is a classifier that uses a logistic function(Sigmoid Curve) to model a binary variable. Now we’re gonna feed the model for training, then predict based on data from the testing set:

#instantiate model and train

logreg = LogisticRegression() # fit model

logreg.fit(X_train, y_train) #use model to make predictions on test set

predictions = logreg.predict(X_test)

Now that we have made predictions, we can compare our labels in y_test and predictions to evaluate our model, SKLearn has a pre-made confusion matrix that puts out values for: True Positive(TP), False Positive(FP), True Negative(TN), and False Negative(FN). Here’s the code to add a confusion matrix:

from sklearn import metrics

from sklearn.metrics import confusion_matrix

print(confusion_matrix(y_test,predictions))

Then from the matrix, we can calculate accuracy:

accuracy = (TP+TN)/(TP+TN+FP+FN)

For this model, the accuracy is 82.9%

Now let’s use K Nearest Neighbor to see if we can get better accuracy.

K Nearest Neighbor(Knn) is a model that can do both classification and regression(prediction based on continuous values). For this case, we are using Knn as a classifier. The model predicts by simply taking the vote of the nearest data points. For example, if we use the 7 nearest neighbors, and 3 of them have the label ‘0’ and the other 4 are ‘1’, the new prediction will be ‘1’. The ‘k’ value is the number of nearest neighbors we’re going to use for prediction. It’s a challenge to find the optimal ‘k’ value, so we’re gonna use different values and see which one works best:

from sklearn.neighbors import KNeighborsClassifier # try ro find best k value

k_scores = [] #train with different values of k

for i in range(1,20):

knn = KNeighborsClassifier(n_neighbors = i)

knn.fit(X_train, y_train)

k_scores.append(knn.score(X_test, y_test))



plt.plot(range(1,20), k_scores)

plt.show()

x-axis = k values, y axis = accuracy score

As we can see in the visualization, when the ‘k’ value is right around 7, the accuracy peaks above 86%, so let’s use 7 as our ‘k’ nearest neighbors.

#instantia knn model based on k value with highest k_score and train

knn = KNeighborsClassifier(n_neighbors = 7)

knn.fit(X_train, y_train) #make metrics report

from sklearn.metrics import classification_report

predictions = knn.predict(X_test)

report = classification_report(predictions,y_test)

print(report)

Metric report for knn:

Sources:

Jupyter Notebook → https://github.com/noirvin/Machine-Learning/blob/master/Final%20Project.ipynb

UCI Dataset → https://www.kaggle.com/ronitf/heart-disease-uci