Will you have heart disease? A heart disease predictor using Machine Learning
By: Ka Hou Sio
Tags: Healthcare Technology, Metis, Data Science, Supervised Learning, Machine Learning

Project Design

The problem that I am trying to solve is to predict/classify whether a person has heart disease or not, based on some of the personal info and medical test results.

That is my primary goal of my project, and the second goal of my project is to build a heart disease predictor — a demo app using Flask.

For model evaluation, I will use recall score as my primary metric. Because I want to take into account of those cases where I predict someone is healthy but they actually have heart disease. I also pay attention to precision score when I am evaluating a model, because I don’t want to tell someone that he/she has heart disease but they actually don’t. This will get people upset and we don’t want that.

Tools

I will use numpy, pandas, matplotlib, seaborn, sklearn and Flask for this project.

Pipeline

Data

The data that I worked with is from UCI repo, The dataset contains 303 entries and 14 features. (The dataset is a little bit old, it is from 1988.)

The dataset includes features such as age, gender, resting blood pressure, chest pain type and etc. Some of these test can only be performed in a clinic, and patient won’t be able to perform those tests at home by themselves. So my target audience will be doctors or nurses.

Cleaning and EDA

The target variable is the last column from the original dataset which is named as num. It is a categorical feature labeled as 0, 1, 2, 3, 4. While 0 means the patient has no heart disease, 1, 2, 3, 4 mean that the patient has some kind of heart disease. Since my problem is to predict whether the person has heart disease or not, I grouped 1, 2, 3, 4 all together as one group to label the person has heart disease.

After grouping, I identified there are 6 NaNs values in my dataset which is just less than 2% of my dataset, so I decided to drop those NaNs. Finally, my class 0, which is the normal patient is about 54% of my whole dataset, and class 1 is about 46% of my dataset. I think my dataset is a balance dataset.

After some EDAs ,data cleanings and using StandScaler to scale my data (because my target variable is between 0&1, so I need to scale my non-0&1 columns to between 0&1), I built a base model that includes all 14 variables, using KNN, Logistic regression, Decision tree, Random forest, SVM, Naive Bayes, and XGboost. The best classifier for my base model is Logistic regression with a cross-validation recall score of 0.81. Then I used grid search to try to find the best hyper-parameter of each models, and still Logistic regression has the best CV recall score.

Modeling and Feature Engineering

Next, I decided to do some feature engineering to try to find out which variables contribute significantly to my models. I used chi-square test from sklearn to find out significance level of each variable to my model, and I selected all the variables that has a score is positive and high.

Model Performance in Training Set

I used those selected variables to build more models using the algorithms that I used for the base model. Logistic regression still has the best CV recall score. After perform grid search again with the new selected variables, my Logistic regression CV recall score got improved to 0.8571 and it is the best among all the models.

ROC curve

Then I used logistic regression with tuned hyper-parameters to make prediction using my test set. The recall score of my test set is 0.87, which is even better than my training CV score, so I think my model doesn’t have overfitting problem.

Precision vs Recall Curve

Then I looked at my confusion matrix, precision vs. recall curve graph and ROC graph of my model. The best threshold that could maximize both precision and recall is about 0.5, which is the threshold that I am using for my model. Also I have a AUC score of 0.9087 which is pretty good as well. So I decided this will be my final model.

Web App

After having my final model ready, I need to prepare a few things before my web app prototype could actually function. I need to create a pipeline and use pickle to export my final model and data for later use in Flask. After defining a function to make prediction, I used HTML, CSS and Flask to create a simple demo web app — Heart Disease Predictor. Users can input their info and the predictor will tell you whether you have a heart disease or not with a confidence score.