HDSC Stage F OSP: Heart Failure Prediction
By: Chigozie Mbonu
Tags: Machine Learning, Hamoye Latest, Hamoye Features

Prediction App Link: https://sites.google.com/view/heart-failure-project/

GitHub Repository Link: https://github.com/HamoyeHQ/stage-f-07-heart-failure

Introduction

In most cultures or geographical settings, it is a common belief that heart failure occurs instantaneously due to immediate stress to the human heart, or perhaps due to the receipt of unaccommodating news, event, or happening which is sometimes considered too large for the mind to process at the moment. This hence leads to increased heart rate, blood pressure, and ultimately cardiovascular breakdown.

This in fact is a very big misconception. In truth, heart failure is a progressive underlying health defect caused by a series of events that have occurred over a prolonged period of time. In essence, most cardiovascular disease can be prevented by addressing behavioral risk factors such as tobacco use, unhealthy diet and obesity, physical inactivity, and harmful use of alcohol amongst others.

Cardiovascular diseases (CVDs) are the number one cause of death globally taking an estimated 17.9 million lives each year, which accounts for 31% of all deaths worldwide.

Objective

People with cardiovascular disease or who are at high cardiovascular disease risk (due to the presence of one or more risk factors such as hypertension, diabetes, hyperlipidemia, or already established disease) need early detection and management measures. A machine learning prediction model can help with this by predicting the likelihood of death/survival of a patient, and hence, help with immediate preventive measures that can be taken to prevent a negative outcome. The machine learning model here was created using twelve features that can be used to predict mortality by heart failure.

Data Used for Training the Machine Learning Model

Data Source: https://www.kaggle.com/andrewmvd/heart-failure-clinical-data

To build this model, we must first understand the underlying factors that lead to mortality by heart failure. This was done by collecting data on people from the past and present time that have either died from or survived heart diseases. People who are also either living or not living with the aforementioned risk factors were taking into consideration.

The dataset used was collected and compiled by Davide Chicco, Guiseppe Juman and was hosted on Kaggle.

Collaborators

This project is an open source project on Github under the Hamoye Data Science Internship that comprised of an eleven-man team of data scientists and data engineers alike. A Project Lead, Assistant Project Lead and a Query Analyst were selected to help coordinate the group. The workflow was evenly distributed among the team members, where each one was assigned a particular task, and his/ her performance was evaluated by the whole team.

In this project, each contributor is expected to make a pull request to the master branch, while the other team members would review and scrutinize the work. Regular meetings/interactions were held for better collaboration.

Members of our team are Micky Nnamdi, Jessica Ojo, Blessing Egharevba, Basil Ihuoma, Gaurav Jain, Chigozie Mbonu, Ayooluwa Adedipe, Charles Echefu, and Gloria Agunanna. (L-R as in the poster).

Understanding the Data (features) Used

Datasets Used for Training

As seen from the table above, the dataset used in modeling has 12 features which are the risk factors contributing to mortality by heart failure. After loading the dataset, an EDA was performed to better understand the data and its values. Some of the results of the EDA are as shown below.

EDA

Bar plots showing some Features counts

Counts for feature labels

Correlation Table

Insight Gained from Research

Performing some research on the features of the dataset, the following discoveries were made about the normal amount of each substance that should be present in the human body, and also what can be classified as being high or low;

i. Serum creatinine: this is the amount of creatinine in the bloodstream. Normal creatinine levels range from 0.9 to 1.3 mg/dl in men and 0.6 to 1.1mg/dl in women who are 18 to 60 years. High serum creatinine levels in the blood indicate dehydration, kidney malfunction or a blocked urinary tract, etc.

ii. Creatinine phosphokinase (CPK); is an enzyme found mainly in the heart, brain, and skeletal muscle. Normal CPK value is around 10 to 120. High CPK Value can be a result of stroke, heart attack, lung tissue death.

iii. Platelets; Platelets are the cells that circulate within our blood and bind together when there are damaged blood vessels. For women, the normal platelet count ranges from 157,000 to 371,000 platelets per microliter of blood, while for men the average platelet count ranges from 135,000 to 371,000 platelets per microliter of blood.

iv. Sodium; the normal range of sodium level is 135 to 145 milliequivalents per liter (mEq/)

Feature Engineering

From the insight gained from research and the EDA on the dataset, we performed feature engineering to create new features on the data based on the ranges of values. We also transformed the categorical binary values of 1 and 0 to True and False (this was particularly done because the data-type was showing int64 and not Boolean and hence would be treated as an integer while scaling the data. The code snippet is shown below:

def set_cpk(row):

if row[‘creatinine_phosphokinase’] >=10 and row[‘creatinine_phosphokinase’] <=120:

return ‘Normal’

else:

return ‘High’

A similar transformation was performed on other columns according to the standard medical values.

Selection of Evaluation Metrics

1. TP: when an observation is positive and the model rightly classifies it as positive.

2. FP: when an observation is negative and the model wrongly classifies it as positive.

3. FN: when an observation is positive and the model wrongly classifies it as negative.

4. TN: when an observation is negative and the model rightly classifies it as negative.

From the EDA performed on the dataset, we gathered that this was a classification problem and as such, our choices of evaluation of the model’s performance were streamlined to the use of;

i. Accuracy: the ratio in the percentage of the total number of correct predictions to the total number of observations in the target class.

ii. Precision: a measure of how precise the model was in predicting the positive class of the target variable.

iii. Recall: a measure of the error made by the model in predicting the Negative class of the target variable.

iv. F1_score: which is the harmonic mean of the precision and recall score.

Selection and Reason for Selection

We must point out that it is not possible to have high precision and recall at the same time, i.e. as precision increases, recall reduces; hence, there exists a tradeoff between the two metrics. So, in picking a preferred metric for evaluation we must consider the cost and applicability of the model to real-life situations.

This model is one that predicts if a patient is at risk of dying or surviving due to presence of one or more underlying risk factors. Considering this, it would be more costly for the model to wrongly predict that a patient would survive(FN) as this is likely to cause the patient to be at ease and take no preventive actions, only to end up dead after a few days. However, if the model wrongly predicts that a patient would die (FP) when the patient stands a high chance of survival, then the patient goes ahead and takes preventive measures which in the long run is a win on both ends.

In essence, we are trying to reduce the FN rate (type2 error) of the model, and hence looking at the mathematical expressions for recall and precision, as the FN outcomes increase the recall score reduces, and as the FP score increases the precision score reduces.

Accuracy was not put into much consideration here, because if there is a class imbalance between the negative class and positive class, and the model always predicts the majority class, then it could still achieve a high accuracy but in essence perform very poorly in real life use.

Base Line Model

After the exploratory data analysis phase was completed, our next line of action was to build a baseline model using the dataset and its features as it is to serve as a yardstick for comparison between the future models that would be created. We performed a K-fold cross-validation on several classification models like XG-Boost, Random-Forest, Decision Tree, etc. Of all the classification models trained, the random-forest classifier performed quite moderately and was the highest performing model among the rest. It gave the following performance metrics;

Accuracy: 0.83

Recall: 0.65

Precision: 0.88

F1_score: 0.75

The accuracy score obtained from the Random Forest classifier is particularly very insignificant due to the class imbalance that exists between the negative and positive observations.

Model Building

Due to the performance of the Random-Forest classifier as the best performing model, we decided to train future models using the Random-Forest classifier, but only after much transformation and pre-processing have been performed on the dataset.

Data Transformation

Transformed Data-set

Smoting

Due to the imbalance of the observations in the target class, we decided to generate some more synthetic data to bridge the imbalance, so we considered using SMOTE function, but the smote function has a drawback as it does not recognize categorical values written as 1 and 0’s. It generates values as type float and hence would not be a good fit for the dataset. So, we used SMOTENC which is another variant of the smote function that takes into consideration the categorical values in the dataset and returns synthetic data based on these values.

Value counts before and after smoting.

Scaling and Labels Encoding

The categorical non-Boolean values were encoded using the pandas built-in get_dummies() function, by specifying the target columns in the dataset. Due to the non-uniformity in the integer values of the dataset, we carried out scaling using Min-Max-Scaler by first dropping the categorical features from the data-frame.

x_bal = pd.get_dummies(x_bal, columns = [“sex”], drop_first = True) from sklearn.preprocessing import MinMaxScaler from sklearn.compose import ColumnTransformer

col = ['age','creatinine_phosphokinase','ejection_fraction',

'platelets','serum_creatinine','serum_sodium','time',

'anaemia','diabetes','high_blood_pressure','smoking',

'sex_Male']

col_trans = ColumnTransformer(remainder=’passthrough’, transformers = [(‘scaler’,MinMaxScaler(),[0,2,4,6,7,8,10])])

trans = col_trans.fit_transform(x) trans = DataFrame(trans,columns = col)

Scaled Data with Labels

Training the Model

Using scikit-learn inbuilt train-test-split, we performed a split on the dataset in a 20 percent test-train ratio. We trained a Random Forest model, but first by carrying out hyper-parameter tuning using grid search cross-validation technique. After the search was completed, the best parameter that would give the best model performance was determined and used to train the model on the train-set, and the train and fitted model evaluated on the test-set.

from sklearn.ensemble import RandomForestClassifier classifier = RandomForestClassifier(max_depth = 7, max_features = 'auto', random_state = 3, min_samples_leaf = 1, min_samples_split = 2,n_estimators = 100) classifier.fit(x_train, y_train)

The metrics obtained from testing the Random Forest model on the test set are as follows:

Classification Report Table

We have made a trade off between recall and precision, as evident from the confusion matrix. From the confusion matrix, the value 1 represents the amount of FN predictions made by the model on the test set, while the number 3 represent the number of FP predictions made by model on the test set, the number 32 and 46 represents the TP and TN predictions made by the model on the test set respectively.

Confusion Matrix

Building our end-to-end Heart Failure Prediction lightweight pipelines components using Python

Step 1: Setup the Environment by installing the following python packages

● We installed or update the pipelines SDK.

● Then imported the kfp and kfp.components packages.

● And created a release experiment in the Kubeflow pipeline

Step 2: Getting started with Python function-based components

This section demonstrates how each Python function-based component was built, by walking through the process of creating a simple component

● We defined our component’s code as a standalone python function.

1. Preprocessing Function: this function carries out the task of preprocessing of the data.

2. Analysis Function: this function carries out the task of data analysis and Exploratory data analysis.

3. Scaling and Transformation Function: this function normalizes and scales the pre-processed data.

4. Training Function: Carries out Model training.

5. Model Validation Function

Step 3: Build a pipeline component from the function

The next step was to convert the individual functions to a pipeline operation.

● Using kfp.components.create_component_from_func, we returned a factory function that you can use to create kfp.dsl.ContainerOp class instances for the pipeline. We also specify the base container image to run this function in.

Step 4: Build Kubeflow Pipeline

i. Our next step was to create the various components that will make up the pipeline. Define the pipeline using the @dsl.pipeline decorator.

ii. The pipeline function was defined and included a number of parameters that will be fed into our various components throughout execution. Kubeflow Pipelines are created declaratively. This means that the code won’t run until the pipeline is compiled.

- Note that while this is a great method to use locally, you could also use a cloud bucket for your persistent storage.

Step 5: Compile and run the pipeline

i. Finally, we fed our pipeline definition into the compiler and run it as an experiment. This will give us 2 links at the bottom that we can follow to the Kubeflow Pipelines UI where you can check logs, artifacts, inputs/outputs, and visually see the progress of your pipeline.

ii. Kubeflow Pipelines lets you group pipelines run by Experiments. You can create a new experiment, or call kfp.Client().list_experiments() to see existing ones. If you don’t specify the experiment name, the Default experiment will be used.

The PipeLine Graph:

Model Deployment

What is the need of creating a beautiful model without putting it out there for use by others? We created a web application using the streamlit web application library and the created web application is hosted on Heroku site. The web application can be used to predict if a patient would suffer mortality if one or more risk factors are present. The application gives a Boolean value of 1 for when a patient would die and 0 for when a patient would survive, accompanied by the degree of certainty that these events would occur in the form of a probability value ranging from 0 to 1. Below is a snippet of the application page.

Prediction App Interface

Recommendation

This project was carried out using limited amount of data-set, further improvement can be made by collecting more data and retraining the model so that the machine learning algorithm can have more data to learn from and hence give more accurate predictions.