Exploratory Data Analysis on Heart Disease UCI data set** | by Aishah Ismail | Towards Data Science

Exploratory Data Analysis (EDA) is a pre-processing step to understand the data. There are numerous methods and steps in performing EDA, however, most of them are specific, focusing on either visualization or distribution, and are incomplete. Therefore, here, I will walk-through step-by-step to understand, explore, and extract the information from the data to answer the questions or assumptions. There are no structured steps or method to follow, however, this project will provide an insight on EDA for you and my future self.
Cardiovascular diseases (CVDs) or heart disease are the number one cause of death globally with 17.9 million death cases each year. CVDs are concertedly contributed by hypertension, diabetes, overweight and unhealthy lifestyles. You can read more on the heart disease statistics and causes for self-understanding. This project covers manual exploratory data analysis and using pandas profiling in Jupyter Notebook, on Google Colab. The dataset used in this project is UCI Heart Disease dataset, and both data and code for this project are available on my GitHub repository.
Initially, the dataset contains 76 features or attributes from 303 patients; however, published studies chose only 14 features that are relevant in predicting heart disease. Hence, here we will be using the dataset consisting of 303 patients with 14 features set.
Import and get to know the dataData Cleaning
Import and get to know the data
Data Cleaning
a) Check the data type
b) Check for the data characters mistakes
c) Check for missing values and replace them
d) Check for duplicate rows
e) Statistics summary
f) Outliers and how to remove them
3. Distributions and Relationship
a) Categorical variable distribution
b) Continuous variable distribution
c) Relationship between categorical and continuous variables
4. Automated EDA using pandas profiling report
Variables or features explanations:
age (Age in years)sex : (1 = male, 0 = female)cp (Chest Pain Type): [ 0: asymptomatic, 1: atypical angina, 2: non-anginal pain, 3: typical angina]trestbps (Resting Blood Pressure in mm/hg )chol (Serum Cholesterol in mg/dl)fps (Fasting Blood Sugar > 120 mg/dl): [0 = no, 1 = yes]restecg (Resting ECG): [0: showing probable or definite left ventricular hypertrophy by Estes’ criteria, 1: normal, 2: having ST-T wave abnormality]thalach (maximum heart rate achieved)exang (Exercise Induced Angina): [1 = yes, 0 = no]oldpeak (ST depression induced by exercise relative to rest)slope (the slope of the peak exercise ST segment): [0: downsloping; 1: flat; 2: upsloping]ca [number of major vessels (0–3)thal : [1 = normal, 2 = fixed defect, 3 = reversible defect]target: [0 = disease, 1 = no disease]
age (Age in years)
sex : (1 = male, 0 = female)
cp (Chest Pain Type): [ 0: asymptomatic, 1: atypical angina, 2: non-anginal pain, 3: typical angina]
trestbps (Resting Blood Pressure in mm/hg )
chol (Serum Cholesterol in mg/dl)
fps (Fasting Blood Sugar > 120 mg/dl): [0 = no, 1 = yes]
restecg (Resting ECG): [0: showing probable or definite left ventricular hypertrophy by Estes’ criteria, 1: normal, 2: having ST-T wave abnormality]
thalach (maximum heart rate achieved)
exang (Exercise Induced Angina): [1 = yes, 0 = no]
oldpeak (ST depression induced by exercise relative to rest)
slope (the slope of the peak exercise ST segment): [0: downsloping; 1: flat; 2: upsloping]
ca [number of major vessels (0–3)
thal : [1 = normal, 2 = fixed defect, 3 = reversible defect]
target: [0 = disease, 1 = no disease]
The dataset description is provided here.
Let’s begin...!!
Here we have 303 rows with 14 variables.
a) Check the data type.
The variables types are
Binary: sex, fbs, exang, target
Categorical: cp, restecg, slope, ca, thal
Continuous: age, trestbps, chol, thalac, oldpeak
Is the type of variable correctly classified by python ? Let’s get to know the data type.
Note here that the binary and categorical variable are classified as different integer type by python. We will need to change them to ‘object’ type.
b. Check for the data characters mistakes
feature ‘ca’ ranges from 0–3, however, df.nunique() listed 0–4. So lets find the ‘4’ and change them to NaN.
feature ‘ca’ ranges from 0–3, however, df.nunique() listed 0–4. So lets find the ‘4’ and change them to NaN.
2. Feature ‘thal’ ranges from 1–3, however, df.nunique() listed 0–3. There are two values of ‘0’. So lets change them to NaN.
c) Check for missing values and replace them
and visualize the missing values using Missingno library. The missing values are represented by the horizontal lines. This library provide an informative way of visualizing the missing values located in each column, and to see whether there is any correlation between missing values of different columns. Here’s a shout out to a great article on Missingno.
msno.matrix(df)
Replace the NaN with median.
df = df.fillna(df.median())df.isnull().sum()
d) Check for duplicate rows
e) Statistics summary
Basically, with df.describe(), we should check on the min and max value for the categorical variables (min-max). Sex (0–1), cp (0–3), fbs (0–1), restecg (0–2), exang (0–1), slope (0–2), ca (0–3), thal (0–3). We should also observe the mean, std, 25% and 75% on the continuous variables.
f) Outliers and how to remove them
There are also other several ways of plotting boxplot.
fig = px.box(df, x=”target”, y=”chol”)fig.show()
or using seaborn
sns.boxplot(x=’target’, y=’oldpeak’, data=df)
Now, let’s define and list out the outliers..!!
and here are the listed outliers.
Let’s drop the outliers.
a) target variable distribution
There are more diseased than healthy patients.
b) Age variable distribution
# print(df.age.value_counts())df[‘age’].hist().plot(kind=’bar’)plt.title(‘Age Distribution’)
The age are normally distributed.
# Analyze distribution in age in range 10print(df.age.value_counts()[:10])sns.barplot(x=df.age.value_counts()[:10].index,y=df.age.value_counts()[:10].values,palette=’Set2')plt.xlabel(‘Age’)plt.ylabel(‘Age distribution’)
Most of the patients are in the age between 50s to 60s. Let’s take a quick look basic stats. The mean age is about 54 years with ±9.08 std, the youngest is at 29 and the oldest is at 77.
c) Gender distribution according to target variable
From the bar graph, we can observe that among disease patients, male are higher than female.
d) Chest pain distribution according to target variable
Chest pain (cp) or angina is a type of discomfort caused when heart muscle doesn’t receive enough oxygen rich blood, which triggered discomfort in arms, shoulders, neck, etc.
However, looking at the bar graph above, its raised a question of higher number of healthy subject having typical_angina. Or in other word, most of the healthy subject having chest pain, which is also discussed here. Chest pain can be subjective due to stress, physical activities and many more and varies between gender. Women and elderly patients usually have atypical symptoms with a history of disease. This article provide analysis comparing typical anginal vs nontypical angina patients in a clinical trial.
e) Fasting blood sugar distribution according to target variable
Fasting blood sugar or fbs is a diabetes indicator with fbs >120 mg/d is considered diabetic (True class). Here, we observe that the number for class true, is lower compared to class false. However, if we look closely, there are higher number of heart disease patient without diabetes. This provide an indication that fbs might not be a strong feature differentiating between heart disease an non-disease patient.
f) Slope distribution according to target variable
g) Distribution plot on continuous variables.
normal distribution for: age, trestbps and almost for chol
oldpeak is left-skewed
thalac is right-skewed
h) Sns pairplot to visualize the distribution.
oldpeak having a linear separation relation between disease and non-disease.
thalach having a mild separation relation between disease and non-disease.
Other features don’t form any clear separation
i) correlation
‘cp’, ‘thalach’, ‘slope’ shows good positive correlation with target
‘oldpeak’, ‘exang’, ‘ca’, ‘thal’, ‘sex’, ‘age’ shows a good negative correlation with target
‘fbs’ ‘chol’, ‘trestbps’, ‘restecg’ has low correlation with our target
! pip install the pandas profiling
! pip install the pandas profiling
! pip install https://github.com/pandas-profiling/pandas-profiling/archive/master.zip
and here is a snapshot of the automated EDA
So there you go, a complete walk-through on UCI Heart Disease EDA.
I hope you find this guide useful and I will continue to explore EDA using another type of data set.
Feel free to comment/ correct/ ask.
Happy exploring !! ❤
You can check out the steps on applying Pandas Profiling Report on Jupyter Google Colab my article below.