# An Enhanced Machine Learning Framework for Type 2 Diabetes Classification Using Imbalanced Data with Missing Values

Authors: Kumarmangal Roy; Muneer Ahmad; Kinza Waqar; Kirthanaah Priyaah; Jamel Nebhen; Sultan S Alshamrani; Muhammad Ahsan Raza; Ihsan Ali
Journal: Complexity
 (2021)

---

## Abstract

Diabetes is one of the most common metabolic diseases that cause high blood sugar. Early diagnosis of such a condition is challenging due to its complex interdependence on various factors. There is a need to develop critical decision support systems to assist medical practitioners in the diagnosis process. This research proposes developing a predictive model that can achieve a high classification accuracy of type 2 diabetes. The study consisted of two fundamental parts. Firstly, the study investigated handling missing data adopting data imputation, namely, median value imputation, K-nearest neighbor imputation, and iterative imputation. Consequently, the study validated the implications of these imputations using various classification algorithms, i.e., linear, tree-based, and ensemble algorithms, to see how each method affected classification accuracy. Secondly, Artificial Neural Network was employed to model the best performing imputed data, balanced with SMOTETomek ensuring each class is represented fairly. This approach provided the best accuracy of 98% on the test data, outperforming accuracies achieved in prior studies using the same dataset. The dataset used in this study is concerned with gender and population. As a prospect, the study recommends adopting a larger population sample without geographic boundaries. Additionally, as the developed Artificial Neural Network model did not undergo any specific hyperparameter tuning, it would be interesting to explore tuning on top of normalized data to optimize accuracy further.


---

## Body

## 1. Introduction

Diabetes mellitus, or more commonly known as only diabetes, is a global health problem affecting people worldwide. According to [1], 463 million people have diabetes globally, and 88 million are from South East Asia. Diabetes is the general term for separate but related chronic diseases. It occurs when the human body is incapable of producing or properly using the hormone known as insulin [2]. Insulin is responsible for converting glucose from the food humans consume into the energy they need for daily life; thus, its imbalance is detrimental to human health. The cause of diabetes remains a mystery, but studies showed genetics and environmental factors [3].Additionally, for most countries, diabetes has evolved with dietary changes, aging populations, reduced physical activity, and other unhealthy lifestyle choices [4]. Essentially, there are two types of diabetes. Type 1 diabetes occurs when the body’s cells responsible for producing insulin are destroyed, causing insulin deficiency. The more prevalent, type 2 diabetes occurs when insulin production is not sufficient or adequate to control blood glucose level, causing relative insulin deficiency [5].Diabetes increases the risk of being susceptible to other diseases as its development often leads to heart attack, stroke, heart disease, and many other severe consequences [6]. The first step in the development of type 2 diabetes is prediabetes. Prediabetes means that the blood sugar level is higher than usual but not high enough to be type 2 diabetes. However, the long-term damage of diabetes, especially to the heart, blood vessels, and kidneys, may already start, making an early diagnosis of diabetes of vital importance [7]. Assessing symptoms of diabetes is a challenging task whereby 50% of patients with type 1 and type 2 symptoms remain underdiagnosed due to no noticeable signs of disease and its complex interdependence to other conditions [2]. There is a need for accurately identifying patterns and relationships from patient data. With the increase in the amount of data in medical databases, it has become too time-consuming and costly to assign staff to conduct traditional manual analysis to identify data patterns and recognize anomalies in patients’ medical records.In the survey conducted by National Diabetes and Diabetic Retinopathy, it is stated that the prevalence of diabetes in India is at 11.8% [8]. According to the National Health and Nutrition Examination Survey data, more people died from the consequences of diabetes than of HIV/AIDS, malaria, and tuberculosis combined. The World Health Organization has predicted that by the year 2030, diabetes will be the seventh leading cause of death [9]. By 2040 around 482 million people will be affected globally by prediabetes and an estimated 642 million by diabetes, according to the International Diabetes Federation [10]. These statistics show that type 2 diabetes does not discriminate. It has an impact on everyone across all classes, societies, countries, and continents. Unless effective diagnosis methods are utilized, this problem will remain prevalent. This issue will stress the healthcare systems, challenge economies, and predominantly affect life quality.A practical solution is to employ data mining techniques for examining data and seeking useful information from medical databases that help clinical staff diagnose type 2 diabetes more accurately [11]. The healthcare industry generates a vast amount of valuable data comprising patient information, electronic medical records, treatment history, etc. Conducting knowledge extraction on these data can identify patterns and support decision-making and cost reduction by developing intelligent medical systems. Early detection can prevent diabetes with higher accuracy [12].Researchers have utilized a variety of classification algorithms to achieve high-performing models. This research builds on previous literature by developing a classification model using data mining and laboratory information of the patients while applying different data imputation approaches to enhance model performance. Besides, we consider the imbalance in data distribution between diabetic patients and ordinary people. It helped in training the classification model to ensure that each class was represented fairly. By identifying the best performing data preprocessing methods, the machine learning model’s performance accuracy was optimized, making it the highest among previous studies.The positive impact of the proposed solution is on both the patients and the medical practitioners. The countries with a high rate of diabetic patients can employ an accurate early prediction system of diabetes. The more precise the prediction, the more accurate the severity estimation of the underlying medical condition [13]. Detecting diabetes at an early stage and treating it is an essential step towards keeping people with diabetes healthy. Early detection will benefit the patients and medical staff as they can make use of computer-based systems rather than relying on traditional manual analysis of data, which is costly, time-consuming, and prone to human errors [14].The rest of the article is organized as follows: Section2 covers a summary of prior literature, Section 3 presents the data and methods used to adopt the proposed solution. Section 4 outlines the results and discussion of the findings, followed by a conclusion in Section 5.


## 2. Literature Review

Diabetes mellitus is a common chronic disease found in numerous countries around the world. While it can seriously deteriorate an individual’s quality of life, its early diagnosis can minimize the long-term adverse effects [15]. With the abundance of medical data from electronic records, patient information, treatment data, etc., researchers have utilized data mining techniques to discover knowledge through this data. Useful knowledge extraction can support decision-making and cost reduction of intelligent diabetes diagnosis systems. Table 1 outlines extensive research conducted in the early prediction of diabetes with a summary of selected 20 prior studies. For a fair comparison, all selected previous studies have used the same diabetes dataset publicly available on UCI Machine Learning Repository to conduct their investigation on the development of a diabetes predictive model [30]. Additionally, we noted data preprocessing in handling missing data and suggested solutions related to class imbalance problems for these studies.Table 1
Summary of machine learning solutions for diabetes classification.
StudyPreprocessing approachAlgorithmSignificanceGuldogan et al. [16]Missing data deletedMultilayer perceptron (MLP) and radial basis function (RBF)MLP giving an accuracy of 78.1%Alam et al. [17]Median value imputationANN, random forest, K-means clusteringANN giving an accuracy of 75.7%Wang et al. [18]naïve Bayes imputation, ADASYN oversamplingRandom forest87.10% accuracySarwar et al. [19]Method details not statedK-nearest neighbors, naïve Bayes, support vector machine, decision tree, logistic regression, random forestK-nearest neighbors and support vector machine giving accuracy 77%Woldemichael and Menaria [20]Mean value imputationBackpropagation, support vector machine, J48, naïve BayesBackpropagation giving an accuracy of 83.11%Sisodia and Sisodia [21]Method details not statedDecision tree, support vector machine, naïve BayesNaïve Bayes giving an accuracy of 76.30%Wu et al. [22]Mean value imputationK-means + logistic regression95.42% accuracyKomi et al. [13]Method details not statedGaussian mixture model, support vector machine, logistic regression, extreme learning machine, ANNANN giving an accuracy of 89.0%AlThunayan et al. [15]Method details not statedBayesian, naïve Bayes, J48, random tree, random forest, REP tree, FT tree, CART, SMORandom forest giving an accuracy of 84.11%Vaishali et al. [23]Missing data kept unchangedNaïve Bayes, J48, MLP, MOE fuzzy classifierMOE fuzzy giving accuracy of 83.04%Geman et al. [24]Missing data kept unchangedHybrid adaptive neural network84.27% accuracyRamesh et al. [25]Missing data kept unchangedRecurrent deep neural Network (RNN)0.145 error rateSoltani and Jafarian [14]Missing data kept unchangedANN types: PNN, MLP, RBF, GRNNPNN giving an accuracy of 81.49%Ahmad et al. [2]Mean value imputationMLP, ID3, J48J48 giving an accuracy of 89.3%AlJarullah [11]Missing data deletedJ4878.17% accuracyMarcano-Cedeño et al. [26]Missing data deletedArtificial metaplasticity on multilayer perceptron algorithm (AMMLP)89.93% accuracyPatil et al. [3]Missing data deletedK-means + C4.592.38% accuracyLekkas and Mikhailov [27]Missing data deletedRule-based fuzzy classification79.37% accuracyKahramanli and Allahverdi [28]Method details not statedANN + fuzzy neural network84.24% accuracyHan et al. [29]Missing data deletedID3, decision treeID3 giving an accuracy of 80.0%Existing literature shows various data mining and machine learning algorithms such as neural networks, tree-based algorithms, rule-based fuzzy classifiers, linear classifiers, and hybrid models used to output a highly accurate predictive model. Artificial Neural Network (ANN) was the most popular classifier where 11 out of 20 prior studies used different ANN types in their experiments. More than half of the preliminary studies result in prediction accuracy within the 80%–90% range. Wu et al. [22] used a hybrid predictive model to get 95.42% accuracy in prediction, the highest in the literature. Medical datasets are likely to contain many missing data values. We reviewed their suggested solutions that influenced the classification accuracy. Earlier studies showed that using mean value imputation [2, 20, 22] led to better accuracy results than when missing data were deleted [11, 16, 27].Consequently, most prior literature chose to delete missing values, with only 5 out of the 20 studies using imputation to replace missing values with meaningful values based on central tendency measures. Another aspect of data preprocessing in binary classification is often found in the medical domain to solve the class imbalance problem. The data contain larger samples of class labeled as not having diabetes than having diabetes. It can lead to a biased predictive model where the accuracy applies only to the majority class. Among prior studies, only [18] addresses both missing values and the class imbalance problem. The authors used naïve Bayes imputation to replace missing values with meaning values and use ADASYN oversampling to get equal representation of both classes in training the predictive model and achieve 87.10% accuracy.A high-quality classification model is one whose accuracy reflects its capability of generalizing unseen data samples. The development of such a model essentially requires more data representing both classes fairly. While deleting missing data values is a simple solution, it risks losing data points with valuable information reducing the model’s ability to generalize [31]. Furthermore, skewed class distribution will ensure that predictive performance applies to both majority and minority classes. Therefore, despite the extensive reported research, prediction of diabetes remains a challenging problem that needs to be further explored, especially in terms of investigating the effect of preprocessing approaches on classification accuracy. There is a need to experiment with various data imputation methods and solve class imbalance problems to ensure that the predicting model’s resulting accuracy is of high quality.


## 3. Methodology

Optimal classification performance is dependent on efficient data preparation and data preprocessing methods. The proposed algorithm fills the research gap identified in the literature review. Handling missing values of attributes through a meaningful approach can substantially improve a machine learning model’s performance [32]. Initially, the investigation focused on a comparative study of three different data imputation methods to identify the optimal way of handling missing values in the dataset. Consequently, data from the highest performing method was employed to achieve optimized performance in modeling.Figure1 presents the proposed framework for type 2 diabetes prediction. It has two phases where the first phase consists of identifying optimal imputation technique. We apply three different imputation techniques after loading the data, namely, median, iterative, and KNN imputations. We handle each attribute outliers and split instances into a proportion of 70% training and 30% testing, modeling using seven different machine learning algorithms. Table 2 outlines these algorithms below.Figure 1
Proposed framework for type 2 diabetes prediction.Table 2
Classification machine learning algorithms.
AlgorithmShort descriptionLogistic regression (LR)It uses a logistic function to model the probability of a specific classK-nearest neighbor (KNN)It uses similarity of features/attributes to predict new data points based on how closely it resembles the points in the training setDecision tree (DT)It uses a tree structure in which each internal node represents an attribute, each branch represents the outcome of a condition set on the feature, and each leaf node represents a class labelRandom forest (RF)It uses multiple decision trees to train and output the class that is the mode of the classes from decision treesSupport vector machine (SVM)It uses the kernel to transform data and identify an optimal boundary between the possible outputsGradient boosting (GB)It uses a combination of many weak learning models together to create a robust predictive modelLight gradient boosting machine (LGBM)It uses tree-based learning algorithms as a GB frameworkWe chose the highest performing outlier-treated imputed dataset for the second phase. In the second phase, we adopt a random sampling technique to produce some noise in the data. It then further preprocess to handle the class imbalance problem using undersampling and oversampling. It gives an equal representation of each class in the machine learning model’s training data. The preprocessed data are then split into 70% train and 30% test datasets. Standard scaling smoothens and prepares the training data for modeling. ANN is chosen as the machine learning model based on its common use in past literature. ANN model architecture is created with 1 initial layer, 5 hidden layers, and 1 final layer. RELU activation function is applied to each layer except the final layer. Instead, the sigmoid activation function is used in the final layer. Adam optimizer is used for model compiling and “binary_crossentropy” is used as a loss function. After a few trials, we could conclude that 10 epochs showed good results in terms of “accuracy”; a primary evaluation metric for this model.Figure2 shows the ANN model architecture with an input layer, all hidden layers, activation function, and the output layer.Figure 2
ANN model architecture.Figure3 represents the various steps taken to achieve the research objectives, i.e., to obtain high classification accuracy of the type 2 diabetes predictive model. Once the dataset is loaded, we create three duplicate copies of the data for three separate implementation approaches.Figure 3
The sequence of methodology steps.Let us consider that the dataset loaded isX.(1)X⟶X1,X2,X3,where (X1,X2,X3) are the duplicate datasets created.(2)X1∗Median Value Imputation⟶X1−new,(3)X2∗Iterative Imputation⟶X2−new,(4)X3∗K−Nearest Neighbor Imputation⟶X3−new.Equations (2), (3), and (4) show individual datasets, with X1,X2,X3 being imputed. X1−new, X2−new and X3−new are the datasets created, respectively, for datasets X1,X2,X3.(5)X1−new,X2−new,X3−new∗Outlier Treatment⟶X1−new′,X2−new′,X3−new′We process individual imputed datasets with outlier treatment in equation (5). X1−new,X2−new,andX3−new are then split into 70% train set and 30% test set individually. Each dataset’s training set is modeled using 10-fold cross-validation across the seven machine learning algorithms. We then use different metrics and accuracies to determine model performance. Test set enables further evaluation of performance, leading to the conclusion of which imputed dataset provides the best accuracy.(6)X1−new′∗Random Sampling⟶X1−new″.X1−new, the outlier-treated dataset, is then exposed to random sampling to introduce some noise in the data, as shown in equation (6).(7)X1−new″∗SMOTETomek⟶X1−new‴.SMOTETomek link is used to create under-oversampling of the data. Equation (7) shows that X1−new‴ created is our current modified dataset. This step helps to convert an imbalanced dataset to a balanced dataset. X1−new‴ is then split into test-train data of ratio 70%–30%. The training data are then scaled for processing in ANN:(8)X1−new‴∗Standard Scaling⟶X1−new‴.ANN model’s performance is judged based on different metrics. Prediction outcome is drawn out of the stable model for better understanding.


### 3.1. Data Description

The dataset used in this study is taken from the National Institute of Diabetes and Digestive and Kidney Diseases. As discussed by [23], this is a publicly available dataset at [30]. A detailed description of all attributes is given in Table 3. It contains several medical predictors: the independent variables and one dependent variable, the “Outcome.”Table 3
Dataset attributes description.
No.Attribute nameAttribute descriptionAttribute type1PregnanciesNumber of pregnancies of a womanNumeric2Glucose (mg/dl)Plasma glucose concentration for 2 hours in an oral glucose tolerance testNumeric3Blood pressure (mmHg)Diastolic blood pressureNumeric4Skin thickness (mm)Triceps skinfold thicknessNumeric5Insulin (mu U/ml)2-hour serum insulinNumeric6BMI (kg/m2)Body mass index (weight in kg/(height in m)2)Continuous7Diabetes pedigree functionDiabetes pedigree functionContinuous8AgeAge in yearsNumeric9OutcomeTarget variable (0 or 1) 0 for negative, 1 for positiveBinaryFigure4 illustrates the histogram of each attribute, explaining the distribution and skewness, and shows which attributes have missing values. Based on the histograms, the attributes age (A), diabetes pedigree function (B), insulin (C), pregnancies (D), and skin thickness (E) show that most of the data are on the left side of the histogram, which makes it skewed to the right. This means that when data are positively skewed and the mean is larger than the median.Figure 4
Dataset attribute distribution (right-skewed distribution). (a) Age, (b) diabetes pedigree function, (c) insulin, (d) pregnancies, and (e) skin thickness.
(a)(b)(c)(d)(e)Figure5 shows histograms for BMI (A), blood pressure (B), and glucose (C) are symmetric; i.e., they have a normal distribution. The data are symmetrically distributed where most of the observations are around the central peak.Figure 5
Dataset attribute distribution (normal distribution). (a) BMI, (b) blood pressure, and (c) glucose.
(a)(b)(c)Most healthcare data contain the missing value, noisy and inconsistent data. This diabetes dataset is no exception where its attributes/features include missing values. Namely, glucose, blood pressure, skin thickness, insulin, and BMI have zero (0) values. We observed several candidates with a zero reading/value, for instance, five candidates with a zero glucose level, 35 candidates with a zero blood pressure, 227 candidates with a zero skin thickness, 374 candidates with a zero insulin level, and 11 candidates with a zero BMI. It reflects an inaccurate scenario; i.e., the zero values present in the data are incorrect and will probably lead to wrong results for diabetes prediction. Thus, we need an imputation of these particular zero values with something more meaningful for each parameter. As any machine learning model’s performance heavily depends on the data, we performed necessary preprocessing techniques to handle the missing information.Figure6 represents the percentage of missing data related to a specific attribute. 48.69% of data are missing for insulin. 29.55% of information is missing for the skin thickness feature. 4.55% of data are missing for the blood pressure attribute. 1.43% of data are missing for BMI, and 0.65% of data are missing for the glucose attribute.Figure 6
Percentage of missing data per attribute.


### 3.2. Data Preprocessing

In this research, we consider data preprocessing as a significant component. In the first phase of the proposed framework, we conducted an investigation on data imputation methods. After we determine the optimal method, we handled the data’s outliers and the class imbalance problem.


#### 3.2.1. Handling Missing Values

For the first phase, we applied three different imputation techniques to fill in the missing information. Method-I refers to imputation where we group by each missing variable median value under the outcome variable and assign those values to miss-values. Method-II refers to the iterative imputation technique. Method-III refers to the KNN imputation technique.(1) Method-I. In order to fill missing values, we are grouping each missing variable’s median values in accordance with the outcome class and assign those values to missing values. For example, the median value of glucose parameter is 107 for outcome class 0 and 140 for outcome class 1. All the missing values with 0 outcome class will be replaced with value 107 and all the missing values with 1 outcome class will be replaced with value 140.The correlation matrix for this imputed dataset is shown in Figure7. The highest positive correlation is 0.57, between skin thickness and BMI. The highest correlated factor for output class is glucose at 0.50, followed by insulin at 0.38 and BMI at 0.32.Figure 7
Median value imputed data correlation matrix.(2) Method-II. Iterative Imputation. A sophisticated practice to find and replace missing values of a dataset is iterative imputation. This involves modeling to predict each feature as a function of all other features and keep repeating the process of estimating feature values multiple times. The repetition process allows the estimated values for other features to be used as input criteria for predicting missing values during subsequent iterations.This is basically a data transformation method. Bayesian Ridge model is used as a function to all other input parameters by default. All parameters are considered in ascending order, filling the parameter with the least missing values first and then proceeding to the next parameters having the next minimum number of missing values. Different regression algorithms can be used to estimate the missing values for each parameter, although linear methods are often used for simplicity.The correlation matrix for this imputation method is shown in Figure8. The highest correlation factor is between skin thickness and BMI (0.71). The highest correlated factor for output class glucose is 0.50, followed by other factors such as insulin (0.36) and BMI (0.32).Figure 8
Iterative imputed data correlation matrix.(3) Method-III. KNN Imputation. Another good practice to find and replace missing values is through KNN imputation using the KNN machine learning model. In this method, the k-nearest neighbor algorithm is applied to impute a missing value by finding the samples in the training set “closest” to it and averaging these nearby points to fill in the value.Similar to the last method, KNN imputer is another data transformation method. KNN imputation requires numeric input values, which in our case is the ideal scenario. It is a lazy learner as it learns on the fly. It checks the nearest observation/neighbor to discover its path. Configuration of KNN imputation often requires selecting the distance measure and the number of nearest neighbors for each prediction. The default distance measure is Euclidean distance and the number of neighbors selected is 5.The correlation matrix for this imputation method is shown in Figure9. The highest correlation factor is between skin thickness and BMI (0.64). The highest correlated factor for output class glucose is 0.50, followed by other factors such as insulin (0.32) and BMI (0.31). Overall, all three imputed dataset correlation matrices show that the most correlated features with the target variable Outcome are glucose, insulin, and BMI.Figure 9
KNN imputed data correlation matrix.


#### 3.2.2. Handling Outliers

The next preprocessing step was to handle outliers in all three imputed datasets. We set a threshold to decide on which data points constituted as outliers. We calculate the first quartile, third quartile, and interquartile range. We set the upper limit as third quartile +1.5∗ interquantile_range of the variable, and the lower limit as quartile 1−1.5∗ interquantile_range of the variable.(9)up_limit=quartile3+1.5∗interquantile_range,low_limit=quartile1−1.5∗interquantile_range.The results shown below illustrate the median value imputed dataset before and after outliers are handled.Figure10 shows outliers for blood pressure feature. We can see that there are multiple data points, which are out of bound. Blood pressure is treated with both upper limit and lower limit outlier correction. All these data points are corrected and shown in Figure 10(b) after the removal of outliers.Figure 10
Handling outliers for blood pressure feature. Before (a) and after (b).
(a)(b)Figure11 shows outliers for the diabetes pedigree function (DPF) feature. The DPF has multiple data points, which are beyond the upper limit bound of the boxplot. Upper limit outlier treatment is applied to DPF. Figure 11(b) shows the DPF feature after the removal of outliers.Figure 11
Handling outliers for DPF feature. Before (a) and after (b).
(a)(b)Figure12 shows outliers for the skin thickness feature. There seems to be only one data point that is beyond the upper limit of the boxplot. Upper limit outlier treatment is applied to skin thickness. Figure 12(b) shows skin thickness features after the removal of outliers.Figure 12
Handling outliers for skin thickness feature. Before (a) and after (b).
(a)(b)Figure13 shows outliers for the insulin feature. Insulin has multiple data points, which are beyond the upper limit bound of the boxplot. Upper limit outlier treatment is applied to insulin. Figure 13(b) shows insulin features after the removal of outliers.Figure 13
Handling outliers for insulin feature. Before (a) and after (b).
(a)(b)Figure14 shows outliers for the pregnancy feature. We can see that there are three data points that are out of bounds before outliers are handled. These particular data points can cause erroneous outcomes if not treated properly. Pregnancy feature is treated with upper limit outlier treatment. These data points were corrected and shown in Figure 14(b) after the removal of outliers.Figure 14
Handling outliers for the pregnancy feature. Before (a) and after (b).
(a)(b)Figure15 shows outliers for the BMI feature. We can see that there are multiple data points that are out of bound. Most of the data points are beyond the upper limit. There is only one data point that is beyond the lower limit. BMI is treated with both upper limit and lower limit outlier correction. Figure 15(b) shows BMI feature after the removal of outliers.Figure 15
Handling outliers for the BMI feature. Before (a) and after (b).
(a)(b)Figure16 shows outliers for the glucose feature. We can see that there is only one data point that is out of bound. Only glucose is treated with lower limit outlier treatment as it has data points breaching the lower limit. This data point is corrected and shown in Figure 16(b) after the removal of outliers.Figure 16
Handling outliers for glucose feature. Before (a) and after (b).
(a)(b)


#### 3.2.3. Handling Class Imbalance

The final preprocessing technique was to handle class imbalance and scale the imputed data. The imbalance in the number of samples that were diabetic compared to those that did not have diabetes was 268 by 500. Data sampling techniques reduced getting biased results in predictive performance and got more data samples for learning. According to [33], a combination of synthetic minority oversampling techniques (SMOTE) and random under- or oversampling performs better than when SMOTE is used by itself. Therefore, in this research, we used a combination of random oversampling and SMOTETomek. Firstly, we used random oversampling to create more data samples and then transform them by SMOTETomek. SMOTETomek combines over- and undersampling using SMOTE and Tomek links. SMOTE randomly increases minority class examples by synthesizing new minority instances between existing minority instances. Tomek links are pair of examples that belong to different classes and are each other’s nearest neighbors. After class imbalance has been dealt with, we normalize data features using standard scaling and prepare them as input into the ANN machine learning model.


## 3.1. Data Description

The dataset used in this study is taken from the National Institute of Diabetes and Digestive and Kidney Diseases. As discussed by [23], this is a publicly available dataset at [30]. A detailed description of all attributes is given in Table 3. It contains several medical predictors: the independent variables and one dependent variable, the “Outcome.”Table 3
Dataset attributes description.
No.Attribute nameAttribute descriptionAttribute type1PregnanciesNumber of pregnancies of a womanNumeric2Glucose (mg/dl)Plasma glucose concentration for 2 hours in an oral glucose tolerance testNumeric3Blood pressure (mmHg)Diastolic blood pressureNumeric4Skin thickness (mm)Triceps skinfold thicknessNumeric5Insulin (mu U/ml)2-hour serum insulinNumeric6BMI (kg/m2)Body mass index (weight in kg/(height in m)2)Continuous7Diabetes pedigree functionDiabetes pedigree functionContinuous8AgeAge in yearsNumeric9OutcomeTarget variable (0 or 1) 0 for negative, 1 for positiveBinaryFigure4 illustrates the histogram of each attribute, explaining the distribution and skewness, and shows which attributes have missing values. Based on the histograms, the attributes age (A), diabetes pedigree function (B), insulin (C), pregnancies (D), and skin thickness (E) show that most of the data are on the left side of the histogram, which makes it skewed to the right. This means that when data are positively skewed and the mean is larger than the median.Figure 4
Dataset attribute distribution (right-skewed distribution). (a) Age, (b) diabetes pedigree function, (c) insulin, (d) pregnancies, and (e) skin thickness.
(a)(b)(c)(d)(e)Figure5 shows histograms for BMI (A), blood pressure (B), and glucose (C) are symmetric; i.e., they have a normal distribution. The data are symmetrically distributed where most of the observations are around the central peak.Figure 5
Dataset attribute distribution (normal distribution). (a) BMI, (b) blood pressure, and (c) glucose.
(a)(b)(c)Most healthcare data contain the missing value, noisy and inconsistent data. This diabetes dataset is no exception where its attributes/features include missing values. Namely, glucose, blood pressure, skin thickness, insulin, and BMI have zero (0) values. We observed several candidates with a zero reading/value, for instance, five candidates with a zero glucose level, 35 candidates with a zero blood pressure, 227 candidates with a zero skin thickness, 374 candidates with a zero insulin level, and 11 candidates with a zero BMI. It reflects an inaccurate scenario; i.e., the zero values present in the data are incorrect and will probably lead to wrong results for diabetes prediction. Thus, we need an imputation of these particular zero values with something more meaningful for each parameter. As any machine learning model’s performance heavily depends on the data, we performed necessary preprocessing techniques to handle the missing information.Figure6 represents the percentage of missing data related to a specific attribute. 48.69% of data are missing for insulin. 29.55% of information is missing for the skin thickness feature. 4.55% of data are missing for the blood pressure attribute. 1.43% of data are missing for BMI, and 0.65% of data are missing for the glucose attribute.Figure 6
Percentage of missing data per attribute.


## 3.2. Data Preprocessing

In this research, we consider data preprocessing as a significant component. In the first phase of the proposed framework, we conducted an investigation on data imputation methods. After we determine the optimal method, we handled the data’s outliers and the class imbalance problem.


### 3.2.1. Handling Missing Values

For the first phase, we applied three different imputation techniques to fill in the missing information. Method-I refers to imputation where we group by each missing variable median value under the outcome variable and assign those values to miss-values. Method-II refers to the iterative imputation technique. Method-III refers to the KNN imputation technique.(1) Method-I. In order to fill missing values, we are grouping each missing variable’s median values in accordance with the outcome class and assign those values to missing values. For example, the median value of glucose parameter is 107 for outcome class 0 and 140 for outcome class 1. All the missing values with 0 outcome class will be replaced with value 107 and all the missing values with 1 outcome class will be replaced with value 140.The correlation matrix for this imputed dataset is shown in Figure7. The highest positive correlation is 0.57, between skin thickness and BMI. The highest correlated factor for output class is glucose at 0.50, followed by insulin at 0.38 and BMI at 0.32.Figure 7
Median value imputed data correlation matrix.(2) Method-II. Iterative Imputation. A sophisticated practice to find and replace missing values of a dataset is iterative imputation. This involves modeling to predict each feature as a function of all other features and keep repeating the process of estimating feature values multiple times. The repetition process allows the estimated values for other features to be used as input criteria for predicting missing values during subsequent iterations.This is basically a data transformation method. Bayesian Ridge model is used as a function to all other input parameters by default. All parameters are considered in ascending order, filling the parameter with the least missing values first and then proceeding to the next parameters having the next minimum number of missing values. Different regression algorithms can be used to estimate the missing values for each parameter, although linear methods are often used for simplicity.The correlation matrix for this imputation method is shown in Figure8. The highest correlation factor is between skin thickness and BMI (0.71). The highest correlated factor for output class glucose is 0.50, followed by other factors such as insulin (0.36) and BMI (0.32).Figure 8
Iterative imputed data correlation matrix.(3) Method-III. KNN Imputation. Another good practice to find and replace missing values is through KNN imputation using the KNN machine learning model. In this method, the k-nearest neighbor algorithm is applied to impute a missing value by finding the samples in the training set “closest” to it and averaging these nearby points to fill in the value.Similar to the last method, KNN imputer is another data transformation method. KNN imputation requires numeric input values, which in our case is the ideal scenario. It is a lazy learner as it learns on the fly. It checks the nearest observation/neighbor to discover its path. Configuration of KNN imputation often requires selecting the distance measure and the number of nearest neighbors for each prediction. The default distance measure is Euclidean distance and the number of neighbors selected is 5.The correlation matrix for this imputation method is shown in Figure9. The highest correlation factor is between skin thickness and BMI (0.64). The highest correlated factor for output class glucose is 0.50, followed by other factors such as insulin (0.32) and BMI (0.31). Overall, all three imputed dataset correlation matrices show that the most correlated features with the target variable Outcome are glucose, insulin, and BMI.Figure 9
KNN imputed data correlation matrix.


### 3.2.2. Handling Outliers

The next preprocessing step was to handle outliers in all three imputed datasets. We set a threshold to decide on which data points constituted as outliers. We calculate the first quartile, third quartile, and interquartile range. We set the upper limit as third quartile +1.5∗ interquantile_range of the variable, and the lower limit as quartile 1−1.5∗ interquantile_range of the variable.(9)up_limit=quartile3+1.5∗interquantile_range,low_limit=quartile1−1.5∗interquantile_range.The results shown below illustrate the median value imputed dataset before and after outliers are handled.Figure10 shows outliers for blood pressure feature. We can see that there are multiple data points, which are out of bound. Blood pressure is treated with both upper limit and lower limit outlier correction. All these data points are corrected and shown in Figure 10(b) after the removal of outliers.Figure 10
Handling outliers for blood pressure feature. Before (a) and after (b).
(a)(b)Figure11 shows outliers for the diabetes pedigree function (DPF) feature. The DPF has multiple data points, which are beyond the upper limit bound of the boxplot. Upper limit outlier treatment is applied to DPF. Figure 11(b) shows the DPF feature after the removal of outliers.Figure 11
Handling outliers for DPF feature. Before (a) and after (b).
(a)(b)Figure12 shows outliers for the skin thickness feature. There seems to be only one data point that is beyond the upper limit of the boxplot. Upper limit outlier treatment is applied to skin thickness. Figure 12(b) shows skin thickness features after the removal of outliers.Figure 12
Handling outliers for skin thickness feature. Before (a) and after (b).
(a)(b)Figure13 shows outliers for the insulin feature. Insulin has multiple data points, which are beyond the upper limit bound of the boxplot. Upper limit outlier treatment is applied to insulin. Figure 13(b) shows insulin features after the removal of outliers.Figure 13
Handling outliers for insulin feature. Before (a) and after (b).
(a)(b)Figure14 shows outliers for the pregnancy feature. We can see that there are three data points that are out of bounds before outliers are handled. These particular data points can cause erroneous outcomes if not treated properly. Pregnancy feature is treated with upper limit outlier treatment. These data points were corrected and shown in Figure 14(b) after the removal of outliers.Figure 14
Handling outliers for the pregnancy feature. Before (a) and after (b).
(a)(b)Figure15 shows outliers for the BMI feature. We can see that there are multiple data points that are out of bound. Most of the data points are beyond the upper limit. There is only one data point that is beyond the lower limit. BMI is treated with both upper limit and lower limit outlier correction. Figure 15(b) shows BMI feature after the removal of outliers.Figure 15
Handling outliers for the BMI feature. Before (a) and after (b).
(a)(b)Figure16 shows outliers for the glucose feature. We can see that there is only one data point that is out of bound. Only glucose is treated with lower limit outlier treatment as it has data points breaching the lower limit. This data point is corrected and shown in Figure 16(b) after the removal of outliers.Figure 16
Handling outliers for glucose feature. Before (a) and after (b).
(a)(b)


### 3.2.3. Handling Class Imbalance

The final preprocessing technique was to handle class imbalance and scale the imputed data. The imbalance in the number of samples that were diabetic compared to those that did not have diabetes was 268 by 500. Data sampling techniques reduced getting biased results in predictive performance and got more data samples for learning. According to [33], a combination of synthetic minority oversampling techniques (SMOTE) and random under- or oversampling performs better than when SMOTE is used by itself. Therefore, in this research, we used a combination of random oversampling and SMOTETomek. Firstly, we used random oversampling to create more data samples and then transform them by SMOTETomek. SMOTETomek combines over- and undersampling using SMOTE and Tomek links. SMOTE randomly increases minority class examples by synthesizing new minority instances between existing minority instances. Tomek links are pair of examples that belong to different classes and are each other’s nearest neighbors. After class imbalance has been dealt with, we normalize data features using standard scaling and prepare them as input into the ANN machine learning model.


## 3.2.1. Handling Missing Values

For the first phase, we applied three different imputation techniques to fill in the missing information. Method-I refers to imputation where we group by each missing variable median value under the outcome variable and assign those values to miss-values. Method-II refers to the iterative imputation technique. Method-III refers to the KNN imputation technique.(1) Method-I. In order to fill missing values, we are grouping each missing variable’s median values in accordance with the outcome class and assign those values to missing values. For example, the median value of glucose parameter is 107 for outcome class 0 and 140 for outcome class 1. All the missing values with 0 outcome class will be replaced with value 107 and all the missing values with 1 outcome class will be replaced with value 140.The correlation matrix for this imputed dataset is shown in Figure7. The highest positive correlation is 0.57, between skin thickness and BMI. The highest correlated factor for output class is glucose at 0.50, followed by insulin at 0.38 and BMI at 0.32.Figure 7
Median value imputed data correlation matrix.(2) Method-II. Iterative Imputation. A sophisticated practice to find and replace missing values of a dataset is iterative imputation. This involves modeling to predict each feature as a function of all other features and keep repeating the process of estimating feature values multiple times. The repetition process allows the estimated values for other features to be used as input criteria for predicting missing values during subsequent iterations.This is basically a data transformation method. Bayesian Ridge model is used as a function to all other input parameters by default. All parameters are considered in ascending order, filling the parameter with the least missing values first and then proceeding to the next parameters having the next minimum number of missing values. Different regression algorithms can be used to estimate the missing values for each parameter, although linear methods are often used for simplicity.The correlation matrix for this imputation method is shown in Figure8. The highest correlation factor is between skin thickness and BMI (0.71). The highest correlated factor for output class glucose is 0.50, followed by other factors such as insulin (0.36) and BMI (0.32).Figure 8
Iterative imputed data correlation matrix.(3) Method-III. KNN Imputation. Another good practice to find and replace missing values is through KNN imputation using the KNN machine learning model. In this method, the k-nearest neighbor algorithm is applied to impute a missing value by finding the samples in the training set “closest” to it and averaging these nearby points to fill in the value.Similar to the last method, KNN imputer is another data transformation method. KNN imputation requires numeric input values, which in our case is the ideal scenario. It is a lazy learner as it learns on the fly. It checks the nearest observation/neighbor to discover its path. Configuration of KNN imputation often requires selecting the distance measure and the number of nearest neighbors for each prediction. The default distance measure is Euclidean distance and the number of neighbors selected is 5.The correlation matrix for this imputation method is shown in Figure9. The highest correlation factor is between skin thickness and BMI (0.64). The highest correlated factor for output class glucose is 0.50, followed by other factors such as insulin (0.32) and BMI (0.31). Overall, all three imputed dataset correlation matrices show that the most correlated features with the target variable Outcome are glucose, insulin, and BMI.Figure 9
KNN imputed data correlation matrix.


## 3.2.2. Handling Outliers

The next preprocessing step was to handle outliers in all three imputed datasets. We set a threshold to decide on which data points constituted as outliers. We calculate the first quartile, third quartile, and interquartile range. We set the upper limit as third quartile +1.5∗ interquantile_range of the variable, and the lower limit as quartile 1−1.5∗ interquantile_range of the variable.(9)up_limit=quartile3+1.5∗interquantile_range,low_limit=quartile1−1.5∗interquantile_range.The results shown below illustrate the median value imputed dataset before and after outliers are handled.Figure10 shows outliers for blood pressure feature. We can see that there are multiple data points, which are out of bound. Blood pressure is treated with both upper limit and lower limit outlier correction. All these data points are corrected and shown in Figure 10(b) after the removal of outliers.Figure 10
Handling outliers for blood pressure feature. Before (a) and after (b).
(a)(b)Figure11 shows outliers for the diabetes pedigree function (DPF) feature. The DPF has multiple data points, which are beyond the upper limit bound of the boxplot. Upper limit outlier treatment is applied to DPF. Figure 11(b) shows the DPF feature after the removal of outliers.Figure 11
Handling outliers for DPF feature. Before (a) and after (b).
(a)(b)Figure12 shows outliers for the skin thickness feature. There seems to be only one data point that is beyond the upper limit of the boxplot. Upper limit outlier treatment is applied to skin thickness. Figure 12(b) shows skin thickness features after the removal of outliers.Figure 12
Handling outliers for skin thickness feature. Before (a) and after (b).
(a)(b)Figure13 shows outliers for the insulin feature. Insulin has multiple data points, which are beyond the upper limit bound of the boxplot. Upper limit outlier treatment is applied to insulin. Figure 13(b) shows insulin features after the removal of outliers.Figure 13
Handling outliers for insulin feature. Before (a) and after (b).
(a)(b)Figure14 shows outliers for the pregnancy feature. We can see that there are three data points that are out of bounds before outliers are handled. These particular data points can cause erroneous outcomes if not treated properly. Pregnancy feature is treated with upper limit outlier treatment. These data points were corrected and shown in Figure 14(b) after the removal of outliers.Figure 14
Handling outliers for the pregnancy feature. Before (a) and after (b).
(a)(b)Figure15 shows outliers for the BMI feature. We can see that there are multiple data points that are out of bound. Most of the data points are beyond the upper limit. There is only one data point that is beyond the lower limit. BMI is treated with both upper limit and lower limit outlier correction. Figure 15(b) shows BMI feature after the removal of outliers.Figure 15
Handling outliers for the BMI feature. Before (a) and after (b).
(a)(b)Figure16 shows outliers for the glucose feature. We can see that there is only one data point that is out of bound. Only glucose is treated with lower limit outlier treatment as it has data points breaching the lower limit. This data point is corrected and shown in Figure 16(b) after the removal of outliers.Figure 16
Handling outliers for glucose feature. Before (a) and after (b).
(a)(b)


## 3.2.3. Handling Class Imbalance

The final preprocessing technique was to handle class imbalance and scale the imputed data. The imbalance in the number of samples that were diabetic compared to those that did not have diabetes was 268 by 500. Data sampling techniques reduced getting biased results in predictive performance and got more data samples for learning. According to [33], a combination of synthetic minority oversampling techniques (SMOTE) and random under- or oversampling performs better than when SMOTE is used by itself. Therefore, in this research, we used a combination of random oversampling and SMOTETomek. Firstly, we used random oversampling to create more data samples and then transform them by SMOTETomek. SMOTETomek combines over- and undersampling using SMOTE and Tomek links. SMOTE randomly increases minority class examples by synthesizing new minority instances between existing minority instances. Tomek links are pair of examples that belong to different classes and are each other’s nearest neighbors. After class imbalance has been dealt with, we normalize data features using standard scaling and prepare them as input into the ANN machine learning model.


## 4. Results and Discussion

Implementing the proposed framework’s first phase generates three datasets based on each imputation method. We applied seven classification algorithms on each dataset, and the results were slightly different as each algorithm’s working criteria are different. We used 10-fold cross-validation to evaluate the training set performance. The primary evaluation metric used throughout this study is accuracy:(10)accuracy=true positives+true negativesall samples,The results of all classifiers were evaluated based on different metrics as follows:(a)
True positive (TP): subject with disease and model correctly predicts that(b)
True negative (TN): subject with no disease and model correctly predicts no disease(c)
False positive (FP): subject with no disease but model predicts incorrectly that the subject is diabetic(d)
False negative (FN): subject with the disease, but the model predicts incorrectly that the subject is not diabeticFor the evaluation of the accuracy of the classification model, the values from the confusion matrix are used. Evaluation metrics, i.e., F1-score, precision, and recall, are calculated.(a)
F1-score:(11)F1−score=2TP2TP+FP+FN.(b)
Precision:(12)precision=TPTP+FP.(c)
Sensitivity (recall):(13)recall=TPTP+FN.(d)
Specificity:(14)specificity=TNTN+FP.


### 4.1. Data Imputation Results

Various evaluation metrics are used to assess the performance of each method of imputation technique applied via 10-fold cross-validation for seven machine learning classifiers.


#### 4.1.1. Prediction Accuracy

Accuracies achieved for each method of imputation technique applied via 10-fold cross-validation using different classifiers are discussed as follows.Figure17 shows that GB is the best performing classifier. GB classifier achieves an accuracy of 91.06%, followed by the LGBM classifier, which achieves an accuracy of 90.69%. LR, KNN, DT, and RF classifiers all show good accuracy but not as good as GB. SVM performs the weakest.Figure 17
Model performance accuracy achieved for Method-I.Figure18 shows the accuracy performance measure of all the classifier algorithms applied to Method-II imputation. LR and RF classifiers have almost similar accuracies, with 78.585% and 78.588%, respectively. GB classifier also performs well with 78.03% accuracy. Like in Method-I, SVM classifier performs the weakest.Figure 18
Model performance accuracy achieved for Method-II.Figure19 shows the accuracy performance measure of all the classifier algorithms applied to Method-III imputation. LR performs the best and achieves an accuracy of 78.21%, followed by GB classifier scoring accuracy of 77.65%. KNN and DT classifiers have achieved the same accuracies of 71.89%. SVM performs the weakest in this method as well.Figure 19
Model performance accuracy achieved for Method-III.


#### 4.1.2. Precision, Recall, and F1-Score

The above figures conclude that Method-I imputation resulted in the highest accuracy. To further validate that Method-I is indeed the optimal data imputation technique among the three methods, performance was observed for other evaluation metrics as well, namely F1-score, Precision, and Recall. As our data is an imbalanced dataset, we are considering a weighted average for the scoring.Table4 describes the performance of each classification model evaluated with the help of mentioned metrics. These scores were achieved by applying 10-fold cross-validation. The precision score is similar for both GB and LGBM classifiers, whereas for recall and precision, GB has a better score compared to LGBM.Table 4
Evaluation metrics for Method-I.
ModelsPrecisionRecallSpecificityF1-scoreLR0.760.760.630.76KNN0.850.850.740.85CART0.830.830.730.83RF0.850.850.810.85SVM0.420.650.000.51LGBM0.860.860.770.86GB0.860.870.790.87Table5 describes the performance of each classification model evaluated with the help of the mentioned metrics for imputation technique Method-II. LGBM classifier has precision, recall, F1-score at 0.74 for each. This is the best, and as per this score, we can see that this model outperforms GB classifier marginally. SVM has the least scores for the individual metrics.Table 5
Evaluation metrics for Method-II.
ModelsPrecisionRecallSpecificityF1-scoreLR0.730.730.550.73KNN0.710.710.580.71CART0.670.680.510.67RF0.730.730.550.73SVM0.420.650.000.51GB0.720.710.600.72LGBM0.740.740.640.74Table6 describes the performance of each classification model evaluated, with the help of the mentioned metrics for imputation technique Method-III. LGBM and LR have precision, recall, F1-score at 0.73 for each. SVM has the least scores for the individual metrics.Table 6
Evaluation metrics for Method-III.
ModelsPrecisionRecallSpecificityF1-scoreLR0.730.730.550.73KNN0.690.690.570.69CART0.680.670.530.67RF0.720.720.540.72SVM0.420.650.000.51GB0.690.70.550.7LGBM0.730.730.620.73


#### 4.1.3. Area under ROC (Receiver Operating Characteristic) Curve

Furthermore, we also considered Receiver Operating Characteristics (ROC) as part of the performance evaluation. We know that the greater the area under the curve, the better the model. An ideal model should be having a TP rate = 1 and a FP rate = 0. So, any curve which is more aligned to theY-axis is more fit deemed to perform better.Figure20 shows the AUC that we have achieved for the Method-I imputation technique. We can see that a solid red line with “o” marker is our curve for GB classifier, strictly followed by the green dashed line, LGBM classifier.Figure 20
AUC analysis for Method-I.Figure21 shows the AUC that we have achieved for the Method-II imputation technique. The green dashed line has more area under its curve than that of other models. Thus, it indicates that the LGBM classifier has a better model performance for the Method-II imputation technique.Figure 21
AUC analysis for Method-II.Figure22 shows the AUC that we have achieved for the Method-III imputation technique. The green dashed line has more area under its curve than that of other models. Thus, it indicates that LGBM classifier has a better model performance for Method-III imputation technique.Figure 22
AUC analysis for Method-III.


#### 4.1.4. Summary

Table7 summarizes the findings of the first phase results, presenting the highest numbers of evaluation metrics achieved by each method.Table 7
Data imputation results summary.
MethodAccuracyPrecisionRecallSpecificityF1-scoreAUCModelBestModelBestModelBestModelBestModelBestModelBestILGBM0.91GB0.86GB0.87GB0.79GB0.87LGBM0.84IILR0.78LGBM0.74LGBM0.74LGBM0.65LGBM0.74LGBM0.78IIILR0.78LGBM0.73LGBM0.73LGBM0.62LGBM0.73LGBM0.70All methods result in favorable performance with more than 60% across all evaluation metrics. LGBM classifier shows up recurrently as the best performing model for all methods. Among the three imputation methods, Method-I can be seen outperforming the other two methods by a significant margin across all the evaluation metrics, while Method-II and Method-III have similar numbers in performance. The correlation matrix of the imputed data presented in Section3.2.1 offers a possible reason for this difference in performance. Method-II and Method-III imputation generated data that had a higher correlation between its variables than Method-I. Method-II and Method-III imputed data may be suffering from the problem of multicollinearity. When predictor variables are significantly dependent or associated with each other, it can hinder the model’s ability to learn the contribution of each predictor variable individually as a change in one would cause a change in the other. A possible reason behind Method-I’s high performance can thus be attributed to a low chance of multicollinearity in its imputed dataset.


### 4.2. ANN Results

The second phase consisted of implementing an ANN model on the highest performing imputed data, i.e., the Method-I imputed dataset. After handling the class imbalance and scaling the training dataset, an accuracy of 0.9817 was achieved. Upon evaluation of the test dataset, it produced an accuracy of 0.9809. Table8 presents the classification report with relevant evaluation metrics.Table 8
Classification report for ANN model.
ModelPrecisionRecallSpecificityF1-scoreANN0.980.980.990.98Precision, recall, and F1-score achieved for outcome “0” are 0.99, 0.97, and 0.98, respectively. Precision, recall, and F1-score achieved for outcome “1” are 0.97, 0.99, and 0.98, respectively. We can also see that the macro average and the weighted average for precision, recall, and F1-score are 0.98. Even though hyperparameter tuning was not conducted for this ANN model, high-performance results were achieved. In the comparison of area under ROC curves, the ANN model outperformed the other seven machine learning algorithms.Figure23 shows an AUC curve plot. The figure is for comparative study among all the classification techniques applied. ANN performs thirteen times better than GB, which had performed highest in the data imputation phase.Figure 23
AUC analysis for all models applied using Method-I imputation.Key findings of this research include the following:(i)
Method-I: missing values imputed with median values showed better results than the KNN imputation technique and the iterative imputation technique. A possible reason for this may be due to lower multicollinearity presence in Method-I imputed dataset than that of the other two methods.(ii)
SMOTETomek, along with the random sampling technique, was applied to the dataset created by the Method-I imputation technique. This combined method for treating imbalanced data was better than using one technique individually.(iii)
ANN was implemented after balancing the dataset, and it produced an accuracy of 0.9809. This is by far the best accuracy achieved in terms of classification compared to past studies done with the same dataset.Apart from this, some interesting facts about the attributes’ relationship with type 2 diabetes were found because of data mining, as follows:(i)
As per the correlation matrix shown in Section3, BMI is highly correlated with diabetic patients. So, it can be concluded that obesity has a high chance of causing type 2 diabetes.(ii)
Insulin resistance has been identified as a major risk factor in the pathogenesis of type 2 diabetes and has been thoroughly studied in the Arizona Pima. Insulin resistance appears to increase incrementally according to BMI levels primarily and body fat levels secondarily. As shown in the data, that obesity is causing high BMI, which is leading to chances of type 2 diabetes. Further, DPF, which indicates diabetes history in relatives and genetic relationship, shows high chances of diabetes.


## 4.1. Data Imputation Results

Various evaluation metrics are used to assess the performance of each method of imputation technique applied via 10-fold cross-validation for seven machine learning classifiers.


### 4.1.1. Prediction Accuracy

Accuracies achieved for each method of imputation technique applied via 10-fold cross-validation using different classifiers are discussed as follows.Figure17 shows that GB is the best performing classifier. GB classifier achieves an accuracy of 91.06%, followed by the LGBM classifier, which achieves an accuracy of 90.69%. LR, KNN, DT, and RF classifiers all show good accuracy but not as good as GB. SVM performs the weakest.Figure 17
Model performance accuracy achieved for Method-I.Figure18 shows the accuracy performance measure of all the classifier algorithms applied to Method-II imputation. LR and RF classifiers have almost similar accuracies, with 78.585% and 78.588%, respectively. GB classifier also performs well with 78.03% accuracy. Like in Method-I, SVM classifier performs the weakest.Figure 18
Model performance accuracy achieved for Method-II.Figure19 shows the accuracy performance measure of all the classifier algorithms applied to Method-III imputation. LR performs the best and achieves an accuracy of 78.21%, followed by GB classifier scoring accuracy of 77.65%. KNN and DT classifiers have achieved the same accuracies of 71.89%. SVM performs the weakest in this method as well.Figure 19
Model performance accuracy achieved for Method-III.


### 4.1.2. Precision, Recall, and F1-Score

The above figures conclude that Method-I imputation resulted in the highest accuracy. To further validate that Method-I is indeed the optimal data imputation technique among the three methods, performance was observed for other evaluation metrics as well, namely F1-score, Precision, and Recall. As our data is an imbalanced dataset, we are considering a weighted average for the scoring.Table4 describes the performance of each classification model evaluated with the help of mentioned metrics. These scores were achieved by applying 10-fold cross-validation. The precision score is similar for both GB and LGBM classifiers, whereas for recall and precision, GB has a better score compared to LGBM.Table 4
Evaluation metrics for Method-I.
ModelsPrecisionRecallSpecificityF1-scoreLR0.760.760.630.76KNN0.850.850.740.85CART0.830.830.730.83RF0.850.850.810.85SVM0.420.650.000.51LGBM0.860.860.770.86GB0.860.870.790.87Table5 describes the performance of each classification model evaluated with the help of the mentioned metrics for imputation technique Method-II. LGBM classifier has precision, recall, F1-score at 0.74 for each. This is the best, and as per this score, we can see that this model outperforms GB classifier marginally. SVM has the least scores for the individual metrics.Table 5
Evaluation metrics for Method-II.
ModelsPrecisionRecallSpecificityF1-scoreLR0.730.730.550.73KNN0.710.710.580.71CART0.670.680.510.67RF0.730.730.550.73SVM0.420.650.000.51GB0.720.710.600.72LGBM0.740.740.640.74Table6 describes the performance of each classification model evaluated, with the help of the mentioned metrics for imputation technique Method-III. LGBM and LR have precision, recall, F1-score at 0.73 for each. SVM has the least scores for the individual metrics.Table 6
Evaluation metrics for Method-III.
ModelsPrecisionRecallSpecificityF1-scoreLR0.730.730.550.73KNN0.690.690.570.69CART0.680.670.530.67RF0.720.720.540.72SVM0.420.650.000.51GB0.690.70.550.7LGBM0.730.730.620.73


### 4.1.3. Area under ROC (Receiver Operating Characteristic) Curve

Furthermore, we also considered Receiver Operating Characteristics (ROC) as part of the performance evaluation. We know that the greater the area under the curve, the better the model. An ideal model should be having a TP rate = 1 and a FP rate = 0. So, any curve which is more aligned to theY-axis is more fit deemed to perform better.Figure20 shows the AUC that we have achieved for the Method-I imputation technique. We can see that a solid red line with “o” marker is our curve for GB classifier, strictly followed by the green dashed line, LGBM classifier.Figure 20
AUC analysis for Method-I.Figure21 shows the AUC that we have achieved for the Method-II imputation technique. The green dashed line has more area under its curve than that of other models. Thus, it indicates that the LGBM classifier has a better model performance for the Method-II imputation technique.Figure 21
AUC analysis for Method-II.Figure22 shows the AUC that we have achieved for the Method-III imputation technique. The green dashed line has more area under its curve than that of other models. Thus, it indicates that LGBM classifier has a better model performance for Method-III imputation technique.Figure 22
AUC analysis for Method-III.


### 4.1.4. Summary

Table7 summarizes the findings of the first phase results, presenting the highest numbers of evaluation metrics achieved by each method.Table 7
Data imputation results summary.
MethodAccuracyPrecisionRecallSpecificityF1-scoreAUCModelBestModelBestModelBestModelBestModelBestModelBestILGBM0.91GB0.86GB0.87GB0.79GB0.87LGBM0.84IILR0.78LGBM0.74LGBM0.74LGBM0.65LGBM0.74LGBM0.78IIILR0.78LGBM0.73LGBM0.73LGBM0.62LGBM0.73LGBM0.70All methods result in favorable performance with more than 60% across all evaluation metrics. LGBM classifier shows up recurrently as the best performing model for all methods. Among the three imputation methods, Method-I can be seen outperforming the other two methods by a significant margin across all the evaluation metrics, while Method-II and Method-III have similar numbers in performance. The correlation matrix of the imputed data presented in Section3.2.1 offers a possible reason for this difference in performance. Method-II and Method-III imputation generated data that had a higher correlation between its variables than Method-I. Method-II and Method-III imputed data may be suffering from the problem of multicollinearity. When predictor variables are significantly dependent or associated with each other, it can hinder the model’s ability to learn the contribution of each predictor variable individually as a change in one would cause a change in the other. A possible reason behind Method-I’s high performance can thus be attributed to a low chance of multicollinearity in its imputed dataset.


## 4.1.1. Prediction Accuracy

Accuracies achieved for each method of imputation technique applied via 10-fold cross-validation using different classifiers are discussed as follows.Figure17 shows that GB is the best performing classifier. GB classifier achieves an accuracy of 91.06%, followed by the LGBM classifier, which achieves an accuracy of 90.69%. LR, KNN, DT, and RF classifiers all show good accuracy but not as good as GB. SVM performs the weakest.Figure 17
Model performance accuracy achieved for Method-I.Figure18 shows the accuracy performance measure of all the classifier algorithms applied to Method-II imputation. LR and RF classifiers have almost similar accuracies, with 78.585% and 78.588%, respectively. GB classifier also performs well with 78.03% accuracy. Like in Method-I, SVM classifier performs the weakest.Figure 18
Model performance accuracy achieved for Method-II.Figure19 shows the accuracy performance measure of all the classifier algorithms applied to Method-III imputation. LR performs the best and achieves an accuracy of 78.21%, followed by GB classifier scoring accuracy of 77.65%. KNN and DT classifiers have achieved the same accuracies of 71.89%. SVM performs the weakest in this method as well.Figure 19
Model performance accuracy achieved for Method-III.


## 4.1.2. Precision, Recall, and F1-Score

The above figures conclude that Method-I imputation resulted in the highest accuracy. To further validate that Method-I is indeed the optimal data imputation technique among the three methods, performance was observed for other evaluation metrics as well, namely F1-score, Precision, and Recall. As our data is an imbalanced dataset, we are considering a weighted average for the scoring.Table4 describes the performance of each classification model evaluated with the help of mentioned metrics. These scores were achieved by applying 10-fold cross-validation. The precision score is similar for both GB and LGBM classifiers, whereas for recall and precision, GB has a better score compared to LGBM.Table 4
Evaluation metrics for Method-I.
ModelsPrecisionRecallSpecificityF1-scoreLR0.760.760.630.76KNN0.850.850.740.85CART0.830.830.730.83RF0.850.850.810.85SVM0.420.650.000.51LGBM0.860.860.770.86GB0.860.870.790.87Table5 describes the performance of each classification model evaluated with the help of the mentioned metrics for imputation technique Method-II. LGBM classifier has precision, recall, F1-score at 0.74 for each. This is the best, and as per this score, we can see that this model outperforms GB classifier marginally. SVM has the least scores for the individual metrics.Table 5
Evaluation metrics for Method-II.
ModelsPrecisionRecallSpecificityF1-scoreLR0.730.730.550.73KNN0.710.710.580.71CART0.670.680.510.67RF0.730.730.550.73SVM0.420.650.000.51GB0.720.710.600.72LGBM0.740.740.640.74Table6 describes the performance of each classification model evaluated, with the help of the mentioned metrics for imputation technique Method-III. LGBM and LR have precision, recall, F1-score at 0.73 for each. SVM has the least scores for the individual metrics.Table 6
Evaluation metrics for Method-III.
ModelsPrecisionRecallSpecificityF1-scoreLR0.730.730.550.73KNN0.690.690.570.69CART0.680.670.530.67RF0.720.720.540.72SVM0.420.650.000.51GB0.690.70.550.7LGBM0.730.730.620.73


## 4.1.3. Area under ROC (Receiver Operating Characteristic) Curve

Furthermore, we also considered Receiver Operating Characteristics (ROC) as part of the performance evaluation. We know that the greater the area under the curve, the better the model. An ideal model should be having a TP rate = 1 and a FP rate = 0. So, any curve which is more aligned to theY-axis is more fit deemed to perform better.Figure20 shows the AUC that we have achieved for the Method-I imputation technique. We can see that a solid red line with “o” marker is our curve for GB classifier, strictly followed by the green dashed line, LGBM classifier.Figure 20
AUC analysis for Method-I.Figure21 shows the AUC that we have achieved for the Method-II imputation technique. The green dashed line has more area under its curve than that of other models. Thus, it indicates that the LGBM classifier has a better model performance for the Method-II imputation technique.Figure 21
AUC analysis for Method-II.Figure22 shows the AUC that we have achieved for the Method-III imputation technique. The green dashed line has more area under its curve than that of other models. Thus, it indicates that LGBM classifier has a better model performance for Method-III imputation technique.Figure 22
AUC analysis for Method-III.


## 4.1.4. Summary

Table7 summarizes the findings of the first phase results, presenting the highest numbers of evaluation metrics achieved by each method.Table 7
Data imputation results summary.
MethodAccuracyPrecisionRecallSpecificityF1-scoreAUCModelBestModelBestModelBestModelBestModelBestModelBestILGBM0.91GB0.86GB0.87GB0.79GB0.87LGBM0.84IILR0.78LGBM0.74LGBM0.74LGBM0.65LGBM0.74LGBM0.78IIILR0.78LGBM0.73LGBM0.73LGBM0.62LGBM0.73LGBM0.70All methods result in favorable performance with more than 60% across all evaluation metrics. LGBM classifier shows up recurrently as the best performing model for all methods. Among the three imputation methods, Method-I can be seen outperforming the other two methods by a significant margin across all the evaluation metrics, while Method-II and Method-III have similar numbers in performance. The correlation matrix of the imputed data presented in Section3.2.1 offers a possible reason for this difference in performance. Method-II and Method-III imputation generated data that had a higher correlation between its variables than Method-I. Method-II and Method-III imputed data may be suffering from the problem of multicollinearity. When predictor variables are significantly dependent or associated with each other, it can hinder the model’s ability to learn the contribution of each predictor variable individually as a change in one would cause a change in the other. A possible reason behind Method-I’s high performance can thus be attributed to a low chance of multicollinearity in its imputed dataset.


## 4.2. ANN Results

The second phase consisted of implementing an ANN model on the highest performing imputed data, i.e., the Method-I imputed dataset. After handling the class imbalance and scaling the training dataset, an accuracy of 0.9817 was achieved. Upon evaluation of the test dataset, it produced an accuracy of 0.9809. Table8 presents the classification report with relevant evaluation metrics.Table 8
Classification report for ANN model.
ModelPrecisionRecallSpecificityF1-scoreANN0.980.980.990.98Precision, recall, and F1-score achieved for outcome “0” are 0.99, 0.97, and 0.98, respectively. Precision, recall, and F1-score achieved for outcome “1” are 0.97, 0.99, and 0.98, respectively. We can also see that the macro average and the weighted average for precision, recall, and F1-score are 0.98. Even though hyperparameter tuning was not conducted for this ANN model, high-performance results were achieved. In the comparison of area under ROC curves, the ANN model outperformed the other seven machine learning algorithms.Figure23 shows an AUC curve plot. The figure is for comparative study among all the classification techniques applied. ANN performs thirteen times better than GB, which had performed highest in the data imputation phase.Figure 23
AUC analysis for all models applied using Method-I imputation.Key findings of this research include the following:(i)
Method-I: missing values imputed with median values showed better results than the KNN imputation technique and the iterative imputation technique. A possible reason for this may be due to lower multicollinearity presence in Method-I imputed dataset than that of the other two methods.(ii)
SMOTETomek, along with the random sampling technique, was applied to the dataset created by the Method-I imputation technique. This combined method for treating imbalanced data was better than using one technique individually.(iii)
ANN was implemented after balancing the dataset, and it produced an accuracy of 0.9809. This is by far the best accuracy achieved in terms of classification compared to past studies done with the same dataset.Apart from this, some interesting facts about the attributes’ relationship with type 2 diabetes were found because of data mining, as follows:(i)
As per the correlation matrix shown in Section3, BMI is highly correlated with diabetic patients. So, it can be concluded that obesity has a high chance of causing type 2 diabetes.(ii)
Insulin resistance has been identified as a major risk factor in the pathogenesis of type 2 diabetes and has been thoroughly studied in the Arizona Pima. Insulin resistance appears to increase incrementally according to BMI levels primarily and body fat levels secondarily. As shown in the data, that obesity is causing high BMI, which is leading to chances of type 2 diabetes. Further, DPF, which indicates diabetes history in relatives and genetic relationship, shows high chances of diabetes.


## 5. Conclusion

With advancements in technology and computing power, intelligent machine systems can be utilized to provide a more efficient alternative to traditional manual analysis for medical decision support. The current solutions include classification models that are highly inaccurate, but they do not consider some essential data preprocessing steps that can positively impact overall accuracy achieved. This research adopted a framework whereby the first phase focused on identifying the optimal handling of missing data. The application of three different data imputation techniques was investigated and observed for its impact on classification model performance. In the second phase, the best performing imputation data were balanced using a combination of SMOTETomek and random oversampling and modeled using ANN.There is a certain assumption that we have considered during data preprocessing. The upper boundary and lower boundary set for treating outliers were based on generic assumptions. ANN performs well with a bigger dataset. But for our case, the dataset was small. Thus, random oversampling applied to imputed data for building ANN is assumed to be close to real data. The number of nodes for starting and hidden layers of ANN is based on multiple trials, and the current network is assumed to work the best. Eight nodes were assumed for starting layer as the target variable is dependent on 8 variables. The next hidden layer had double the number of starting nodes, 16. For the consecutive following hidden layers, every time, the number of nodes was reduced down by 4. The final layer of the network had 1 node as a standard process.There are several limitations to this study. The data used in this study are of a specific region, restricted to the population of Native Americans. All patients considered in this study are females of age 21 and above. Future works should include data across different geographic regions and of various origins. It would be interesting to see how the model behaves with a mixed group of males and females. From a modeling point of view, hyperparameter optimization of ANN can also be adapted to study the outcome/accuracy.



---

*Source: 9953314-2021-07-06.xml*