# Development of Hepatitis Disease Detection System by Exploiting Sparsity in Linear Support Vector Machine to Improve Strength of AdaBoost Ensemble Model

Authors: Wasif Akbar; Wei-ping Wu; Sehrish Saleem; Muhammad Farhan; Muhammad Asim Saleem; Ashir Javeed; Liaqat Ali
Journal: Mobile Information Systems
 (2020)

---

## Abstract

Hepatitis disease is a deadliest disease. The management and diagnosis of hepatitis disease is expensive and requires high level of human expertise which poses challenges for the health care system in underdeveloped and developing countries. Hence, development of automated methods for accurate prediction of hepatitis disease is inevitable. In this paper, we develop a diagnostic system which hybridizes a linear support vector machine (SVM) model with adaptive boosting (AdaBoost) model. We exploit sparsity in linear SVM that is caused byL1 regularization. The sparse L1-regularized SVM is capable of eliminating redundant or irrelevant features from feature space. After filtering features through the sparse linear SVM, the output of the SVM is applied to the AdaBoost ensemble model which is used for classification purposes. Two types of numerical experiments are performed on the clinical features of hepatitis disease collected from UCI machine learning repository. In the first experiment, only conventional AdaBoost model is used, while in the second experiment, a feature vector is applied to the sparse linear SVM before its application to the AdaBoost model. Simulation results demonstrate that the strength of a conventional AdaBoost model is enhanced by 6.39% by the proposed method, and its time complexity is also reduced. In addition, the proposed method shows better performance than many previously developed methods for hepatitis disease prediction.


---

## Body

## 1. Introduction

Hepatitis is considered a major chronic liver disease worldwide. The liver is considered to be the heaviest and one of the largest organs of the human body [1]. The liver is one of the key organs of a human body responsible for different functions. These functions include bile secretion, protein formation, and elimination of toxins from body. Hence, inflammation of liver (caused by hepatitis) results in dysfunction of the liver, and consequently, the health of the subject is deteriorated. The symptoms of hepatitis are different in different patients, with some subjects showing no signs. Well-known symptoms include yellowish eyes and skin, abdominal pain, poor appetite, and tiredness [2, 3]. Hepatitis can be acute or chronic depending on duration. If it lasts for less than six months, it is acute; however, if it lasts for more than six months, it is chronic [4]. It has been reported that hepatitis results in more than a million deaths each year. Diagnosis of hepatitis through conventional methods is a difficult job and requires expensive medical tests [5]. Additionally, the diagnosis of such disease through intelligent system reduces the cost and also examines the patient in shorter time. Hence, development of intelligent diagnostic systems for such type of disease prediction is very important.In the past, numerous hybrid models for disease detection have been developed by different researchers. These include automated systems for Parkinson’s disease prediction [6–8], mortality prediction [9, 10], cancer detection [11, 12], and heart disease [6, 13, 14]. These models are developed by hybridizing data mining models (for feature preprocessing) such as principal component analysis (PCA) and Fisher discriminant analysis (FDA) with machine learning models such as decision trees, logistic regression, support vector machine (SVM), Naive Bayes, neural network models, ensembles of neural networks, K-nearest neighbors, deep neural networks, and optimized and stacked SVMs [15–24]. For example, Adamczak developed different automated models for hepatitis prediction. These models include MLP + BP, RBF (Tooldiag), and FSM without rotation and achieved a prediction accuracy of 77.4%, 79%, and 88.5%, respectively [25]. In another study conducted by Passi, MLO was developed for hepatitis which resulted in hepatitis prediction of 79.70% [26, 27]. Stern and Dobnikar developed AIS, LDA, and FDA models which achieved the hepatitis prediction accuracy of 82%, 84.5%, and 86.40%, respectively [27]. Nilashi et al. developed KNN, ANFIS, NN, and SVM and achieved hepatitis prediction accuracy of 71.41%, 79.67%, 78.31%, and 81.17%, respectively [28]. Recently, Polat and Gunes discussed the hybridization of the feature extraction through the principal component analysis model with classification through artificial immune recognition system for the prediction of hepatitis disease [1, 29].In this paper, we develop a hybrid intelligent diagnostic system. To improve the strength of AdaBoost predictive model, we propose to useL1-penalized linear SVM. The L1 penalty makes the linear SVM sparse, thus making it capable of eliminating redundant features by making their coefficients zero through sparse solutions. After elimination of redundant features through the sparse linear SVM, the remaining features are supplied to the AdaBoost model for classification. In order to analyze the impact of the sparse linear SVM on the AdaBoost model, we performed two types of numerical experiments. In the first experiment, we developed the conventional AdaBoost model, while in the second experiment, we constructed a learning system by stacking the sparse SVM with the AdaBoost model. The performance of both the models, developed in the two experiments, was evaluated using an online hepatitis disease data. Experimental results demonstrated that the sparse linear SVM enhances the accuracy of conventional AdaBoost (for the hepatitis disease prediction based on the collected clinical features). Additionally, the sparse linear SVM also reduces AdaBoost model’s complexity as the optimal subset of features contains less number of features.The rest of the manuscript is organized as follows. Datasets, the proposed sparse linear SVM, and AdaBoost-based learning system are elaborated in Section2. Section 3 discusses various schemes for validation as well as multiple metrics for evaluation used in the manuscript. Section 4 discusses experimental setup and obtained results, whereas the last section concludes the paper.


## 2. Materials and Methods

### 2.1. Dataset Description

The hepatitis dataset consists of 155 samples, and each sample contains 19 features. Details about the 19 commonly used features for the hepatitis dataset are given in Table1. The label of the dataset is binary, i.e., it can have a value of 1 or 2, where 1 means the sample belongs to a patient who died, while 2 means the sample is that of a subject who survived. There are 32 samples having label 1 and 123 samples having the label value of 2, i.e., the dataset contains 123 samples belonging to healthy class and 32 samples belonging to patient class. In machine learning, we split the data into two parts, namely, training and testing. The training part is used to train the model, and its performance is checked by testing the trained model on the testing data. In this study, the dataset is divided into training and testing datasets using 70–30 data portioning. Hence, out of the 155 samples, 108 samples are used for training purposes, and the remaining 47 samples are used for testing purposes. Out of the 108 training samples, 23 samples belong to the patient class, and 85 patients belong to healthy class. On the other hand, out of the 47 testing samples, 7 samples belong to the patient group, and 38 samples belong to the healthy group. It can be noticed that lower class distribution of the patient class is a limitation of the dataset.Table 1
Details of the 19 hepatitis features.
Feature noFeature codeFeature descriptionValues1D1Age10, 20, 30, … , 70, 802D2SexMale, female3D3Steroid1, 24D4Antivirals1, 25D5Fatigue1, 26D6Malaise1, 27D7Anorexia1, 28D8Liver big1, 29D9Liver firm1, 210D10Spleen palpable1, 211D11Spiders1, 212D12Ascites1, 213D13Varices1, 214D14Bilirubin0.39, 0.8, 1.2, 2.0, 3.0, 4.015D15Alkaline phosphatase33, 80, 120, 160, 200, 25016D16SGOT100, 200, 300, 400, 50017D17Albumin2.1, 3.0, 3.8, 4.5, 5.0, 6.018D18Protime10, 20, 30, 40, … , 80, 9019D19Histology1, 2


### 2.2. Proposed Method

As discussed above, in this paper, we exploit the sparsity in linear SVM to improve the strength of machine learning models, namely,k-nearest neighbours (KNN), Gaussian Naive Bayes (GNB), linear discriminant analysis (LDA), and AdaBoost ensemble model. Initially, L1-penalized linear SVM is used to generate sparse features, i.e., to process the full set of features, null the redundant features, and yield a subset of features containing relevant features only. The generated subset of features by sparse linear SVM is supplied to machine learning models for classification purposes. The sparsity of the linear SVM is controlled by its hyperparameter λ. Hence, for distinct values of λ, various distinct features will be nullified resulting in different subsets of features. Thus, for achieving better hepatitis prediction accuracies, it is necessary to develop a sparse linear SVM that would nullify the most redundant or irrelevant features and generate a subset of the most relevant features. This can be accomplished by tuning the hyperparameter λ. In order to better comprehend the functioning of the proposed learning system, it is pertinent to briefly discuss the L1-penalized linear SVM model and its formulation. The formulation is as follows.Support vector machines (SVMs) are considered powerful learning methods and have been widely used in different biomedical- and health informatics-related problems [30]. During the training process, SVM tries to construct an optimal hyperplane that can better differentiate the data points of the two classes (in case of binary classification) [31]. The major reason that motivates machine learning researchers to use SVM for their problems is that SVMs have powerful generalization capabilities to unseen data and they depend on very small number of hyperparameters [32].Considering a datasetDS with S instances D=pi,qi|pi∈RQ,qi∈−1,1i=1S, where pi stands for ith instance, Q represents the dimension of the original feature space of hepatitis data, and qi denotes the class labels, i.e., presence or absence of hepatitis disease. The value is 19 for the hepatitis dataset considered in this paper. The SVM model determines a hyperplane calculated by gx=βT∗x+δ, where δ represents the bias and β denotes the weight vector. Based on the training data, the hyperplane gx of SVM augments the margin, whereas it curtails the classification error [33]. The sum of the distances between the closest negative and closest positive instances is called margin. In other words, the hyperplane augments the margin distance 2/β22.SVM uses a set of slack variables denoted byθi, i=1,….,S and a penalty parameter, i.e., λ, and attempts to maximize β22 and minimize the errors of misclassification [34]. This fact is formulated as follows:(1)minβ,δ,θ12β22︸Regularizer+λ∑i=1Sθi︸Error loss,subject to yiβxi+δ≥1−ξiξi≥0,i=1,…,S, where θ is the slack variable that calibrates the degree of misclassification and Euclidean norm or L2-norm is the penalty term. A varied version of SVM was introduced by Bradley and Mangasarian which replaces the Euclidean norm, i.e., L2-norm with L1-penalty function [35]. The L1-penalized SVM produces sparse solutions and has the feature selection property due to its competence of overthrowing irrelevant or noisy features automatically and hence can be used for feature selection. The formulation of L1-penalized SVM is given as follows:(2)minβ,β,ξβ1︸Regularizer+λ∑i=1Sθi︸Error loss,subject toyiβxi+δ≥1−θiθi≥0,i=1,…,S.From the above formulas, it can be seen that, for different settings of the hyperparameter of theL1 SVM, i.e., λ, different features will be nulled; consequently, a different subset of features will be produced [36]. The goal is to tune the value of λ in such a way to produce a subset of features which will show best performance in terms of hepatitis disease prediction accuracies. This is done by using exhaustive search methodology. After production of the features’ subset, its application to AdaBoost machine learning models is carried out. The AdaBoost model is used for classification task.AdaBoost (also known as adaptive boosting classifier) is an ensemble learning model. It utilizes boosting approach to construct a metaclassifier by combining the strengths of base classifiers, i.e., weak estimators. The boosting operation helps convert the weak estimators into a stronger or boosted model. During the process of boosting, weighted sum of the base learners or estimators is evaluated to produce the final output of the boosted model. This fact is reflected in the following formulation:(3)Gx=sign∑m=1MαmBmx,where the mth base classifier is denoted by Bm and αm denotes the weight of the mth classifier or estimator. To implement the AdaBoost model, we used scikit-learn python API [37]. In the following discussion, E denotes the total number of classifiers or estimators used for constructing the eventual AdaBoost model.The primary objective of this paper is to investigate and exploit the sparsity in the linearL1-regularized SVM to further improve the strength of the AdaBoost model. To meet this objective, we develop a cascade of the L1 linear sparse SVM and AdaBoost model. The full feature set is supplied at the input of L1 SVM which produces different subset of features based on the value of its hyperparameter λ. Performance of the subset of features is evaluated by their application to AdaBoost model. Thus, in the initial stages, we need to discretize the λ hyperparameter. After discretization of λ, we will have to search the optimal value of λ that will produce optimal subset of features which will show best classification performance. The whole process of the proposed method is shown in the Figure 1. From the figure, it can be seen that initially, a subset of features is generated by utilizing a specific value of λ. The subset of features is given to the AdaBoost model which is trained using one value of E. For the subset of features, performance is evaluated under optimal E. Furthermore, another subset of features is generated by utilizing another discrete value of λ, and again the AdaBoost model is trained and evaluated under optimal value of E. The process is repeated until all the subset of features are evaluated and tested. At the end, the optimal subset of features is selected based on the performance.Figure 1
Block diagram of the proposed diagnostic system.E: number of estimators used by the AdaBoost model, N: size of subset of features, and λ: hyperparameter of the L1 SVM model that controls the sparsity.


## 2.1. Dataset Description

The hepatitis dataset consists of 155 samples, and each sample contains 19 features. Details about the 19 commonly used features for the hepatitis dataset are given in Table1. The label of the dataset is binary, i.e., it can have a value of 1 or 2, where 1 means the sample belongs to a patient who died, while 2 means the sample is that of a subject who survived. There are 32 samples having label 1 and 123 samples having the label value of 2, i.e., the dataset contains 123 samples belonging to healthy class and 32 samples belonging to patient class. In machine learning, we split the data into two parts, namely, training and testing. The training part is used to train the model, and its performance is checked by testing the trained model on the testing data. In this study, the dataset is divided into training and testing datasets using 70–30 data portioning. Hence, out of the 155 samples, 108 samples are used for training purposes, and the remaining 47 samples are used for testing purposes. Out of the 108 training samples, 23 samples belong to the patient class, and 85 patients belong to healthy class. On the other hand, out of the 47 testing samples, 7 samples belong to the patient group, and 38 samples belong to the healthy group. It can be noticed that lower class distribution of the patient class is a limitation of the dataset.Table 1
Details of the 19 hepatitis features.
Feature noFeature codeFeature descriptionValues1D1Age10, 20, 30, … , 70, 802D2SexMale, female3D3Steroid1, 24D4Antivirals1, 25D5Fatigue1, 26D6Malaise1, 27D7Anorexia1, 28D8Liver big1, 29D9Liver firm1, 210D10Spleen palpable1, 211D11Spiders1, 212D12Ascites1, 213D13Varices1, 214D14Bilirubin0.39, 0.8, 1.2, 2.0, 3.0, 4.015D15Alkaline phosphatase33, 80, 120, 160, 200, 25016D16SGOT100, 200, 300, 400, 50017D17Albumin2.1, 3.0, 3.8, 4.5, 5.0, 6.018D18Protime10, 20, 30, 40, … , 80, 9019D19Histology1, 2


## 2.2. Proposed Method

As discussed above, in this paper, we exploit the sparsity in linear SVM to improve the strength of machine learning models, namely,k-nearest neighbours (KNN), Gaussian Naive Bayes (GNB), linear discriminant analysis (LDA), and AdaBoost ensemble model. Initially, L1-penalized linear SVM is used to generate sparse features, i.e., to process the full set of features, null the redundant features, and yield a subset of features containing relevant features only. The generated subset of features by sparse linear SVM is supplied to machine learning models for classification purposes. The sparsity of the linear SVM is controlled by its hyperparameter λ. Hence, for distinct values of λ, various distinct features will be nullified resulting in different subsets of features. Thus, for achieving better hepatitis prediction accuracies, it is necessary to develop a sparse linear SVM that would nullify the most redundant or irrelevant features and generate a subset of the most relevant features. This can be accomplished by tuning the hyperparameter λ. In order to better comprehend the functioning of the proposed learning system, it is pertinent to briefly discuss the L1-penalized linear SVM model and its formulation. The formulation is as follows.Support vector machines (SVMs) are considered powerful learning methods and have been widely used in different biomedical- and health informatics-related problems [30]. During the training process, SVM tries to construct an optimal hyperplane that can better differentiate the data points of the two classes (in case of binary classification) [31]. The major reason that motivates machine learning researchers to use SVM for their problems is that SVMs have powerful generalization capabilities to unseen data and they depend on very small number of hyperparameters [32].Considering a datasetDS with S instances D=pi,qi|pi∈RQ,qi∈−1,1i=1S, where pi stands for ith instance, Q represents the dimension of the original feature space of hepatitis data, and qi denotes the class labels, i.e., presence or absence of hepatitis disease. The value is 19 for the hepatitis dataset considered in this paper. The SVM model determines a hyperplane calculated by gx=βT∗x+δ, where δ represents the bias and β denotes the weight vector. Based on the training data, the hyperplane gx of SVM augments the margin, whereas it curtails the classification error [33]. The sum of the distances between the closest negative and closest positive instances is called margin. In other words, the hyperplane augments the margin distance 2/β22.SVM uses a set of slack variables denoted byθi, i=1,….,S and a penalty parameter, i.e., λ, and attempts to maximize β22 and minimize the errors of misclassification [34]. This fact is formulated as follows:(1)minβ,δ,θ12β22︸Regularizer+λ∑i=1Sθi︸Error loss,subject to yiβxi+δ≥1−ξiξi≥0,i=1,…,S, where θ is the slack variable that calibrates the degree of misclassification and Euclidean norm or L2-norm is the penalty term. A varied version of SVM was introduced by Bradley and Mangasarian which replaces the Euclidean norm, i.e., L2-norm with L1-penalty function [35]. The L1-penalized SVM produces sparse solutions and has the feature selection property due to its competence of overthrowing irrelevant or noisy features automatically and hence can be used for feature selection. The formulation of L1-penalized SVM is given as follows:(2)minβ,β,ξβ1︸Regularizer+λ∑i=1Sθi︸Error loss,subject toyiβxi+δ≥1−θiθi≥0,i=1,…,S.From the above formulas, it can be seen that, for different settings of the hyperparameter of theL1 SVM, i.e., λ, different features will be nulled; consequently, a different subset of features will be produced [36]. The goal is to tune the value of λ in such a way to produce a subset of features which will show best performance in terms of hepatitis disease prediction accuracies. This is done by using exhaustive search methodology. After production of the features’ subset, its application to AdaBoost machine learning models is carried out. The AdaBoost model is used for classification task.AdaBoost (also known as adaptive boosting classifier) is an ensemble learning model. It utilizes boosting approach to construct a metaclassifier by combining the strengths of base classifiers, i.e., weak estimators. The boosting operation helps convert the weak estimators into a stronger or boosted model. During the process of boosting, weighted sum of the base learners or estimators is evaluated to produce the final output of the boosted model. This fact is reflected in the following formulation:(3)Gx=sign∑m=1MαmBmx,where the mth base classifier is denoted by Bm and αm denotes the weight of the mth classifier or estimator. To implement the AdaBoost model, we used scikit-learn python API [37]. In the following discussion, E denotes the total number of classifiers or estimators used for constructing the eventual AdaBoost model.The primary objective of this paper is to investigate and exploit the sparsity in the linearL1-regularized SVM to further improve the strength of the AdaBoost model. To meet this objective, we develop a cascade of the L1 linear sparse SVM and AdaBoost model. The full feature set is supplied at the input of L1 SVM which produces different subset of features based on the value of its hyperparameter λ. Performance of the subset of features is evaluated by their application to AdaBoost model. Thus, in the initial stages, we need to discretize the λ hyperparameter. After discretization of λ, we will have to search the optimal value of λ that will produce optimal subset of features which will show best classification performance. The whole process of the proposed method is shown in the Figure 1. From the figure, it can be seen that initially, a subset of features is generated by utilizing a specific value of λ. The subset of features is given to the AdaBoost model which is trained using one value of E. For the subset of features, performance is evaluated under optimal E. Furthermore, another subset of features is generated by utilizing another discrete value of λ, and again the AdaBoost model is trained and evaluated under optimal value of E. The process is repeated until all the subset of features are evaluated and tested. At the end, the optimal subset of features is selected based on the performance.Figure 1
Block diagram of the proposed diagnostic system.E: number of estimators used by the AdaBoost model, N: size of subset of features, and λ: hyperparameter of the L1 SVM model that controls the sparsity.


## 3. Evaluation of the Proposed Method

In literature, different researchers have utilized various metrics for performance evaluation of their proposed methods. However, for a more realistic evaluation of the performance of our proposed method, we utilized the following five evaluation metrics known as accuracy (ACC), specificity (Spec.), sensitivity (Sen.), and Matthews correlation coefficient (MCC). Accuracy gives information about the total number of correctly classified subjects (whether healthy or patients). Specificity conveys information about the number of healthy subjects which are classified correctly. Similarly, sensitivity represents the percentage of subjects which are classified correctly. MCC is used to measure the quality of binary classification. The basic formulas for these metrics are given as follows:(4)ACC=tp+tntp+fp+tn+fn,Sen=tptp+fn,Spec=tntn+fp,MCC=tp×tn−fp×fntp+fptp+fntn+fptn+fn.


## 4. Results and Discussion

In this section, the experimental setting and the obtained results are analyzed and discussed. All the experiments (including conventional machine learning-based experiments and the proposed method-based experiments) are performed using Python software (scikit-learn). The experiments were simulated using Intel Core i5 processor with 8 GB RAM and 64-bit operating systems. For the purpose of comparison, we performed two types of experiments. First, the conventional AdaBoost model is developed for the prediction of hepatitis disease. Second, the proposed hybrid model is developed to predict hepatitis disease based on the filtered set of features.


### 4.1. Simulation of Conventional AdaBoost Model on Hepatitis Data

In this experiment, we develop the conventional AdaBoost model for the hepatitis disease data. The model is trained using 70% of the dataset and tested on the remaining 30% of the data. An exhaustive grid search algorithm is used to search the optimized version of the AdaBoost model. The results on both optimal hyperparameters and nonoptimal hyperparameters are given in Table2. It is evident from the table that best performance of 82.97% accuracy, 11.11% sensitivity, 100% specificity, and MCC of 0.302 is obtained at optimal hyperparameter, i.e., E=3.Table 2
Performance of the conventional AdaBoost model on HF data.
EAcctestAcctrain%Sens. (%)Spec. (%)MCC382.9785.1811.11100.00.3021076.5993.5111.1192.100.0451274.4696.2911.1189.470.0071474.4697.2211.1189.470.007Bold values indicate optimal performance.


### 4.2. Simulation of the Proposed Method Using the Sparse Linear SVM and AdaBoost Model on Hepatitis Data

In this experiment, the proposed learning system is developed by using both the models, i.e., sparse linear SVM and AdaBoost model. The simulation results are reported in Table3. As can be seen in the table, different values of λ for the sparse SVM generate different subsets of features with different sizes. For subset of features with sizes from N = 1–10, no improvement in the performance is observed. However, from N=10 onwards, we see changes in performance of the system. It is evident from the table that best performance of 89.36% is obtained at N=16, i.e., with subset of features having only 16 features. However, the best performance on full feature set, i.e., on conventional AdaBoost is 82.97% which is shown in the last row of the table. Hence, it can be observed that coupling the conventional AdaBoost model with sparse linear SVM model improves the performance by 6.39%.Table 3
Performance of the proposed sparse SVM and AdaBoost-based learning system at optimal hyperparameters of the two models on hepatitis disease data.
NλEAcctestAcctrain%Sens. (%)Spec. (%)MCC10.01180.8586.1122.2294.730.23920.015180.8586.1122.2294.730.23930.02180.8586.1122.2294.730.23940.04180.8586.1122.2294.730.23950.06180.8586.1122.2294.730.23960.065180.8586.1122.2294.730.23970.07180.8586.1122.2294.730.23980.085180.8586.1122.2294.730.23990.088180.8586.1122.2294.730.239100.09180.8586.1122.2294.730.239110.1882.9790.7422.2297.360.315160.37589.36100.044.441000.626170.93687.23100.033.33100.00.536183382.9785.1811.11100.00.30219—382.9785.1811.11100.00.302Bold values indicate optimal performance.To statistically analyze the results on the testing data, we utilize confusion matrix. As discussed above, the dataset is divided into training and testing datasets using 70–30 data portioning. Hence, out of the 155 samples, 108 samples are used for training purposes, and the remaining 47 samples are used for testing purposes. Out of the 108 training samples, 23 samples belong to the patient class, and 85 patients belong to healthy class. On the other hand, out of the 47 testing samples, 7 samples belong to the patient group, and 38 samples belong to the healthy group. The predicted results of the proposedL1 SVM-AdaBoost model are depicted statistically in the confusion matrix in Figure 2.Figure 2
Graphical depiction of statistics of the obtained results on the testing dataset in terms of confusion matrix. Spec: specificity; Sens: sensitivity.To further show that the coupling of the sparse linear SVM with conventional AdaBoost model enhances the performance of conventional AdaBoost model, we use AUC. The AUC in case of conventional AdaBoost model is 0.587, while AUC in case of the proposed method is 0.649. Hence, the ROC charts further validate the fact that the coupling of the sparse linear SVM enhances the performance of AdaBoost for hepatitis disease data.


### 4.3. Comparison of the Proposed Method with Some Other Proposed Methods Applied to Hepatitis Data

The above discussion validates that the learning system proposed in this paper significantly augments the strength of the conventional AdaBoost model. In this section, the effectiveness of the learning system thus developed is further validated by carrying out a comparison of its performance with some of the well-known models presented in previous studies. The prediction accuracies and brief details about the models are given in Table4. It is evident that our proposed method promises better performance upon 23 other machine learning models.Table 4
Comparison of the proposed method with some well-known methods proposed for hepatitis disease in terms of prediction accuracy [25, 28, 38, 39].
Model or method numberModel or methodStudy or authorsAcc. (%)1K-nearest neighbours (KNN)Nilashi et al.71.412Neural networkNilashi et al.78.313ANaFISNilashi et al.79.674SVMNilashi et al.81.175ASIStern and Dobnikar82.06Multilayer perceptron + backpropagationAdamczak77.47Linear discriminant analysis (LDA)Stern and Dobnikar86.48Multilayer perceptron (MLP)Ozyildirim, yildirim74.379Radial basis function (Tooldiag)Adamczak79.0101NNStern and Dobnikar85.311Radial basis function (RBF)Ozyildirim, yildirim83.751215NN, stand. EuclideanGrudzinski89.013FSM with rotationsAdamczak89.714FSM without rotationsAdamczak88.515Multilayer perceptron with backpropagationStern and Dobnikar82.116Quadratic discriminant analysisStern and Dobnikar85.817(NB and semi-NB), i.e., Naive Bayes and semi-NBStern and Dobnikar86.318Fisher discriminant analysis (FDA)Stern and Dobnikar84.519LVQStern and Dobnikar83.220GRNNOzyildirim, yildirim80.021ASRStern and Dobnikar85.022IncNetNorbert jankowski86.023Classification and regression tree (decision tree)Stern and Dobnikar82.724LFCStern and Dobnikar81.925L1-SVM-AdaBoostThe proposed method89.36By analyzing Table4, it can be seen that previous methods have exploited various machine learning-based methods to improve the hepatitis disease prediction accuracy. For example, Stern and Dobnikar developed methods based on discriminant analysis (including linear discriminant analysis and quadratic discriminant analysis) and could achieve a classification accuracy of 85.8% with quadratic discriminant analysis. Similarly, Ozyildirim and Yildirim developed a number of models for searching out optimum model with better classification accuracy. They obtained the highest classification accuracy of 83.75% using radial basis function (RBF). Moreover, if we analyze the results tabulated in Table 4, the previous methods have carried out analysis of their proposed method by only considering classification accuracy. In this paper, we analyzed the results of the proposed hybrid method with a number of metrics and proved the robustness of the proposed method from two key metrics, i.e., classification accuracy and area under the curve (AUC).


### 4.4. Limitations of the Study

Although this paper demonstrated the effectiveness of exploitation of sparsity in feature space to improve the performance of the machine learning models, the main limitation is lower sensitivity rate. This is due to the low representation of the patient class in the dataset. The main limitation of the hepatitis disease dataset is its imbalanced nature. The dataset has uneven class distribution, i.e., out of 155 samples, 123 samples belong to the healthy class, and 32 samples belong to the patient class. Recent research pointed out that machine learning models trained under such imbalanced classes show biased performance against the minority class (i.e., the models show very poor performance on the minority class) [40]. On the other hand, the models are biased towards the majority class, i.e., the models will show very good performance on the majority class. In case of the hepatitis disease dataset, the minority class is the patient class, and the majority class is the healthy class. From the results, it can be seen that the majority class has 100% detection accuracy (i.e. 100% specificity) while the minority class has poor detection accuracy, i.e., 44%. In future studies, we need to collect balance datasets, i.e., having the same representation for both the classes. Machine learning models trained under such balanced scenario are supposed to show better sensitivity. Moreover, the exhaustive search method for hyperparameters optimization is time-consuming. In future, application of metaheuristic algorithms [41, 42] should be explored.


## 4.1. Simulation of Conventional AdaBoost Model on Hepatitis Data

In this experiment, we develop the conventional AdaBoost model for the hepatitis disease data. The model is trained using 70% of the dataset and tested on the remaining 30% of the data. An exhaustive grid search algorithm is used to search the optimized version of the AdaBoost model. The results on both optimal hyperparameters and nonoptimal hyperparameters are given in Table2. It is evident from the table that best performance of 82.97% accuracy, 11.11% sensitivity, 100% specificity, and MCC of 0.302 is obtained at optimal hyperparameter, i.e., E=3.Table 2
Performance of the conventional AdaBoost model on HF data.
EAcctestAcctrain%Sens. (%)Spec. (%)MCC382.9785.1811.11100.00.3021076.5993.5111.1192.100.0451274.4696.2911.1189.470.0071474.4697.2211.1189.470.007Bold values indicate optimal performance.


## 4.2. Simulation of the Proposed Method Using the Sparse Linear SVM and AdaBoost Model on Hepatitis Data

In this experiment, the proposed learning system is developed by using both the models, i.e., sparse linear SVM and AdaBoost model. The simulation results are reported in Table3. As can be seen in the table, different values of λ for the sparse SVM generate different subsets of features with different sizes. For subset of features with sizes from N = 1–10, no improvement in the performance is observed. However, from N=10 onwards, we see changes in performance of the system. It is evident from the table that best performance of 89.36% is obtained at N=16, i.e., with subset of features having only 16 features. However, the best performance on full feature set, i.e., on conventional AdaBoost is 82.97% which is shown in the last row of the table. Hence, it can be observed that coupling the conventional AdaBoost model with sparse linear SVM model improves the performance by 6.39%.Table 3
Performance of the proposed sparse SVM and AdaBoost-based learning system at optimal hyperparameters of the two models on hepatitis disease data.
NλEAcctestAcctrain%Sens. (%)Spec. (%)MCC10.01180.8586.1122.2294.730.23920.015180.8586.1122.2294.730.23930.02180.8586.1122.2294.730.23940.04180.8586.1122.2294.730.23950.06180.8586.1122.2294.730.23960.065180.8586.1122.2294.730.23970.07180.8586.1122.2294.730.23980.085180.8586.1122.2294.730.23990.088180.8586.1122.2294.730.239100.09180.8586.1122.2294.730.239110.1882.9790.7422.2297.360.315160.37589.36100.044.441000.626170.93687.23100.033.33100.00.536183382.9785.1811.11100.00.30219—382.9785.1811.11100.00.302Bold values indicate optimal performance.To statistically analyze the results on the testing data, we utilize confusion matrix. As discussed above, the dataset is divided into training and testing datasets using 70–30 data portioning. Hence, out of the 155 samples, 108 samples are used for training purposes, and the remaining 47 samples are used for testing purposes. Out of the 108 training samples, 23 samples belong to the patient class, and 85 patients belong to healthy class. On the other hand, out of the 47 testing samples, 7 samples belong to the patient group, and 38 samples belong to the healthy group. The predicted results of the proposedL1 SVM-AdaBoost model are depicted statistically in the confusion matrix in Figure 2.Figure 2
Graphical depiction of statistics of the obtained results on the testing dataset in terms of confusion matrix. Spec: specificity; Sens: sensitivity.To further show that the coupling of the sparse linear SVM with conventional AdaBoost model enhances the performance of conventional AdaBoost model, we use AUC. The AUC in case of conventional AdaBoost model is 0.587, while AUC in case of the proposed method is 0.649. Hence, the ROC charts further validate the fact that the coupling of the sparse linear SVM enhances the performance of AdaBoost for hepatitis disease data.


## 4.3. Comparison of the Proposed Method with Some Other Proposed Methods Applied to Hepatitis Data

The above discussion validates that the learning system proposed in this paper significantly augments the strength of the conventional AdaBoost model. In this section, the effectiveness of the learning system thus developed is further validated by carrying out a comparison of its performance with some of the well-known models presented in previous studies. The prediction accuracies and brief details about the models are given in Table4. It is evident that our proposed method promises better performance upon 23 other machine learning models.Table 4
Comparison of the proposed method with some well-known methods proposed for hepatitis disease in terms of prediction accuracy [25, 28, 38, 39].
Model or method numberModel or methodStudy or authorsAcc. (%)1K-nearest neighbours (KNN)Nilashi et al.71.412Neural networkNilashi et al.78.313ANaFISNilashi et al.79.674SVMNilashi et al.81.175ASIStern and Dobnikar82.06Multilayer perceptron + backpropagationAdamczak77.47Linear discriminant analysis (LDA)Stern and Dobnikar86.48Multilayer perceptron (MLP)Ozyildirim, yildirim74.379Radial basis function (Tooldiag)Adamczak79.0101NNStern and Dobnikar85.311Radial basis function (RBF)Ozyildirim, yildirim83.751215NN, stand. EuclideanGrudzinski89.013FSM with rotationsAdamczak89.714FSM without rotationsAdamczak88.515Multilayer perceptron with backpropagationStern and Dobnikar82.116Quadratic discriminant analysisStern and Dobnikar85.817(NB and semi-NB), i.e., Naive Bayes and semi-NBStern and Dobnikar86.318Fisher discriminant analysis (FDA)Stern and Dobnikar84.519LVQStern and Dobnikar83.220GRNNOzyildirim, yildirim80.021ASRStern and Dobnikar85.022IncNetNorbert jankowski86.023Classification and regression tree (decision tree)Stern and Dobnikar82.724LFCStern and Dobnikar81.925L1-SVM-AdaBoostThe proposed method89.36By analyzing Table4, it can be seen that previous methods have exploited various machine learning-based methods to improve the hepatitis disease prediction accuracy. For example, Stern and Dobnikar developed methods based on discriminant analysis (including linear discriminant analysis and quadratic discriminant analysis) and could achieve a classification accuracy of 85.8% with quadratic discriminant analysis. Similarly, Ozyildirim and Yildirim developed a number of models for searching out optimum model with better classification accuracy. They obtained the highest classification accuracy of 83.75% using radial basis function (RBF). Moreover, if we analyze the results tabulated in Table 4, the previous methods have carried out analysis of their proposed method by only considering classification accuracy. In this paper, we analyzed the results of the proposed hybrid method with a number of metrics and proved the robustness of the proposed method from two key metrics, i.e., classification accuracy and area under the curve (AUC).


## 4.4. Limitations of the Study

Although this paper demonstrated the effectiveness of exploitation of sparsity in feature space to improve the performance of the machine learning models, the main limitation is lower sensitivity rate. This is due to the low representation of the patient class in the dataset. The main limitation of the hepatitis disease dataset is its imbalanced nature. The dataset has uneven class distribution, i.e., out of 155 samples, 123 samples belong to the healthy class, and 32 samples belong to the patient class. Recent research pointed out that machine learning models trained under such imbalanced classes show biased performance against the minority class (i.e., the models show very poor performance on the minority class) [40]. On the other hand, the models are biased towards the majority class, i.e., the models will show very good performance on the majority class. In case of the hepatitis disease dataset, the minority class is the patient class, and the majority class is the healthy class. From the results, it can be seen that the majority class has 100% detection accuracy (i.e. 100% specificity) while the minority class has poor detection accuracy, i.e., 44%. In future studies, we need to collect balance datasets, i.e., having the same representation for both the classes. Machine learning models trained under such balanced scenario are supposed to show better sensitivity. Moreover, the exhaustive search method for hyperparameters optimization is time-consuming. In future, application of metaheuristic algorithms [41, 42] should be explored.


## 5. Conclusion and Future Work

This work developed an automatic hepatitis disease detection system by using machine learning methods. The AdaBoost model was developed for the hepatitis disease prediction. To improve the classification strength of the AdaBoost model, sparsity in the linear SVM model was exploited. The SVM model eliminated redundant or irrelevant features and thus improved the prediction accuracy of the AdaBoost model. It was also shown that the proposed sparse linear SVM also proves helpful in decreasing the time complexity of the AdaBoost model. Moreover, as evident by the simulation results, our proposed method surpassed many previously published methods in terms of hepatitis disease prediction accuracy. Given the experimental quantitative figures and results, it can thus be safely concluded that the proposed methodology can also be exploited to improve performance of other machine learning models and thus can help to make quality decisions in various other disease detection problems as well.As discussed above, although the proposed method can be used as a tool to improve the performance of machine learning models, the obtained accuracy still needs considerable amount of improvement. Thus, in future studies, more robust cascaded models should be developed by using deep learning approaches for classification. Additionally, the low rate of sensitivity that is caused by lower class representation of the patient class in the dataset is also a limitation of the study that should be considered as an open challenge for the future work. In future studies, extended hepatitis disease datasets should be collected that will have balanced class distribution.



---

*Source: 8870240-2020-11-03.xml*