# Hybrid Hypercube Optimization Search Algorithm and Multilayer Perceptron Neural Network for Medical Data Classification

Authors: Mustafa Tunay; Elnaz Pashaei; Elham Pashaei
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

The hypercube optimization search (HOS) approach is a new efficient and robust metaheuristic algorithm that simulates the dove’s movement in quest of new food sites in nature, utilizing hypercubes to depict the search zones. In medical informatics, the classification of medical data is one of the most challenging tasks because of the uncertainty and nature of healthcare data. This paper proposes the use of the HOS algorithm for training multilayer perceptrons (MLP), one of the most extensively used neural networks (NNs), to enhance its efficacy as a decision support tool for medical data classification. The proposed HOS-MLP model is tested on four significant medical datasets: orthopedic patients, diabetes, coronary heart disease, and breast cancer, to assess HOS’s success in training MLP. For verification, the results are compared with eleven different classifiers and eight well-regarded MLP trainer metaheuristic algorithms: particle swarm optimization (PSO), biogeography-based optimizer (BBO), the firefly algorithm (FFA), artificial bee colony (ABC), genetic algorithm (GA), bat algorithm (BAT), monarch butterfly optimizer (MBO), and the flower pollination algorithm (FPA). The experimental results demonstrate that the MLP trained by HOS outperforms the other comparative models regarding mean square error (MSE), classification accuracy, and convergence rate. The findings also reveal that the HOS help the MLP to produce more accurate results than other classification algorithms for the prediction of diseases.


---

## Body

## 1. Introduction

Medical data classification is a growing field of research that provides pathologists with vital knowledge for diagnosing and treating diseases. Multilayer perceptron (MLP) neural network is one of the most powerful classification algorithms that has been successfully and widely employed for many medical problems [1–4]. The key point in using MLP is to determine the best values for the parameters of the neural network. The most commonly used gradient-based learning method of MLP is the backpropagation (BP) algorithm. However, this learning method has some flaws, including sluggish convergence, high reliance on the initial solutions, and a proclivity for getting stuck in local optima. Optimization algorithms (OAs) are offered as viable alternatives to gradient-based MLP training approaches in this regard. Several works have been published in the literature, including group search optimizer (GSO) [5], symbiotic organisms search (SOS) [6] algorithm, lightning search algorithm (LSA) [7], ant lion optimizer (ALO) [8], Krill herd algorithm (KHA) [9], grasshopper optimization algorithm (GOA) [10, 11], artificial bee colony (ABC) [12], social spider optimization algorithm (SSO) [13], hybrid of ABC and dragonfly algorithm (DA) [14], artificial ant colony optimization (ACO) [15], particle swarm optimization (PSO) [16], cuckoo search (CS) [17, 18], moth-flame optimization (MFO) [19, 20], whale optimization algorithm (WOA) [21], gray wolf optimizer (GWO) [22, 23], black hole algorithm (BHA) [24], invasive weed optimization [25], multiverse optimizer algorithm (MOA) [26, 27], bat algorithm [28], and salp swarm algorithm (SSA) [29].Although various OAs have already been investigated for training MLP neural networks, because of the duality of Aos’ exploration and exploitation capabilities, there is still room for new designs and upgrades to current ones [30]. Also, in training MLP, the issue of slow convergence rate and trapping in local optima remains partially unsolved.The purpose of this study is to introduce a new optimization technique, called the hypercube optimization search (HOS) algorithm, for training MLP to present an improved classification approach for medical data by optimizing the MLP’s weights and biases parameters. The HOS is recommended for training MLP to overcome the aforementioned challenges due to its outstanding performance in escaping local optima and fast convergence speed [31, 32]. Also, HOS have fewer parameters and is easy to use, simple in principle, and adaptable when compared to other swarm-based OAs.This paper’s contributions can be summed up as follows:(i)
To propose a new stochastic learning approach for training MLPs, in order to boost the MLP’s performance in the classification of health data.(ii)
To evaluate HOS-MLP’s performance on four important medical datasets: diabetes, breast cancer, coronary heart disease, and orthopedic patients, and compare its performance against eleven different classifiers and eight well-known OA-based MLP trainer techniques.(iii)
To achieve better outcomes than previous studies, using suggested HOS-MLP in terms of mean square error (MSE), classification accuracy, and convergence rate.This paper is structured as follows: Section2 presents the MLP. The HOS algorithm is explained in Section 3, whereas the proposed HOS-MLP approach is introduced in Section 4. Section 5 shows the experimental results and discussion. Finally, section 6 gives a conclusion as well as recommendations for further work.


## 2. Multilayer Perceptron Neural Network

The feedforward neural network (FNN) is one of the most prevalent forms of artificial neural network (ANN) and MLP is a well-known type of FNN that is widely used in solving realistic classification problems [10]. An MLP is made up of three groups of layers: input (i), hidden layer (j), and output (k). Each layer consists of a specific number of neurons, and each neuron has full-weighted connections with the adjacent layer neurons. A single hidden layer MLP network was used in this paper, which was demonstrated in Figure 1.Figure 1
Structure of single hidden layer MLP.Each neuron can carry out two functions in the MLP: weighted summation and activation. The weighted sum is calculated using equation (1) for each hidden neuron j as follow:(1)sj=∑i=1nwijxi+βj,where wij describes the connection weight, βj is the biased term, xi denotes the input i, and n shows the total number of inputs. In the second step, using the outcome of equation (1), an activation function is utilized to calculate the neurons’ output. The function is illustrated as follows:(2)fsj=11+e−sj.The most commonly used sigmoid activation function was selected in the MLP [22, 26]. Utilizing the results of the hidden neurons, the final productions of the output neurons are computed as follows:(3)yk=fok=11+e−sk,sk=∑j=1hwjkfsj+βk.MLP’s performance depends highly on weights and biases, and training MLP aims to find optimal weights and biases. MLP training is a challenging task that leads to a high-performance MLP [21]. The HOS algorithm is used as a training method for MLPs in the following sections.


## 3. Hypercube Optimization Search Algorithm

The HOS algorithm, inspired by a dove’s behavior in exploring new food zones, was proposed by Abiyev and Tunay for solving high-dimensional numerical problems [31]. The HOS algorithm is based on a randomly distributed set of points inside an m-dimensional hypercube (HC). HOS exhibit fast population convergence by shrinking the area of the HC at each iteration. The HOS algorithm consists of three stages: (A) the initialization process, (B) the displacement-shrink process, and (C) the searching areas process. These stages can be described in detail as follows.


### 3.1. Stage A: Initialization Process

The HOS algorithm begins with the initialization process, in which randomly generated points within a given HC form the candidate solutions matrix. Several starting conditions in the initialization phase should be computed, including (1) lower-upper boundaries (lb,ub), (2) size rdim, (3) central value (xc), and (4) dimension of the HC m.(4)m=lengthxc,xc=ub+lb2,d=lb−ub,rdim=d2,lb=minXbounds,ub=maxXbounds.At the starting stage, the first HC is created by assigning random values tordim and xc. The uniformly distributed N points xi=xi1,xi2,…,xim are then randomly produced inside the HC. These points could also be represented in matrix form X with the size of (N×m). The upper and lower boundaries of the first HC are then calculated using the X matrix. The rdim and xc of the next HC are determined using those boundaries. The X matrix is also utilized for evaluation, in which the best value of the fitness function Fbest and corresponding xbest point is determined within the population at i th iteration. Using local search, the xbest point is improved as follows:(5)xbestnew=xbest+ρΔF,where F is the fitness function, and 0≤ρ≤1.


### 3.2. Stage B: Displacement-Shrink Process

The displacement-shrink phase aims to determine the center of the next hypercube (new hypercube)xcnew and evaluate the fitness function. The center of the next hypercube is obtained using the average of the sum of the previous hypercube’s center and the present best point (xbest) as follows:(6)xcnew=xbestnew+xc2.In this process, each iteration generates fresh data points, and the fitness function is evaluated. The hypercube size has been modified based on the evaluation results. This process is used as a conservative measure to reduce excessive variability in the search space. As a consequence, the size of HC is decreased and the search space is reduced, which is called “shrinking.” The density of the search points (population) increases as the hypercube size decreases. The movement of the best value is governed by contraction. For smaller movements, the contraction is stronger. This ensures rapid convergence while also preventing the algorithm from becoming trapped at an undesirable (local) minimum.The algorithm will cycle through a sequence of points starting from the current position to estimate the maximum distance. The value of theFbest is first compared with the Fmean=Fxbest+xlast−center/2. If Fmean value is less than Fbest in the given iteration, x displacements (or x movements) is computed and normalized twice at each iteration using the following formulas:(7)Normalizedxthe previousxfor minimumxn=x−xcd,(8)Normalizedxmincurrentxfor minimumxn=xmin−xcd,(9)Normalize distancedn=sumxn−xminn20.5d,(10)Re−normalize distancednn=dnm.To convert the displacement into unity-sided points, each element ofx is first divided by the associated beginning interval (equations (7) and (8)), and then this number is again normalized by dividing it to the diagonal of the points, i.e., m (equations (9) and (10)). If Fmean value in the specified iteration is greater than Fbest, x displacements will not occur and dnn will be assigned to 1. The searching areas process is carried out in the next step if the conditions are not met.


### 3.3. Stage C: Searching Areas Process

The phase of the search area generates a new HC by initializing new values tordim and xc according to the value of dnn. If the 0≤dnn≤1 condition is satisfied, the factor of convergence S is calculated and values of the rdim and xc are updated accordingly using the following formulas:(11)xc=xbest,rdim=rdim∗S,S=1−0.2e−3dnn.The size of the HC is reduced by multiplyingrdim with S factor. If 0≤dnn≤1 condition is met, the size of HC remains unchanged. HOS ensure the quick arrival of candidate solutions to a global minimum by reducing the area of the hypercube after each iteration. The entire procedure is repeated till particular termination criteria are met. The HOS algorithm is depicted in Figure 2. More details are provided in [31, 32].Figure 2
The flowchart of the HOS algorithm.


## 3.1. Stage A: Initialization Process

The HOS algorithm begins with the initialization process, in which randomly generated points within a given HC form the candidate solutions matrix. Several starting conditions in the initialization phase should be computed, including (1) lower-upper boundaries (lb,ub), (2) size rdim, (3) central value (xc), and (4) dimension of the HC m.(4)m=lengthxc,xc=ub+lb2,d=lb−ub,rdim=d2,lb=minXbounds,ub=maxXbounds.At the starting stage, the first HC is created by assigning random values tordim and xc. The uniformly distributed N points xi=xi1,xi2,…,xim are then randomly produced inside the HC. These points could also be represented in matrix form X with the size of (N×m). The upper and lower boundaries of the first HC are then calculated using the X matrix. The rdim and xc of the next HC are determined using those boundaries. The X matrix is also utilized for evaluation, in which the best value of the fitness function Fbest and corresponding xbest point is determined within the population at i th iteration. Using local search, the xbest point is improved as follows:(5)xbestnew=xbest+ρΔF,where F is the fitness function, and 0≤ρ≤1.


## 3.2. Stage B: Displacement-Shrink Process

The displacement-shrink phase aims to determine the center of the next hypercube (new hypercube)xcnew and evaluate the fitness function. The center of the next hypercube is obtained using the average of the sum of the previous hypercube’s center and the present best point (xbest) as follows:(6)xcnew=xbestnew+xc2.In this process, each iteration generates fresh data points, and the fitness function is evaluated. The hypercube size has been modified based on the evaluation results. This process is used as a conservative measure to reduce excessive variability in the search space. As a consequence, the size of HC is decreased and the search space is reduced, which is called “shrinking.” The density of the search points (population) increases as the hypercube size decreases. The movement of the best value is governed by contraction. For smaller movements, the contraction is stronger. This ensures rapid convergence while also preventing the algorithm from becoming trapped at an undesirable (local) minimum.The algorithm will cycle through a sequence of points starting from the current position to estimate the maximum distance. The value of theFbest is first compared with the Fmean=Fxbest+xlast−center/2. If Fmean value is less than Fbest in the given iteration, x displacements (or x movements) is computed and normalized twice at each iteration using the following formulas:(7)Normalizedxthe previousxfor minimumxn=x−xcd,(8)Normalizedxmincurrentxfor minimumxn=xmin−xcd,(9)Normalize distancedn=sumxn−xminn20.5d,(10)Re−normalize distancednn=dnm.To convert the displacement into unity-sided points, each element ofx is first divided by the associated beginning interval (equations (7) and (8)), and then this number is again normalized by dividing it to the diagonal of the points, i.e., m (equations (9) and (10)). If Fmean value in the specified iteration is greater than Fbest, x displacements will not occur and dnn will be assigned to 1. The searching areas process is carried out in the next step if the conditions are not met.


## 3.3. Stage C: Searching Areas Process

The phase of the search area generates a new HC by initializing new values tordim and xc according to the value of dnn. If the 0≤dnn≤1 condition is satisfied, the factor of convergence S is calculated and values of the rdim and xc are updated accordingly using the following formulas:(11)xc=xbest,rdim=rdim∗S,S=1−0.2e−3dnn.The size of the HC is reduced by multiplyingrdim with S factor. If 0≤dnn≤1 condition is met, the size of HC remains unchanged. HOS ensure the quick arrival of candidate solutions to a global minimum by reducing the area of the hypercube after each iteration. The entire procedure is repeated till particular termination criteria are met. The HOS algorithm is depicted in Figure 2. More details are provided in [31, 32].Figure 2
The flowchart of the HOS algorithm.


## 4. HOS for MLP Training

The suggested HOS-MLP method, in which the HOS algorithm is utilized for training the MLP, is explained in detail in this section. When the method is designed, two important aspects are considered: (1) the representation of candidate solutions in HOS for training MLP, and (2) the definition of a fitness function for solution assessment.The matrix encoding approach is utilized in HOS-MLP to represent candidate solutions. For MLP’s weight and bias parameters, each solution provides a set of values. A solution can be represented as follows:(12)W1=w1,1⋯wn,1⋮⋱⋮w1,h⋯wn,h,β1=β1⋮βh,W2'=w1,1⋯wh,1⋮⋱⋮w1,k⋯wh,k,β2=β1⋮βk,solutioni=W1,β1,W2',β2.where W1 indicates the weight matrix of linkages between hidden neurons and input and W2′ demonstrates the transpose of the weights matrix of the linkages between the hidden neurons and output. For hidden and output neurons, the β1, and β2 represent bias values, respectively.It is worth mentioning that the number of neurons in the input and output layers is specified by the dataset’s total number of features and labels, while the Kolmogorov theorem is utilized to determine the number of neurons within the hidden layer (H) using the following equation:(13)H=2∗Input+1.The MSE is utilized as the objective function for measuring the fitness value of candidate solutions in the proposed HOS-MLP approach as follows:(14)MSE=1n∑i=1ny−y^2,where y and y^ symbolize the actual and predicted class label, and n is the number of samples in training data. The HOS based MLP training approach is carried out in the following stages:(1)
Initialization: within an HC, the initial solutions (points) are generated randomly. Each solution represents the possible values for the parameters of MLP.(2)
Fitness evaluation: the solutions are assigned to MLPs, and the goodness of each MLP is then evaluated using MSE. The objective is to find the best MLP with the least MSE based on the dataset’s training samples.(3)
Creating new HC and update solutions.(4)
Steps 2 and 3 are repeated until the full number of iterations is completed. Eventually, the solution with the least MSE is reported as the optimal solution. It is then allocated to MLP and the performance of the trained MLP is assessed on the test dataset.Figure3 illustrates the suggested HOS-MLP framework.Figure 3
The flowchart of suggested HOS-MLP for medical data classification.


## 5. Results and Discussions

In this section, the proposed HOS-MLP model is examined on four medical datasets: orthopedic patients (vertebral column) [33], diabetes [34], coronary heart disease (Saheart) [35], and Wisconsin breast cancer [36]. The characteristics of the medical datasets are summarized in Table 1.Table 1
Datasets description.
Dataset#Features#Class#Training sample#Testing sampleMLP structure (input-hidden-output)Breast cancer924692309-19-2Diabetes825152538-17-2Saheart923101529-19-2Vertebral column62161796-13-3All medical datasets are split into two parts: 66.66% of the data is used for the training set, and the remaining (33.33%) is used for the test set. In this partitioning, stratified sampling is used to retain the initial data distribution in the training and testing. The algorithms have been run 20 different times to produce statistically valid results. The Min-max scaling method was utilized to standardize all feature values within the range [0, 1] using the following equation:(15)x′=xi−minmax−min.The suggested HOS-MLP is compared with eight well-known and recent Oas, including ABC [12], PSO [16], BAT [28], GA [37], BBO [38], firefly algorithm (FF/FA) [39], monarch butterfly optimization (MBO) [40], and flower pollination algorithm (FPA) [41]. For all OAs, the population size was set to 70, and the maximum number of iterations was set to 250 in all experiments. Two optional parameters in the HOS algorithm, tolF, and tolX, were set to 1e-09 and 1e-01, respectively. The parameters tolF and tolX represent relative tolerance for fitness function and vector x to stop the algorithm. The evaluation measures employed in this work are accuracy, MSE, box plot, and coverage rate. The rest of the parameters were set as suggested in [42].


### 5.1. Breast Cancer Dataset

Many binary classification problems use accuracy and MSE metrics to show the model's ability to split the two-class labels. Table2 summarizes the testing set results for the suggested HOS-MLP model compared to other OAs models from the literature. From Table 2, Figures 4(a) and 4(b), it can be noticed that the suggested method performs very better than other methods in terms of convergence rate. Although all algorithms achieved high ratios in terms of average accuracy, the suggested HOS-MLP shows reasonable and competitive results with the lowest MSE average (Figures 4(c) and 4(d)).Table 2
The performance results of comparative methods on the breast cancer dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.034000.038430.0363998.31994.95896.891BAT-MLP0.016240.060230.0343298.31992.43796.218BBO-MLP0.029380.034420.0319598.31996.63997.255FF-MLP0.030430.032790.0320997.89996.63997.311FPA-MLP0.030790.033200.0321698.31995.37897.241GA-MLP0.027860.032060.0302097.47995.37896.751MBO-MLP0.023510.038870.0297497.89994.95896.695PSO-MLP0.030620.039990.0349697.89995.37897.045HOS-MLP0.018330.028190.02346497.52193.473396.833Figure 4
Performance analysis of the breast cancer dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)


### 5.2. Diabetes Dataset

The diabetes dataset evaluation results are illustrated in Table3 and Figure 5. When the convergence curves in Figure 5(a) and 5(b) are compared to the other algorithms, the suggested strategy has a very high convergence rate, while most methods, such as GOA and ABC, have stagnated after 98 iterations. The proposed approach displays the maximum ratios in terms of average and best accuracy (Table 3 and Figure 5(c)). The boxplot (Figure 5(d)) indicates that, while GOA has a more compact box, the proposed approach has the lowest error and acceptable stability.Table 3
The performance results of comparative methods on the diabetes dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.156350.169000.1627680.53470.99274.822BAT-MLP0.143570.172670.1529979.38973.28276.374BBO-MLP0.149470.155740.1523077.09973.28275.611FF-MLP0.150740.153000.1519977.48175.19176.349FPA-MLP0.151610.160080.1558278.24472.13775.242GA-MLP0.145710.157390.1515678.24472.13775.165MBO-MLP0.153260.173850.1635578.62670.61174.733PSO-MLP0.154380.169160.1637278.24467.17673.104HOS-MLP0.140510.153140.1454880.23774.70177.840Figure 5
Performance analysis of the diabetes dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)


### 5.3. Saheart Dataset

Comparing the HOS-ML model with other OAs models from Table4, we obtained better accuracy and MSE. This observation proves that HOS-ML can accurately model classification tasks. Figure 6 demonstrates the proposed HOS-ML model’s accuracy, MSE, convergence speed, and stability. In terms of convergence speed, Figures 6(a) and 6(b) illustrate that, relative to the other algorithms, the proposed MLP-based trainer has a very fast convergence rate and the smallest MSE average (see Figure 6(d)). The suggested strategy produces an improved performance in contrast to other methods in terms of average accuracy (Figure 6(c)).Table 4
The performance results of comparative methods on the Saheart dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.175560.184740.1800776.58267.72271.160BAT-MLP0.158430.176200.1651075.94968.98772.405BBO-MLP0.163080.170000.1664675.31669.62072.911FF-MLP0.1673970.170720.1691674.05169.62071.730FPA-MLP0.168520.176570.1727376.58268.35472.869GA-MLP0.163910.171650.1677675.94968.35471.814MBO-MLP0.170050.195850.1827476.58268.35472.932PSO-MLP0.175050.181580.1786777.84864.55772.658HOS-MLP0.150630.170960.1601576.97367.10174.842Figure 6
Performance analysis of the Saheart dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)


### 5.4. Vertebral Dataset

The results of the evaluations for the vertebral dataset is shown in Table5 and Figure 7. For this dataset, the evaluation results of all MLP-trainers were very close and competitive, but our proposed approach showed very faster convergence as can be seen in Figures 7(a) and 7(b). The boxplot (Figure 7(d)) also confirms that the proposed approach has the smallest MSE. Moreover, our suggested algorithm has obtained outstanding performance in terms of worst, average, and best accuracy (Table 5 and Figure 7(c)).Table 5
The performance results of comparative methods on the vertebral dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.0813480.0997840.09121190.56677.35884.371BAT-MLP0.0691380.1557410.09939589.62368.86883.553BBO-MLP0.0801430.0960750.08886988.67984.90686.730FF-MLP0.0843140.0991390.09227787.73684.90686.384FPA-MLP0.0924320.1006730.09653388.67983.96286.730GA-MLP0.0804550.0861190.08350688.67983.96285.786MBO-MLP0.0887450.1198800.10239790.56676.41585.660PSO-MLP0.0999440.1166380.10817890.56677.35885.031HOS-MLP0.0467930.0780070.06262993.67091.13992.405Figure 7
Performance analysis of the vertebral dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)The average classification accuracy of eleven different classifiers on 4 medical datasets is shown in Table6 and Figure 8. These classifiers are Naïve Bayes (NB), Bayes network learning (BayesNet), support vector machine (SVM) [43, 44], MLP using backpropagation (MLP), K nearest neighbor (KNN), AdaboostM1 [45], bagging, fuzzy lattice reasoning (FLR) classifier, random forest (RF) [46], fuzzy unordered rule induction algorithm (FURIA), and logistic model tree (LMT).Table 6
Comparison of accuracy (%) for the twelve classification algorithms. Bold values represent the best results.
DatasetNBBayesNetSVMMLPKNNAdaboostM1BaggingFLRRFFURIALMTHOS-MLPBreast cancer94.9596.2195.7995.3795.7994.5394.9589.4995.7994.9596.2196.833Diabetes77.0178.1680.8474.3272.7977.0177.7767.0476.2477.3979.3177.840Saheart70.1270.1272.7266.8860.3871.4270.1264.2864.9374.6774.0274.842Vertebral72.578.7588.7590.0081.2585.0088.7587.590.0083.7590.0092.405Figure 8
The classification performance of twelve classifiers on 4 medical datasets.As shown in Table6 and Figure 8, the proposed algorithm has the best performance among the eleven algorithms on 3 medical datasets. For the diabetes dataset, the proposed HOS-ML ranked 4th, after SVM, BayesNet, and LMT.Overall, the experimental findings demonstrate that the MSE results of the proposed HOS-MLP are greatly better relative to other MLP-based optimization techniques for all medical datasets. The outstanding advantage of HOS is that it can achieve accurate results with a significantly higher convergence rate than other existing methods. However, some parameters in HOS should be adjusted, and some elements of HOS can be tweaked to increase the algorithm’s classification accuracy in certain datasets.


## 5.1. Breast Cancer Dataset

Many binary classification problems use accuracy and MSE metrics to show the model's ability to split the two-class labels. Table2 summarizes the testing set results for the suggested HOS-MLP model compared to other OAs models from the literature. From Table 2, Figures 4(a) and 4(b), it can be noticed that the suggested method performs very better than other methods in terms of convergence rate. Although all algorithms achieved high ratios in terms of average accuracy, the suggested HOS-MLP shows reasonable and competitive results with the lowest MSE average (Figures 4(c) and 4(d)).Table 2
The performance results of comparative methods on the breast cancer dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.034000.038430.0363998.31994.95896.891BAT-MLP0.016240.060230.0343298.31992.43796.218BBO-MLP0.029380.034420.0319598.31996.63997.255FF-MLP0.030430.032790.0320997.89996.63997.311FPA-MLP0.030790.033200.0321698.31995.37897.241GA-MLP0.027860.032060.0302097.47995.37896.751MBO-MLP0.023510.038870.0297497.89994.95896.695PSO-MLP0.030620.039990.0349697.89995.37897.045HOS-MLP0.018330.028190.02346497.52193.473396.833Figure 4
Performance analysis of the breast cancer dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)


## 5.2. Diabetes Dataset

The diabetes dataset evaluation results are illustrated in Table3 and Figure 5. When the convergence curves in Figure 5(a) and 5(b) are compared to the other algorithms, the suggested strategy has a very high convergence rate, while most methods, such as GOA and ABC, have stagnated after 98 iterations. The proposed approach displays the maximum ratios in terms of average and best accuracy (Table 3 and Figure 5(c)). The boxplot (Figure 5(d)) indicates that, while GOA has a more compact box, the proposed approach has the lowest error and acceptable stability.Table 3
The performance results of comparative methods on the diabetes dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.156350.169000.1627680.53470.99274.822BAT-MLP0.143570.172670.1529979.38973.28276.374BBO-MLP0.149470.155740.1523077.09973.28275.611FF-MLP0.150740.153000.1519977.48175.19176.349FPA-MLP0.151610.160080.1558278.24472.13775.242GA-MLP0.145710.157390.1515678.24472.13775.165MBO-MLP0.153260.173850.1635578.62670.61174.733PSO-MLP0.154380.169160.1637278.24467.17673.104HOS-MLP0.140510.153140.1454880.23774.70177.840Figure 5
Performance analysis of the diabetes dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)


## 5.3. Saheart Dataset

Comparing the HOS-ML model with other OAs models from Table4, we obtained better accuracy and MSE. This observation proves that HOS-ML can accurately model classification tasks. Figure 6 demonstrates the proposed HOS-ML model’s accuracy, MSE, convergence speed, and stability. In terms of convergence speed, Figures 6(a) and 6(b) illustrate that, relative to the other algorithms, the proposed MLP-based trainer has a very fast convergence rate and the smallest MSE average (see Figure 6(d)). The suggested strategy produces an improved performance in contrast to other methods in terms of average accuracy (Figure 6(c)).Table 4
The performance results of comparative methods on the Saheart dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.175560.184740.1800776.58267.72271.160BAT-MLP0.158430.176200.1651075.94968.98772.405BBO-MLP0.163080.170000.1664675.31669.62072.911FF-MLP0.1673970.170720.1691674.05169.62071.730FPA-MLP0.168520.176570.1727376.58268.35472.869GA-MLP0.163910.171650.1677675.94968.35471.814MBO-MLP0.170050.195850.1827476.58268.35472.932PSO-MLP0.175050.181580.1786777.84864.55772.658HOS-MLP0.150630.170960.1601576.97367.10174.842Figure 6
Performance analysis of the Saheart dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)


## 5.4. Vertebral Dataset

The results of the evaluations for the vertebral dataset is shown in Table5 and Figure 7. For this dataset, the evaluation results of all MLP-trainers were very close and competitive, but our proposed approach showed very faster convergence as can be seen in Figures 7(a) and 7(b). The boxplot (Figure 7(d)) also confirms that the proposed approach has the smallest MSE. Moreover, our suggested algorithm has obtained outstanding performance in terms of worst, average, and best accuracy (Table 5 and Figure 7(c)).Table 5
The performance results of comparative methods on the vertebral dataset in the terms of mean square error (MSE) and classification accuracy.
Comparative algorithmsMSEAccuracy (%)BestWorstMeanBestWorstMeanABC-MLP0.0813480.0997840.09121190.56677.35884.371BAT-MLP0.0691380.1557410.09939589.62368.86883.553BBO-MLP0.0801430.0960750.08886988.67984.90686.730FF-MLP0.0843140.0991390.09227787.73684.90686.384FPA-MLP0.0924320.1006730.09653388.67983.96286.730GA-MLP0.0804550.0861190.08350688.67983.96285.786MBO-MLP0.0887450.1198800.10239790.56676.41585.660PSO-MLP0.0999440.1166380.10817890.56677.35885.031HOS-MLP0.0467930.0780070.06262993.67091.13992.405Figure 7
Performance analysis of the vertebral dataset: (a) convergence curve of the proposed approach, (b) convergence curve for the compared methods, (c) accuracy for the compared methods, and (d) box plot for the compared methods.
(a)(b)(c)(d)The average classification accuracy of eleven different classifiers on 4 medical datasets is shown in Table6 and Figure 8. These classifiers are Naïve Bayes (NB), Bayes network learning (BayesNet), support vector machine (SVM) [43, 44], MLP using backpropagation (MLP), K nearest neighbor (KNN), AdaboostM1 [45], bagging, fuzzy lattice reasoning (FLR) classifier, random forest (RF) [46], fuzzy unordered rule induction algorithm (FURIA), and logistic model tree (LMT).Table 6
Comparison of accuracy (%) for the twelve classification algorithms. Bold values represent the best results.
DatasetNBBayesNetSVMMLPKNNAdaboostM1BaggingFLRRFFURIALMTHOS-MLPBreast cancer94.9596.2195.7995.3795.7994.5394.9589.4995.7994.9596.2196.833Diabetes77.0178.1680.8474.3272.7977.0177.7767.0476.2477.3979.3177.840Saheart70.1270.1272.7266.8860.3871.4270.1264.2864.9374.6774.0274.842Vertebral72.578.7588.7590.0081.2585.0088.7587.590.0083.7590.0092.405Figure 8
The classification performance of twelve classifiers on 4 medical datasets.As shown in Table6 and Figure 8, the proposed algorithm has the best performance among the eleven algorithms on 3 medical datasets. For the diabetes dataset, the proposed HOS-ML ranked 4th, after SVM, BayesNet, and LMT.Overall, the experimental findings demonstrate that the MSE results of the proposed HOS-MLP are greatly better relative to other MLP-based optimization techniques for all medical datasets. The outstanding advantage of HOS is that it can achieve accurate results with a significantly higher convergence rate than other existing methods. However, some parameters in HOS should be adjusted, and some elements of HOS can be tweaked to increase the algorithm’s classification accuracy in certain datasets.


## 6. Conclusion

This study introduced an improved classification approach, HOS-MLP, to increase the precision of medical diagnosis. The HOS algorithm was employed to adjust the MLP weights and bias values. The high-performance, simplicity, and fast convergence speed of the HOS algorithm were the inspiration behind the choice of HOS for training MLP. To evaluate the efficacy of the suggested HOS-MLP, its classification performance was assessed on four challenging real biomedical datasets: coronary heart disease, orthopedic patients, diabetes, and breast cancer. The performance of the model was compared with eleven different classifiers and eight well-known OA-based MLP-trainers such as ABC, GA, BAT, BBO, PSO, FF, FPA, and MBO. The experimental results of HOS on those biomedical classification problems are promising in terms of convergence rate compared to existing OAs. It managed to demonstrate better classification accuracy in most cases. We conclude that the HOS can train MLP well for classifying biomedical datasets since the HOS-trained MLP presents a higher convergence speed and better classification accuracy than current MLP training techniques and existing state-of-the-art classifiers.In future work, HOS can be utilized to find the optimal structure of the MLP neural network, including the number of hidden layers and nodes. HOS can also be employed to train other forms of ANNs, such as the radial basis function (RBF). It may also be a valuable contribution to solving engineering classification problems using the proposed HOS-MLP.



---

*Source: 1612468-2022-03-25.xml*