# GGA-MLP: A Greedy Genetic Algorithm to Optimize Weights and Biases in Multilayer Perceptron

Authors: Priti Bansal; Rishabh Lamba; Vaibhav Jain; Tanmay Jain; Sanchit Shokeen; Sumit Kumar; Pradeep Kumar Singh; Baseem Khan
Journal: Contrast Media & Molecular Imaging
 (2022)

---

## Abstract

The task of designing an Artificial Neural Network (ANN) can be thought of as an optimization problem that involves many parameters whose optimal value needs to be computed in order to improve the classification accuracy of an ANN. Two of the major parameters that need to be determined during the design of an ANN are weights and biases. Various gradient-based optimization algorithms have been proposed by researchers in the past to generate an optimal set of weights and biases. However, due to the tendency of gradient-based algorithms to get trapped in local minima, researchers have started exploring metaheuristic algorithms as an alternative to the conventional techniques. In this paper, we propose the GGA-MLP (Greedy Genetic Algorithm-Multilayer Perceptron) approach, a learning algorithm, to generate an optimal set of weights and biases in multilayer perceptron (MLP) using a greedy genetic algorithm. The proposed approach increases the performance of the traditional genetic algorithm (GA) by using a greedy approach to generate the initial population as well as to perform crossover and mutation. To evaluate the performance of GGA-MLP in classifying nonlinear input patterns, we perform experiments on datasets of varying complexities taken from the University of California, Irvine (UCI) repository. The experimental results of GGA-MLP are compared with the existing state-of-the-art techniques in terms of classification accuracy. The results show that the performance of GGA-MLP is better than or comparable to the existing state-of-the-art techniques.


---

## Body

## 1. Introduction

Artificial Neural networks (ANNs) are computing models inspired by the biological nervous system. An ANN consists of an interconnected network of nodes called artificial neurons which are organized in the form of layers, namely, input layer, hidden layers, and output layer [1]. A set of synaptic weights is used to interconnect the nodes that form these layers. ANNs have been applied to a broad range of problems like classification, regression, prediction, pattern recognition, and disease diagnosis [2–6]. Classification is one of the important areas of research in the field of data science. Many classification models exist, out of which ANNs are among the most widely used models.In this paper, our focus is on multilayer perceptron (MLP) which is a multilayer feedforward neural network. Classification using MLP is basically a two-step process. The first step is the learning (training) phase in which a classifier is built to describe a predetermined set of data classes for a given dataset (training data). In the second step, the model which has been built in the training phase is used for the classification of the unclassified data (test data) for estimating the accuracy of the classifier. During the learning phase, MLP learns by adjusting synaptic weights and biases iteratively in an attempt to correctly predict the class labels of the input data. The process of weight and bias update continues until the acquired knowledge is sufficient and the network reaches a specified level of accuracy; i.e., a predefined error measure is minimized, or the maximum number of epochs is reached [7]. After the completion of the learning phase, it is mandatory to assess the performance of MLP, i.e., its generalization and predictive capabilities, using samples of data (test data) that are different from those used during the training phase for the given dataset. To achieve generalization, MLPs need to avoid the issues of both underfitting and overfitting during the training phase. To achieve the best results, it is therefore required that the number of training patterns should be sufficiently larger than the total number of connections in the neural network. The performance of MLP is highly dependent on the learning method used to train it during the training phase. Several learning algorithms exist in the literature with the aim of finding an optimal MLP. These learning algorithms can be broadly classified into three categories, namely, conventional methods [8–12], metaheuristic-based methods [13–37], and hybrid methods [20, 38–44].Despite the existence of a large number of learning algorithms, researchers continue to apply new optimization techniques like multimean particle swarm optimization (MMPSO) [28], whale optimization algorithm (WOA) [23], multiverse optimizer (MVO) [34], grasshopper optimization algorithm (GOA) [35], and firefly algorithm [36] to generate an optimal set of synaptic weights in an attempt to further improve the accuracy and performance of MLP. As stated in No-Free-Lunch (NFL) theorem [45], there is no optimization technique that solves all optimization problems. It is quite possible that an existing learning algorithm may train an MLP well for some datasets while it fails to do the same for some other datasets. This makes the field of generating optimal connection weights a dynamic research area. This is the main motivation behind the work presented in this paper, in which we propose a hybrid learning algorithm to train MLP.GA is an evolutionary algorithm (EA) and is one of the most widely investigated algorithms among the metaheuristic algorithms in designing neural networks. Over the years, GA and its variants have been successfully applied in several domains for ANN weight [13–20], topology [46–48], and feature set optimization [49, 50], as well as parameter tuning [51, 52]. A comprehensive review of optimization of neural networks using GA can be found in [53]. The efficiency, effectiveness, and ease of use of GA motivated us to further improve the performance of GA in optimizing weights of MLP by integrating greedy techniques with GA. The proposed algorithm Greedy Genetic Algorithm–Multilayer Perceptron (GGA-MLP) improves the performance of traditional GA by using a greedy approach to generate the initial population as well as to perform crossover and mutation. Some of the application areas of the proposed work are disease identification, e-mail spam identification, prediction of the stock market, and fruit classification. The main challenge with the proposed approach is that it may not work well with some of the datasets, as stated by No-Free-Lunch (NFL) theorem [45] mentioned above. Finally, the performance of GGA-MLP is compared with various classifiers as well as the existing state-of-the-art metaheuristic algorithms for training MLP. The key contributions of this paper are as follows:(1)
A hybrid learning algorithm, GGA-MLP, that integrates greedy techniques with GA is proposed to train MLP(2)
GGA-MLP is evaluated and compared with existing state-of-the-art algorithms on 10 datasets of different complexitiesThe paper is organized as follows. Related work is presented in Section2. A brief overview of GA is given in Section 3. In Section 4, the proposed GGA-MLP for optimization of MLP weights and biases is presented. In Section 5, experiments conducted to evaluate the effectiveness of GGA-MLP are presented, and results are discussed. Finally, the conclusion and future work are discussed in Section 6.


## 2. Related Work

In conventional methods, backpropagation (BP) is the most widely used algorithm to train multilayer feedforward networks (MLFFNs). BP uses a gradient descent rule that tries to minimize the error of the network by moving in a direction opposite to that of the gradient of the error function. However, BP has certain limitations. It has a tendency to converge toward the local optima, as it is good only at exploiting the current solution, which may result in unsatisfactory classification accuracies. It also has slow convergence as well as scaling problems [54]. To overcome these problems, many improvements of BP such as quickprop [8], RPROP [9], and improved BP [10] have been proposed by researchers in the past. Besides, conjugate gradient methods [11] and other derivative-based conventional methods such as Levenberg–Marquardt method [12] are also used for weight optimization, but sometimes these methods can be expensive. Conventional methods are computationally faster as compared to their metaheuristic counterparts because they operate on a single solution; however, they have certain limitations as discussed above.Due to the global search capabilities of metaheuristic algorithms, they are being widely used by researchers to generate optimal weights and biases in MLP. In [13–21], GA was applied to train MLP, and its performance was compared to BP. Valian et al. [22] proposed an improved cuckoo search (ICS) to train MLFNN. Unlike cuckoo search, the proposed ICS tunes the parameters of CS. The performances of ICS and CS are compared on two datasets. A number of approaches have been proposed by researchers to train MLFNN using differential evolution (DE) and evolutionary strategies [23–26]. Apart from EA, bioinspired algorithms and their variants are proposed and used by researchers to generate optimal connection weights in MLFNN. Karaboga et al. [27] applied artificial bee colony (ABC) algorithm to train MLFNN and compared the performance of ABC with that of GA. In [28], multimean particle swarm optimization (MMPSO) is proposed by the authors to generate optimal connection weights of MLFNN. MMPSO is derived from PSO, and unlike PSO it uses multiple swarms. The performance of MMPSO is compared with PSO on 10 datasets, and the results prove the effectiveness of MMPSO. In [29], krill herd algorithm (KHA) is applied to train ANN and is compared with BP, GA, and harmony search (HS). Bolaji et al. [30] and Kattan et al. [31] used fireworks algorithm (FWA) and HS, respectively, to train ANN. Mirjalili [32] applied gray wolf optimizer (GWO) to train MLP, and the comparison results on 8 datasets show the GWO algorithm’s capability of avoiding local optima. Aljarah et al. [33] applied a whale optimization algorithm (WOA) to generate an optimal set of connection weights in MLP. The performance of the proposed WOA-based trainer is evaluated on 20 datasets by comparing it with the trainers obtained using ant colony optimization (ACO), GA, PSO, DE, ES, population-based incremental learning (PBIL), and BP. The results indicate that WOA-based trainer avoids premature convergence and generates the best optimal weights in most of the cases for binary pattern classification. In [34], nature-inspired multiverse optimizer is used to train MLP. Heidari et al. [35], proposed GOAMLP that uses GOA to train single hidden layer MLP and is applied on five datasets. When compared with state-of-the-art algorithms, MLP trained using GOAMLP resulted in improved classification accuracy. Elakkiya and Selvakumar [36] used enhanced step size firefly algorithm to generate optimal weights of feedforward neural network for spam detection. In [37], adaptive GA has been proposed for weight optimization of BPNN for capacitive accelerometers. The optimized BPNN is used in the capacitive accelerometer.Sometimes, metaheuristic algorithms suffer from premature convergence. To overcome the problems faced by conventional methods and metaheuristic algorithms, hybrid approaches were proposed by researchers. In [38, 39], GA and PSO, respectively, have been combined with BP, which helped in fast convergence and avoidance of getting trapped in local optima. In [40], a hybrid approach that combines PSO and gravitational search algorithm is presented to train feedforward networks. In [41], a hybrid training algorithm, LPSONS, is proposed to train feedforward neural networks. It combines the velocity operator of PSO with Mantegna Levy distribution to increase the diversity of the population. To avoid local optima and premature convergence, Mantegna Levy distribution is further combined with neighborhood search. In [20], an improved GA coupled with BP neural network (IGA-BPNN) is proposed to improve the forecast performance of ANN. This model uses improved genetic adaptive strategies to avoid getting stuck in local optima. The experimental results show that IGA-BPNN performs better than traditional GA-BPNN. In [42], a hybrid algorithm, namely, constriction coefficient-based particle swarm optimization and gravitational search algorithm (CPSOGSA), is proposed to train MLP. It helps to avoid premature convergence and getting stuck in local optima problems of MLP. In [43], an optimized adaptive GA in the backpropagation neural network (OAGA-BPNN) is proposed to optimize BPNN for traffic flow prediction. In [44], a hybrid grasshopper and new cat swarm optimization algorithm was proposed for feature selection and weight and architecture optimization of MLP. In a similar way, other optimization approaches are also discussed by various researchers like MLP-LOA [55], improved teaching learning (TLB), and cat swarm optimization to get better results in respect of similar applications [56, 57].


## 3. Genetic Algorithm

Genetic algorithm (GA) is a metaheuristic algorithm proposed by Holland [58]. This algorithm imitates the process of natural selection where the chances of survival of fitter individuals are more as compared to other individuals in a competing environment. It is a global search technique characterized by evolution in every generation. GA starts with a randomly generated initial population of chromosomes where each chromosome represents a possible solution to the given problem. Each chromosome is associated with a ﬁtness value that is a measure of how good a solution is for the given problem. In each generation, the population evolves toward better fitness using evolutionary operators such as selection, crossover, and mutation. This process continues until a solution is found or the maximum number of iterations is reached.


## 4. Proposed Model: GGA-MLP

In this section, we present our proposed approach GGA-MLP which applies a greedy GA to generate an optimal set of synaptic weights and biases of MLP, keeping the architecture and activation function fixed. The various steps of GGA-MLP are explained below.


### 4.1. Representation of Candidate Solutions and Fitness Function

An important aspect that needs to be considered during the design of GGA-MLP is the representation of the possible solutions in the search space in the form of chromosomes and the encoding scheme used to encode the chromosomes. In GGA-MLP, each chromosome represents a candidate MLP. A chromosome is basically divided into different segments, where each segment contains the encoded weights between two layers (input-hidden, hidden-hidden (if any), hidden-output) and the last segment contains the encoded bias values for the MLP. Chromosome encoding for an MLP having two hidden layers is shown in Figure1. However, the length of the chromosome can easily be changed to train MLP having one or more hidden layers. A real value encoding scheme is used to encode the chromosomes.Figure 1
Chromosome encoding of MLP for weight and bias optimization.As it is clear from Figure1, if there are n input nodes, m hidden layers with h1,h2,…..,hm hidden nodes in each hidden layer, and k output nodes, then the length of the chromosome will be calculated using(1)Clength=n×h1+∑i=1m−1hi×hi+1+hm×k+∑i=1mhi+k.Each chromosome in the population is represented byMLPj|1≤j≤PS where PS is the population size. Each MLPj|1≤j≤PS in the population is associated with a fitness value which is the measure of its quality. In our case, mean square error (MSE) is chosen as the fitness function. To calculate the fitness of an MLP, the training data sample is made to run on it and the mean square error value is calculated using(2)FitnessMLP=MSE=1n∑k=1nyk−y^k2,where yk is the actual output, y^k is the predicted output, and n is the number of samples in the training dataset. This process is repeated for each MLPj. The goal of GGA is to find an MLP that minimizes the objective function f|f:MLPj⟶R+, where R+ represents a set of real numbers. The objective function f can be calculated using (3), and it tells us about the quality of the solution.(3)fMLPj=fitnessMLPj.Now, GGA tries to find the best MLP that minimizes the objective functionf as shown in(4)MLPbest=MLPl|fMLPl<fMLPj∀j∧j≠l.


### 4.2. Generation of Initial Population

In evolutionary algorithms (EAs), the initial population plays a major role in determining the quality of the final solution as well as the convergence speed [59]. Several population initialization methods exist in literature, but in most cases, the initial population is generated randomly. However, due to the dependence of the final solution’s quality on the initial population, GGA uses a greedy population initialization method that uses domain-specific knowledge to generate good quality MLPs (chromosomes). Initially, the synaptic weights and biases are chosen randomly in the interval [−2, 2]. After this, GGA analyzes the features of the dataset on which the MLP needs to be trained. In most cases, it has been observed that certain features contribute more than others to determining the correct class of the input pattern. GGA exploits this property of the dataset and finds important features using domain-specific knowledge. The weights of these identified features are increased by a random number in the interval [0.0,1.0) in the entire initial population, thereby giving them a higher weightage as compared to other features from the very beginning.


### 4.3. Mean-Based Crossover (MBC)

After population initialization, the next step is the application of various operators such as selection, crossover, and mutation repeatedly to obtain an MLP with optimal weights and biases. Maintaining diversity is important, but sometimes it is also vital to retain the best individuals of one generation into the next. GGA-MLP uses elitism to the transfer best chromosome(s) from one generation to another. Crossover and mutation are performed to generate offspring by selecting chromosomes from the current generation. The crossover operator takes two chromosomes and combines them to produce new offsprings. It is based on the idea that the exchange of information between good chromosomes will generate even better offsprings. Extreme care should be taken while performing selection and crossover operation as it may reduce the genetic diversity, which may ultimately lead to premature convergence. To avoid premature convergence, we present a crossover technique, known as mean-based crossover (MBC), that aims at improving the fitness of the top individuals of the population with the help of the worst members of the population. The proposed crossover technique involves the calculation of the mean of the fittest chromosomes in the population, thereby generating offsprings that are closer to the solution having minimum losses. Before applying MBC, GGA-MLP sorts the chromosomes in ascending order based on their fitness values. MBC starts by selecting the top 30% of the chromosomes and calculates the gene-wise mean of these chromosomes. The mean chromosomeCmean is an indicator of the ideal gene value which minimizes the MSE. In order to move toward a global optimum, this mean chromosome is used as a comparison parameter against individuals having low fitness values in the population. From the top 30% chromosomes, a chromosome Cbf is selected randomly for crossover. Another parent Chs for crossover is chosen from the worst 30% individuals in such a way that it can contribute the most toward the fitness of chromosome Cbf. The method of selection of Chs is shown in Figure 2. After selecting Cbf and Chs, MBC is performed by exchanging the genes of Cbf and Chs as shown in Figure 2. Out of the two children obtained from MBC, the offspring having higher fitness improves the quality of the population. The other offspring adds randomness to the population, thereby decreasing the probability of the population converging to a local optimum.Figure 2
Pseudocode for MBC.After crossover,Cbf is inserted into a set S to prevent it from being selected again for MBC in the current iteration. This is done to ensure that a unique chromosome is selected from the population each time MBC is performed, thereby preventing the problem of generating duplicate children. This process continues till the desired number of offsprings is generated. The steps of MBC are shown in Figure 2.


### 4.4. Greedy Mutation

In GA, the mutation operator is vital for maintaining diversity in the population. Mutation operator introduces diversity in the evolving population. It randomly modifies one or more genes of a chromosome depending upon the mutation probability which avoids getting stuck in the local minima. In traditional GA, every chromosome has an equal probability of getting mutated irrespective of its fitness [60]. It means both the best and the worst chromosomes have an equal probability of getting disrupted by mutation. In this paper, we propose a greedy mutation that aims to (i) avoid disruption of good quality chromosomes and (ii) at the same time maintain diversity in the population by mutating low-quality chromosomes, thereby improving the quality of the overall population.Greedy mutation starts by calculating the gene-wise mean of the top 30% (N) chromosomes to generate a mean chromosomeCmean. It then selects a chromosome Cj randomly from the worst 30% (M) chromosomes in the population for mutation. A random number R is generated for every gene of Cj and is compared with the mutation probability Pm. If R>Pm, difference “d” between the value of the selected gene of Cj and that of the corresponding gene of the mean chromosome is calculated, and a random number “r” is generated. The product of r and d is then subtracted from the corresponding gene value in Cj. This helps the chromosome in approaching good gene values, thereby increasing its overall fitness.Due to the use of greedy approaches at each step, diversity of the population may decrease leading to premature convergence. To avoid this, it is important to introduce diversity in the population. GGA-MLP introduces diversity in the population in each iteration by generating 30% of the population using elitism, 50% of the population using MBC and greedy mutation, and the remaining 20% randomly by choosing synaptic weights and biases within the range [−2, 2].


## 4.1. Representation of Candidate Solutions and Fitness Function

An important aspect that needs to be considered during the design of GGA-MLP is the representation of the possible solutions in the search space in the form of chromosomes and the encoding scheme used to encode the chromosomes. In GGA-MLP, each chromosome represents a candidate MLP. A chromosome is basically divided into different segments, where each segment contains the encoded weights between two layers (input-hidden, hidden-hidden (if any), hidden-output) and the last segment contains the encoded bias values for the MLP. Chromosome encoding for an MLP having two hidden layers is shown in Figure1. However, the length of the chromosome can easily be changed to train MLP having one or more hidden layers. A real value encoding scheme is used to encode the chromosomes.Figure 1
Chromosome encoding of MLP for weight and bias optimization.As it is clear from Figure1, if there are n input nodes, m hidden layers with h1,h2,…..,hm hidden nodes in each hidden layer, and k output nodes, then the length of the chromosome will be calculated using(1)Clength=n×h1+∑i=1m−1hi×hi+1+hm×k+∑i=1mhi+k.Each chromosome in the population is represented byMLPj|1≤j≤PS where PS is the population size. Each MLPj|1≤j≤PS in the population is associated with a fitness value which is the measure of its quality. In our case, mean square error (MSE) is chosen as the fitness function. To calculate the fitness of an MLP, the training data sample is made to run on it and the mean square error value is calculated using(2)FitnessMLP=MSE=1n∑k=1nyk−y^k2,where yk is the actual output, y^k is the predicted output, and n is the number of samples in the training dataset. This process is repeated for each MLPj. The goal of GGA is to find an MLP that minimizes the objective function f|f:MLPj⟶R+, where R+ represents a set of real numbers. The objective function f can be calculated using (3), and it tells us about the quality of the solution.(3)fMLPj=fitnessMLPj.Now, GGA tries to find the best MLP that minimizes the objective functionf as shown in(4)MLPbest=MLPl|fMLPl<fMLPj∀j∧j≠l.


## 4.2. Generation of Initial Population

In evolutionary algorithms (EAs), the initial population plays a major role in determining the quality of the final solution as well as the convergence speed [59]. Several population initialization methods exist in literature, but in most cases, the initial population is generated randomly. However, due to the dependence of the final solution’s quality on the initial population, GGA uses a greedy population initialization method that uses domain-specific knowledge to generate good quality MLPs (chromosomes). Initially, the synaptic weights and biases are chosen randomly in the interval [−2, 2]. After this, GGA analyzes the features of the dataset on which the MLP needs to be trained. In most cases, it has been observed that certain features contribute more than others to determining the correct class of the input pattern. GGA exploits this property of the dataset and finds important features using domain-specific knowledge. The weights of these identified features are increased by a random number in the interval [0.0,1.0) in the entire initial population, thereby giving them a higher weightage as compared to other features from the very beginning.


## 4.3. Mean-Based Crossover (MBC)

After population initialization, the next step is the application of various operators such as selection, crossover, and mutation repeatedly to obtain an MLP with optimal weights and biases. Maintaining diversity is important, but sometimes it is also vital to retain the best individuals of one generation into the next. GGA-MLP uses elitism to the transfer best chromosome(s) from one generation to another. Crossover and mutation are performed to generate offspring by selecting chromosomes from the current generation. The crossover operator takes two chromosomes and combines them to produce new offsprings. It is based on the idea that the exchange of information between good chromosomes will generate even better offsprings. Extreme care should be taken while performing selection and crossover operation as it may reduce the genetic diversity, which may ultimately lead to premature convergence. To avoid premature convergence, we present a crossover technique, known as mean-based crossover (MBC), that aims at improving the fitness of the top individuals of the population with the help of the worst members of the population. The proposed crossover technique involves the calculation of the mean of the fittest chromosomes in the population, thereby generating offsprings that are closer to the solution having minimum losses. Before applying MBC, GGA-MLP sorts the chromosomes in ascending order based on their fitness values. MBC starts by selecting the top 30% of the chromosomes and calculates the gene-wise mean of these chromosomes. The mean chromosomeCmean is an indicator of the ideal gene value which minimizes the MSE. In order to move toward a global optimum, this mean chromosome is used as a comparison parameter against individuals having low fitness values in the population. From the top 30% chromosomes, a chromosome Cbf is selected randomly for crossover. Another parent Chs for crossover is chosen from the worst 30% individuals in such a way that it can contribute the most toward the fitness of chromosome Cbf. The method of selection of Chs is shown in Figure 2. After selecting Cbf and Chs, MBC is performed by exchanging the genes of Cbf and Chs as shown in Figure 2. Out of the two children obtained from MBC, the offspring having higher fitness improves the quality of the population. The other offspring adds randomness to the population, thereby decreasing the probability of the population converging to a local optimum.Figure 2
Pseudocode for MBC.After crossover,Cbf is inserted into a set S to prevent it from being selected again for MBC in the current iteration. This is done to ensure that a unique chromosome is selected from the population each time MBC is performed, thereby preventing the problem of generating duplicate children. This process continues till the desired number of offsprings is generated. The steps of MBC are shown in Figure 2.


## 4.4. Greedy Mutation

In GA, the mutation operator is vital for maintaining diversity in the population. Mutation operator introduces diversity in the evolving population. It randomly modifies one or more genes of a chromosome depending upon the mutation probability which avoids getting stuck in the local minima. In traditional GA, every chromosome has an equal probability of getting mutated irrespective of its fitness [60]. It means both the best and the worst chromosomes have an equal probability of getting disrupted by mutation. In this paper, we propose a greedy mutation that aims to (i) avoid disruption of good quality chromosomes and (ii) at the same time maintain diversity in the population by mutating low-quality chromosomes, thereby improving the quality of the overall population.Greedy mutation starts by calculating the gene-wise mean of the top 30% (N) chromosomes to generate a mean chromosomeCmean. It then selects a chromosome Cj randomly from the worst 30% (M) chromosomes in the population for mutation. A random number R is generated for every gene of Cj and is compared with the mutation probability Pm. If R>Pm, difference “d” between the value of the selected gene of Cj and that of the corresponding gene of the mean chromosome is calculated, and a random number “r” is generated. The product of r and d is then subtracted from the corresponding gene value in Cj. This helps the chromosome in approaching good gene values, thereby increasing its overall fitness.Due to the use of greedy approaches at each step, diversity of the population may decrease leading to premature convergence. To avoid this, it is important to introduce diversity in the population. GGA-MLP introduces diversity in the population in each iteration by generating 30% of the population using elitism, 50% of the population using MBC and greedy mutation, and the remaining 20% randomly by choosing synaptic weights and biases within the range [−2, 2].


## 5. Results and Discussion

First, we present the datasets that are selected to evaluate the effectiveness of GGA-MLP, in terms of accuracy achieved in classifying the input data, in Section5.1. The implementation details, experimental setup used for performing experiments, and results are presented in Section 5.2.


### 5.1. Datasets

To evaluate the effectiveness of the proposed approach GGA-MLP, ten standard binary classification datasets are selected from the UCI Machine Learning Repository [61]: Parkinson, Indian Liver Patient Dataset (ILPD), Diabetes, Vertebral Column, Spambase, QSAR Biodegradation, Blood Transfusion, HTRU2, Drug Consumption: Amyl Nitrite, and Drug Consumption: Ketamine. The description of the selected datasets is shown in Table 1. In each dataset, 80% of the instances are used for training (out of which 20% is used for validation), and the remaining 20% are used for testing. It can easily be seen from Table 1 that the selected datasets have different numbers of features ranging from 4 to 57 as well as instances ranging from 197 to 17898, which helps us to evaluate the proposed approach on datasets of varying complexities. It also makes the task of evaluating GGA-MLP even more challenging.Table 1
Binary classification dataset.
#Dataset# features# instances# training instances# testing instances1Parkinson22195158372ILPD55834671163Diabetes87686151534Vertebral Column6310248625Spambase57460136819206QSAR Biodegradation4110558442117Blood Transfusion47485981508HTRU28178981431835809Drug Consumption: Amyl Nitrite121885150837710Drug Consumption: Ketamine1218851508377


### 5.2. Experimental Design and Results

To evaluate the effectiveness of GGA-MLP, the performance of MLP trained using GGA-MLP is compared with the classification accuracy of MLP trained using existing algorithms, namely, GA [21], ABC [27], MMPSO [28], WOA [33], MVO [34], and GOA [35], and on each dataset given in Table 1. All the algorithms are implemented in Python 3.6.4 using the Anaconda framework. As these are randomized algorithms, 30 runs of each algorithm are performed on every dataset. After each run, the best MLP is selected, and its classification accuracy on the test dataset is calculated using(5)ClassificationAccuracy=NCN,where NC is the number of correctly classified testing data samples and N is the total number of samples in the testing dataset.Before the start of the training phase, it is required to decide the architecture of MLP for each dataset. To perform a fair comparison, the architecture of MLP is kept the same for each algorithm. Here, we take only one hidden layer, as one hidden layer is sufficient to classify the datasets shown in Table1. The number of neurons in the hidden layer is decided by using the method proposed by [25]. The number of neurons in the hidden layer is calculated using the formula 2×N+1, where N is number of relevant features of the dataset. In some cases, the number of hidden neurons taken is 5×N+1. The architecture of MLP used for each dataset is shown in Table 2.Table 2
MLP architecture and selected attributes for each dataset.
#DatasetMLP architecture (input-hidden-output)Selected attributes for greedy population initialization1Parkinson22-111-1Average vocal fundamental frequency, minimum vocal fundamental frequency, maximum vocal fundamental frequency2ILPD5-26-1SGPT alanine aminotransferase, SGOT aspartate aminotransferase, direct bilirubin3Diabetes8-17-1Blood pressure, insulin, BMI4Vertebral Column6-31-1Lumbar lordosis angle, sacral slope, pelvic radius5Spambase57-115-1Length of the longest uninterrupted sequence of capital letters, average length of uninterrupted sequences of capital letters, total number of capital letters in the e-mail6QSAR Biodegradation41-83-1SpMax_L: leading eigenvalue from Laplace matrix, J_Dz(e): Balaban-like index from Barysz matrix weighted by Sanderson electronegativity7Blood Transfusion4-9-1R (recency - months since last donation), F (frequency - total number of donation), M (monetary - total blood donated in c.c.)8HTRU28-17-1Mean of the DM-SNR curve, standard deviation of the DM-SNR curve, excess kurtosis of the DM-SNR curve, skewness of the DM-SNR curve9Drug Consumption: Amyl Nitrite12-25-1Nscore, Escore, Oscore, Ascore, Cscore10Drug Consumption: Ketamine12-25-1Nscore, Escore, Oscore, Ascore, CscoreThe values of the controlling parameters of ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP are listed in Table3. Various performance metrics such as classification accuracy, specificity, and sensitivity are used to assess the performance of GGA-MLP with respect to the existing state-of-the-art algorithms. The average, best, and standard deviation of classification accuracy, specificity, and sensitivity of the best MLP trained using these metaheuristic algorithms during 30 runs for the given datasets are shown in Tables 4–6, respectively. Data is collected under Windows 10 on Intel core i5-7200U 3.1 GHz processor with 8.00 GB DDR4 and Nvidia GT 940MX 2 GB VRAM.Table 3
Controlling parameters of metaheuristic algorithms.
Optimization algorithmParameterValueGGA-MLP, GA, ABC, WOA, MMPSO, MVO, GOAInitial population size30# iterations200GA, ABC, WOA, MMPSO, MVO, GOAInitial population generationRandom population initialization (synaptic weights and biases are initialized randomly in the range [−2, 2])GGA-MLPInitial population generationGreedy population initializationGA, GGA-MLPProbability of crossoverPc0.8Probability of mutationPm0.05Elitism30%ABCRandom number (ɸ)[−1, 1]WOAVectora⟶Linearly decreasing from 2 to 0Random vectorr⟶[0, 1]Constant (b)1Random number (p)[0, 1]Random number (l)[−1, 1]MMPSOAcceleration coefficientsc1,c21.48Inertia weights (w)0.729Number of swarms3Swarm size10MVOMinimum wormhole existence probability0.2Minimum wormhole existence probability1GOAcmax1cmin0.00001L1.5F0.5Table 4
Comparison results of classification accuracy of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP.
Dataset\algorithmABC [27]WOA [33]MMPSO [28]MVO [34]GOA [35]GA [21]GGA-MLPParkinsonAvg0.80420.75380.74810.90230.92940.82090.8666Std0.0620.09550.06090.06670.05310.06700.0772Best0.91890.86490.83780.94590.97300.89190.9189ILPDAvg0.69910.70090.70060.70000.71410.70180.7196Std0.00470.00000.00160.00210.02010.02080.0302Best0.70090.70090.70090.70940.76070.72650.7863DiabetesAvg0.74160.72460.66470.75680.75760.75630.7697Std0.03170.02960.06150.02630.02850.02670.0255Best0.80390.75160.76470.78430.77780.79740.8301Vertebral ColumnAvg0.83760.85040.79540.84530.86310.83170.8851Std0.06320.04720.06390.03890.03530.04670.0365Best0.88710.90320.83870.87100.87100.90320.9355SpambaseAvg0.76390.74910.82020.84320.85640.84650.9176Std0.02950.04090.02790.03270.02390.02140.0176Best0.79350.81520.85330.87610.91200.90220.9370QSAR BiodegradationAvg0.71660.73290.72160.75640.86560.80490.8498Std0.06980.07770.05630.05380.03020.03360.0422Best0.81040.82460.78670.81040.90520.84830.8957Blood TransfusionAvg0.76130.78000.77360.79760.83450.83420.8479Std0.035030.005260.02100.0050.00450.00590.0039Best0.78000.78670.79330.81330.85330.84000.8667HTRU2Avg0.94540.96780.97470.96560.97860.97990.9805Std0.02250.00370.00150.00140.00220.00120.0010Best0.97010.97630.97790.97120.98100.98160.9827Drug Consumption: Amyl NitriteAvg0.82980.83630.83590.81680.82420.81980.8356Std0.02190.00130.00870.02020.04100.01130.0012Best0.84350.84880.84620.83290.85410.83550.8462Drug Consumption: KetamineAvg0.82430.82160.82210.80320.82680.81110.8204Std0.01620.01210.01290.02640.03010.01190.0116Best0.84620.84350.84620.82760.85410.82760.8382Table 5
Comparison results of specificity of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP.
Dataset\algorithmABC [27]WOA [33]MMPSO [28]MVO [34]GOA [35]GA [21]GGA-MLPParkinsonAvg0.79140.67340.75830.84600.81260.78650.7914Std0.08050.08450.06320.06680.07980.05210.0601Best0.93100.82760.86210.96550.96550.86210.8966ILPDAvg0.45700.42220.45650.41260.42180.35460.4788Std0.00640.00000.00310.00400.00830.00920.001Best0.48890.42220.46670.42220.44440.37780.4889DiabetesAvg0.69340.65210.59870.69650.70120.70510.7124Std0.06210.02640.07430.00860.00430.02180.0723Best0.76700.69900.71840.71840.71840.74760.7767Vertebral ColumnAvg0.80260.81260.79620.82640.85560.84320.8643Std0.07560.03160.05280.02540.04040.0310.0264Best0.88890.85190.85190.85190.88890.88890.8889SpambaseAvg0.66530.65210.75870.78650.79170.76640.8942Std0.00850.03210.02620.03220.08530.09120.0010Best0.70400.72910.79430.82270.87630.86290.9064QSAR BiodegradationAvg0.65140.69540.72560.73810.82160.70210.8126Std0.09540.07320.00630.00570.01560.03780.0054Best0.78260.78990.76810.76810.87680.78260.8478Blood TransfusionAvg0.72650.73550.75430.76320.81230.80110.8268Std0.07650.00320.00820.00280.00140.00320.0021Best0.76530.75510.79590.78570.82650.82650.8367HTRU2Avg0.94520.95420.97020.96780.97210.97240.9742Std0.00310.00260.00160.00150.00180.00150.0014Best0.97240.97710.98020.97120.98140.98200.9824Drug Consumption: Amyl NitriteAvg0.81650.83210.82560.79820.79200.78140.7925Std0.00760.00210.00620.00750.01680.00620.0047Best0.85170.84860.84540.82650.84230.82970.8297Drug Consumption: KetamineAvg0.89450.92070.91120.87650.89650.86430.8621Std0.03220.00510.04430.05560.00650.02340.0186Best0.93660.94370.94010.91200.91900.89440.8944Table 6
Comparison results of sensitivity of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP.
Dataset\algorithmABC [27]WOA [33]MMPSO [28]MVO [34]GOA [35]GA [21]GGA-MLPParkinsonAvg0.08640.98210.07150.84760.98200.97760.9845Std0.00280.00320.00720.00680.00320.00360.0021Best0.8751.00000.75000.8751.00001.00001.0000ILPDAvg0.79450.85200.80250.87680.93270.91220.9543Std0.00940.00460.00690.00380.00290.00370.0025Best0.83330.87500.84720.88890.95830.94440.9722DiabetesAvg0.83560.84320.79650.89760.88980.86570.9316Std0.04120.01420.06840.00020.00020.00530.0001Best0.88000.86000.86000.92000.90000.90000.94Vertebral ColumnAvg0.82560.92060.76810.85420.83210.87650.9543Std0.02090.00510.05640.00340.00320.00260.0010Best0.88570.94290.82860.88570.85710.91430.9714SpambaseAvg0.86750.93420.91250.94300.95320.93550.9785Std0.06220.03320.02040.00310.00250.00780.0021Best0.95960.97520.96270.97520.97830.97520.9938QSAR BiodegradationAvg0.72170.85360.76210.87650.93320.94520.9720Std0.07210.05010.04200.00280.00240.00320.0001Best0.86300.89040.82190.89040.95890.97260.9863Blood TransfusionAvg0.85750.89210.85430.89650.93290.90560.9486Std0.00220.00020.00360.00030.00020.00040.0002Best0.88240.90240.87640.91670.94190.92050.9535HTRU2Avg0.89700.94320.91230.95470.95890.95460.9698Std0.01070.00370.00680.00410.00280.00300.0026Best0.94860.96860.95710.97140.97710.97710.9857Drug Consumption: Amyl NitriteAvg0.78860.82220.79860.84430.90650.82450.9124Std0.03100.02300.08650.00200.00030.00480.0022Best0.800.850.850.86670.91670.86670.9333Drug Consumption: KetamineAvg0.75890.72340.73080.77650.90100.84380.9216Std0.00760.00370.00830.00450.00370.00430.0010Best0.80300.75760.78790.80300.92420.87880.9394It is evident from Tables4 and 5 that GGA-MLP gives the highest average and best accuracy as well as specificity for the datasets except Parkinson, QSAR Biodegradation, Drug Consumption: Amyl Nitrite, and Drug Consumption: Ketamine. Despite having low accuracies and specificity on the four datasets, GGA-MLP achieves higher sensitivity as compared to the existing algorithms, as evident from Table 6, which shows the superiority of GGA-MLP in classifying the positive samples correctly. GGA-MLP also has low standard deviation as compared to existing state-of-the-art algorithms. This shows the robustness of the proposed approach.In Figure3, MSE values of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP for the given datasets are calculated at an interval of 10 iterations and plotted to visualize convergence rate. The convergence curves show that although GGA-MLP takes more time to converge as compared to other metaheuristic algorithms, it avoids getting trapped in local minima. In most of the cases, the performance of GGA-MLP is better than the existing algorithms. To assess the efficacy of MLP trained using GGA-MLP as a classifier, we compare the classification accuracy of GGA-MLP with that of the classifiers built using other machine learning algorithms such as logistic regression, Naïve Bayes, and decision tree, as well as the MLP trained using BP. Similar to decision tree algorithms, BP algorithms like GGA-MLP are also randomized algorithms; every dataset is run 30 times on each of them, and the average, best, and standard deviation of classification accuracy are reported in Table 7. To prevent overfitting, validation set is used for early stopping during training of logistic regression, Naïve Bayes, and decision tree as well as the MLP using BP. It is clear from Table 7 that GGA-MLP gives the best result in all the cases. However, the standard deviation over 30 runs is least in case of decision tree. From Tables 4–7, it is clear that GGA-MLP performance is better than or comparable to the existing algorithms in classifying input patterns correctly.Figure 3
Convergence graph of MSE of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP for 10 classification datasets.
(a)(b)Table 7
Comparison of performances of classifiers.
Dataset\algorithmLogistic regressionNaïve BayesDecision treeBPGGA-MLPParkinsonAvg0.82050.61540.81620.84960.8666Std——0.03160.00890.0772Best0.82050.61540.87180.87180.9189ILPDAvg0.70940.51280.67860.71790.7196Std——0.0203—0.0302Best0.70940.51280.71790.71790.7863DiabetesAvg0.74030.75970.65410.75580.7697Std0.01130.01220.0255Best0.74030.75970.68180.78570.8301Vertebral ColumnAvg0.80650.70970.63980.85000.8851Std——0.01910.01280.0365Best0.80650.70970.67740.88710.9355SpambaseAvg0.90540.82390.90920.92410.9176Std——0.00400.00220.0176Best0.90540.82390.91850.92830.9370QSAR BiodegradationAvg0.84360.66820.80520.85610.8498Std——0.01070.01050.0422Best0.84360.66820.82460.87200.8957Blood TransfusionAvg0.78670.77330.82670.76000.8479Std———0.00630.0039Best0.78670.77330.82670.77330.8667HTRU2Avg0.97650.94110.96240.97030.9805Std——0.00130.00140.0010Best0.97650.94110.96450.97240.9827Drug Consumption: Amyl NitriteAvg0.79310.69760.69430.79260.8356Std——0.00880.00980.0012Best0.79310.69760.71350.80900.8462Drug Consumption: KetamineAvg0.79310.74540.70800.77240.8204Std——0.02430.02160.0116Best0.79310.74540.72680.80110.8382


## 5.1. Datasets

To evaluate the effectiveness of the proposed approach GGA-MLP, ten standard binary classification datasets are selected from the UCI Machine Learning Repository [61]: Parkinson, Indian Liver Patient Dataset (ILPD), Diabetes, Vertebral Column, Spambase, QSAR Biodegradation, Blood Transfusion, HTRU2, Drug Consumption: Amyl Nitrite, and Drug Consumption: Ketamine. The description of the selected datasets is shown in Table 1. In each dataset, 80% of the instances are used for training (out of which 20% is used for validation), and the remaining 20% are used for testing. It can easily be seen from Table 1 that the selected datasets have different numbers of features ranging from 4 to 57 as well as instances ranging from 197 to 17898, which helps us to evaluate the proposed approach on datasets of varying complexities. It also makes the task of evaluating GGA-MLP even more challenging.Table 1
Binary classification dataset.
#Dataset# features# instances# training instances# testing instances1Parkinson22195158372ILPD55834671163Diabetes87686151534Vertebral Column6310248625Spambase57460136819206QSAR Biodegradation4110558442117Blood Transfusion47485981508HTRU28178981431835809Drug Consumption: Amyl Nitrite121885150837710Drug Consumption: Ketamine1218851508377


## 5.2. Experimental Design and Results

To evaluate the effectiveness of GGA-MLP, the performance of MLP trained using GGA-MLP is compared with the classification accuracy of MLP trained using existing algorithms, namely, GA [21], ABC [27], MMPSO [28], WOA [33], MVO [34], and GOA [35], and on each dataset given in Table 1. All the algorithms are implemented in Python 3.6.4 using the Anaconda framework. As these are randomized algorithms, 30 runs of each algorithm are performed on every dataset. After each run, the best MLP is selected, and its classification accuracy on the test dataset is calculated using(5)ClassificationAccuracy=NCN,where NC is the number of correctly classified testing data samples and N is the total number of samples in the testing dataset.Before the start of the training phase, it is required to decide the architecture of MLP for each dataset. To perform a fair comparison, the architecture of MLP is kept the same for each algorithm. Here, we take only one hidden layer, as one hidden layer is sufficient to classify the datasets shown in Table1. The number of neurons in the hidden layer is decided by using the method proposed by [25]. The number of neurons in the hidden layer is calculated using the formula 2×N+1, where N is number of relevant features of the dataset. In some cases, the number of hidden neurons taken is 5×N+1. The architecture of MLP used for each dataset is shown in Table 2.Table 2
MLP architecture and selected attributes for each dataset.
#DatasetMLP architecture (input-hidden-output)Selected attributes for greedy population initialization1Parkinson22-111-1Average vocal fundamental frequency, minimum vocal fundamental frequency, maximum vocal fundamental frequency2ILPD5-26-1SGPT alanine aminotransferase, SGOT aspartate aminotransferase, direct bilirubin3Diabetes8-17-1Blood pressure, insulin, BMI4Vertebral Column6-31-1Lumbar lordosis angle, sacral slope, pelvic radius5Spambase57-115-1Length of the longest uninterrupted sequence of capital letters, average length of uninterrupted sequences of capital letters, total number of capital letters in the e-mail6QSAR Biodegradation41-83-1SpMax_L: leading eigenvalue from Laplace matrix, J_Dz(e): Balaban-like index from Barysz matrix weighted by Sanderson electronegativity7Blood Transfusion4-9-1R (recency - months since last donation), F (frequency - total number of donation), M (monetary - total blood donated in c.c.)8HTRU28-17-1Mean of the DM-SNR curve, standard deviation of the DM-SNR curve, excess kurtosis of the DM-SNR curve, skewness of the DM-SNR curve9Drug Consumption: Amyl Nitrite12-25-1Nscore, Escore, Oscore, Ascore, Cscore10Drug Consumption: Ketamine12-25-1Nscore, Escore, Oscore, Ascore, CscoreThe values of the controlling parameters of ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP are listed in Table3. Various performance metrics such as classification accuracy, specificity, and sensitivity are used to assess the performance of GGA-MLP with respect to the existing state-of-the-art algorithms. The average, best, and standard deviation of classification accuracy, specificity, and sensitivity of the best MLP trained using these metaheuristic algorithms during 30 runs for the given datasets are shown in Tables 4–6, respectively. Data is collected under Windows 10 on Intel core i5-7200U 3.1 GHz processor with 8.00 GB DDR4 and Nvidia GT 940MX 2 GB VRAM.Table 3
Controlling parameters of metaheuristic algorithms.
Optimization algorithmParameterValueGGA-MLP, GA, ABC, WOA, MMPSO, MVO, GOAInitial population size30# iterations200GA, ABC, WOA, MMPSO, MVO, GOAInitial population generationRandom population initialization (synaptic weights and biases are initialized randomly in the range [−2, 2])GGA-MLPInitial population generationGreedy population initializationGA, GGA-MLPProbability of crossoverPc0.8Probability of mutationPm0.05Elitism30%ABCRandom number (ɸ)[−1, 1]WOAVectora⟶Linearly decreasing from 2 to 0Random vectorr⟶[0, 1]Constant (b)1Random number (p)[0, 1]Random number (l)[−1, 1]MMPSOAcceleration coefficientsc1,c21.48Inertia weights (w)0.729Number of swarms3Swarm size10MVOMinimum wormhole existence probability0.2Minimum wormhole existence probability1GOAcmax1cmin0.00001L1.5F0.5Table 4
Comparison results of classification accuracy of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP.
Dataset\algorithmABC [27]WOA [33]MMPSO [28]MVO [34]GOA [35]GA [21]GGA-MLPParkinsonAvg0.80420.75380.74810.90230.92940.82090.8666Std0.0620.09550.06090.06670.05310.06700.0772Best0.91890.86490.83780.94590.97300.89190.9189ILPDAvg0.69910.70090.70060.70000.71410.70180.7196Std0.00470.00000.00160.00210.02010.02080.0302Best0.70090.70090.70090.70940.76070.72650.7863DiabetesAvg0.74160.72460.66470.75680.75760.75630.7697Std0.03170.02960.06150.02630.02850.02670.0255Best0.80390.75160.76470.78430.77780.79740.8301Vertebral ColumnAvg0.83760.85040.79540.84530.86310.83170.8851Std0.06320.04720.06390.03890.03530.04670.0365Best0.88710.90320.83870.87100.87100.90320.9355SpambaseAvg0.76390.74910.82020.84320.85640.84650.9176Std0.02950.04090.02790.03270.02390.02140.0176Best0.79350.81520.85330.87610.91200.90220.9370QSAR BiodegradationAvg0.71660.73290.72160.75640.86560.80490.8498Std0.06980.07770.05630.05380.03020.03360.0422Best0.81040.82460.78670.81040.90520.84830.8957Blood TransfusionAvg0.76130.78000.77360.79760.83450.83420.8479Std0.035030.005260.02100.0050.00450.00590.0039Best0.78000.78670.79330.81330.85330.84000.8667HTRU2Avg0.94540.96780.97470.96560.97860.97990.9805Std0.02250.00370.00150.00140.00220.00120.0010Best0.97010.97630.97790.97120.98100.98160.9827Drug Consumption: Amyl NitriteAvg0.82980.83630.83590.81680.82420.81980.8356Std0.02190.00130.00870.02020.04100.01130.0012Best0.84350.84880.84620.83290.85410.83550.8462Drug Consumption: KetamineAvg0.82430.82160.82210.80320.82680.81110.8204Std0.01620.01210.01290.02640.03010.01190.0116Best0.84620.84350.84620.82760.85410.82760.8382Table 5
Comparison results of specificity of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP.
Dataset\algorithmABC [27]WOA [33]MMPSO [28]MVO [34]GOA [35]GA [21]GGA-MLPParkinsonAvg0.79140.67340.75830.84600.81260.78650.7914Std0.08050.08450.06320.06680.07980.05210.0601Best0.93100.82760.86210.96550.96550.86210.8966ILPDAvg0.45700.42220.45650.41260.42180.35460.4788Std0.00640.00000.00310.00400.00830.00920.001Best0.48890.42220.46670.42220.44440.37780.4889DiabetesAvg0.69340.65210.59870.69650.70120.70510.7124Std0.06210.02640.07430.00860.00430.02180.0723Best0.76700.69900.71840.71840.71840.74760.7767Vertebral ColumnAvg0.80260.81260.79620.82640.85560.84320.8643Std0.07560.03160.05280.02540.04040.0310.0264Best0.88890.85190.85190.85190.88890.88890.8889SpambaseAvg0.66530.65210.75870.78650.79170.76640.8942Std0.00850.03210.02620.03220.08530.09120.0010Best0.70400.72910.79430.82270.87630.86290.9064QSAR BiodegradationAvg0.65140.69540.72560.73810.82160.70210.8126Std0.09540.07320.00630.00570.01560.03780.0054Best0.78260.78990.76810.76810.87680.78260.8478Blood TransfusionAvg0.72650.73550.75430.76320.81230.80110.8268Std0.07650.00320.00820.00280.00140.00320.0021Best0.76530.75510.79590.78570.82650.82650.8367HTRU2Avg0.94520.95420.97020.96780.97210.97240.9742Std0.00310.00260.00160.00150.00180.00150.0014Best0.97240.97710.98020.97120.98140.98200.9824Drug Consumption: Amyl NitriteAvg0.81650.83210.82560.79820.79200.78140.7925Std0.00760.00210.00620.00750.01680.00620.0047Best0.85170.84860.84540.82650.84230.82970.8297Drug Consumption: KetamineAvg0.89450.92070.91120.87650.89650.86430.8621Std0.03220.00510.04430.05560.00650.02340.0186Best0.93660.94370.94010.91200.91900.89440.8944Table 6
Comparison results of sensitivity of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP.
Dataset\algorithmABC [27]WOA [33]MMPSO [28]MVO [34]GOA [35]GA [21]GGA-MLPParkinsonAvg0.08640.98210.07150.84760.98200.97760.9845Std0.00280.00320.00720.00680.00320.00360.0021Best0.8751.00000.75000.8751.00001.00001.0000ILPDAvg0.79450.85200.80250.87680.93270.91220.9543Std0.00940.00460.00690.00380.00290.00370.0025Best0.83330.87500.84720.88890.95830.94440.9722DiabetesAvg0.83560.84320.79650.89760.88980.86570.9316Std0.04120.01420.06840.00020.00020.00530.0001Best0.88000.86000.86000.92000.90000.90000.94Vertebral ColumnAvg0.82560.92060.76810.85420.83210.87650.9543Std0.02090.00510.05640.00340.00320.00260.0010Best0.88570.94290.82860.88570.85710.91430.9714SpambaseAvg0.86750.93420.91250.94300.95320.93550.9785Std0.06220.03320.02040.00310.00250.00780.0021Best0.95960.97520.96270.97520.97830.97520.9938QSAR BiodegradationAvg0.72170.85360.76210.87650.93320.94520.9720Std0.07210.05010.04200.00280.00240.00320.0001Best0.86300.89040.82190.89040.95890.97260.9863Blood TransfusionAvg0.85750.89210.85430.89650.93290.90560.9486Std0.00220.00020.00360.00030.00020.00040.0002Best0.88240.90240.87640.91670.94190.92050.9535HTRU2Avg0.89700.94320.91230.95470.95890.95460.9698Std0.01070.00370.00680.00410.00280.00300.0026Best0.94860.96860.95710.97140.97710.97710.9857Drug Consumption: Amyl NitriteAvg0.78860.82220.79860.84430.90650.82450.9124Std0.03100.02300.08650.00200.00030.00480.0022Best0.800.850.850.86670.91670.86670.9333Drug Consumption: KetamineAvg0.75890.72340.73080.77650.90100.84380.9216Std0.00760.00370.00830.00450.00370.00430.0010Best0.80300.75760.78790.80300.92420.87880.9394It is evident from Tables4 and 5 that GGA-MLP gives the highest average and best accuracy as well as specificity for the datasets except Parkinson, QSAR Biodegradation, Drug Consumption: Amyl Nitrite, and Drug Consumption: Ketamine. Despite having low accuracies and specificity on the four datasets, GGA-MLP achieves higher sensitivity as compared to the existing algorithms, as evident from Table 6, which shows the superiority of GGA-MLP in classifying the positive samples correctly. GGA-MLP also has low standard deviation as compared to existing state-of-the-art algorithms. This shows the robustness of the proposed approach.In Figure3, MSE values of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP for the given datasets are calculated at an interval of 10 iterations and plotted to visualize convergence rate. The convergence curves show that although GGA-MLP takes more time to converge as compared to other metaheuristic algorithms, it avoids getting trapped in local minima. In most of the cases, the performance of GGA-MLP is better than the existing algorithms. To assess the efficacy of MLP trained using GGA-MLP as a classifier, we compare the classification accuracy of GGA-MLP with that of the classifiers built using other machine learning algorithms such as logistic regression, Naïve Bayes, and decision tree, as well as the MLP trained using BP. Similar to decision tree algorithms, BP algorithms like GGA-MLP are also randomized algorithms; every dataset is run 30 times on each of them, and the average, best, and standard deviation of classification accuracy are reported in Table 7. To prevent overfitting, validation set is used for early stopping during training of logistic regression, Naïve Bayes, and decision tree as well as the MLP using BP. It is clear from Table 7 that GGA-MLP gives the best result in all the cases. However, the standard deviation over 30 runs is least in case of decision tree. From Tables 4–7, it is clear that GGA-MLP performance is better than or comparable to the existing algorithms in classifying input patterns correctly.Figure 3
Convergence graph of MSE of MLP trained using ABC, WOA, MMPSO, MVO, GOA, GA, and GGA-MLP for 10 classification datasets.
(a)(b)Table 7
Comparison of performances of classifiers.
Dataset\algorithmLogistic regressionNaïve BayesDecision treeBPGGA-MLPParkinsonAvg0.82050.61540.81620.84960.8666Std——0.03160.00890.0772Best0.82050.61540.87180.87180.9189ILPDAvg0.70940.51280.67860.71790.7196Std——0.0203—0.0302Best0.70940.51280.71790.71790.7863DiabetesAvg0.74030.75970.65410.75580.7697Std0.01130.01220.0255Best0.74030.75970.68180.78570.8301Vertebral ColumnAvg0.80650.70970.63980.85000.8851Std——0.01910.01280.0365Best0.80650.70970.67740.88710.9355SpambaseAvg0.90540.82390.90920.92410.9176Std——0.00400.00220.0176Best0.90540.82390.91850.92830.9370QSAR BiodegradationAvg0.84360.66820.80520.85610.8498Std——0.01070.01050.0422Best0.84360.66820.82460.87200.8957Blood TransfusionAvg0.78670.77330.82670.76000.8479Std———0.00630.0039Best0.78670.77330.82670.77330.8667HTRU2Avg0.97650.94110.96240.97030.9805Std——0.00130.00140.0010Best0.97650.94110.96450.97240.9827Drug Consumption: Amyl NitriteAvg0.79310.69760.69430.79260.8356Std——0.00880.00980.0012Best0.79310.69760.71350.80900.8462Drug Consumption: KetamineAvg0.79310.74540.70800.77240.8204Std——0.02430.02160.0116Best0.79310.74540.72680.80110.8382


## 6. Conclusion and Future Work

In this paper, a greedy genetic algorithm, GGA-MLP, is presented to train MLP. The use of domain-specific knowledge enables the generation of good quality initial population. Mean-based crossover and greedy mutation help algorithm in moving toward global optima by exploring the search space thoroughly. Datasets of varying complexities are used to evaluate the performance of GGA-MLP and to compare it with existing state-of-the-art algorithms as well as existing classifiers such as Naïve Bayes, decision tree, logistic regression, and MLP trained using BP. The results show that although GGA-MLP takes more time to converge as compared to other metaheuristic algorithms, the performance of GGA-MLP is better than or comparable to the existing techniques in classifying datasets, especially large datasets, as GGA-MLP searches the solution space properly by maintaining a balance between exploration and exploitation.In future, we plan to extend our work to train other types of ANNs and incorporate architecture optimization in it.



---

*Source: 4036035-2022-02-24.xml*