# An Expanded Heterogeneous Particle Swarm Optimization Based on Adaptive Inertia Weight

Authors: Sami Zdiri; Jaouher Chrouta; Abderrahmen Zaafouri
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

In this study, a modified version of multiswarm particle swarm optimization algorithm (MsPSO) is proposed. However, the classical MsPSO algorithm causes premature stagnation due to the limitation of particle diversity; as a result, it is simple to slip into a local optimum. To overcome the above feebleness, this work presents a heterogeneous multiswarm PSO algorithm based on adaptive inertia weight strategies called (A-MsPSO). The MsPSO’s main advantages are that it is simple to use and that there are few settings to alter. In the MsPSO method, the inertia weight is a key parameter affecting considerably convergence, exploration, and exploitation. In this manuscript, an adaptive inertia weight is adopted to ameliorate the global search ability of the classical MsPSO algorithm. Its performance is based on exploration, which is defined as an algorithm’s capacity to search through a variety of search spaces. It also aids in determining the best ideal capability for searching a small region and determining the candidate answer. If a swarm discovers a global best location during iterations, the inertia weight is increased, and exploration in that direction is enhanced. The standard tests and indications provided in the specialized literature are used to show the efficiency of the proposed algorithm. Furthermore, findings of comparisons between A-MsPSO and six other common PSO algorithms show that our proposal has a highly promising performance for handling various types of optimization problems, leading to both greater solution accuracy and more efficient solution times.


---

## Body

## 1. Introduction

Particle swarm optimization (PSO) is a stochastic population-based algorithm developed by Eberhart and Kennedy [1]. PSO was derived from the intelligent collective behavior of animal species (e.g., flocks of birds), especially that of searching food. At the start of plowing, one bird discovers the food source. Then, it is followed very quickly by another and so on. In this process, the information regarding a potential feast is widely disseminated within the group of gulls that fly in search of food in a more or less orderly manner. The gathering takes place through a (voluntary or not) social exchange of information between individuals of the same species. One of them found a solution and the others adopt it. Kennedy and Eberhart [2] initially attempted to simulate the birds’ ability to fly synchronously and to shift a path suddenly while staying in optimum shape. Then, they expanded their model into a simple and efficient optimization algorithm and the particles are individuals moving through the research hyperspace. PSO has been effectively deployed in a number of locations in recent years. Due to its multiple advantages, such as a wide search range, speedy convergence, and ease of implementation, it has been used in a variety of research fields and applications. On another side, since the PSO algorithm is easily stuck in a local optimum, it was greatly improved. The abovementioned enhancements are organized by the particular gap in the PSO they seek to fill.The Binary PSO. To evolve the network architecture, Eberhart and Kennedy introduced a binary version of the PSO [3] to represent and solve problems of a binary nature as well as to compare genetic algorithms (GA) encoded binarily with PSO.Rate of Convergence Improvements. Different strategies have been suggested to increase the PSO’s convergence rate. They usually require improving the PSO update equations without altering the algorithm structure, which generally enhances the optimization of local performance, in several local minima functions. There are three parameters that change the PSO update equations: the inertia weight control [4–6] and the two acceleration coefficients. In fact, the inertia weight is among the earliest improvements of the original PSO made to further enhance the algorithm convergence rate. One of the most common enhancements is the implementation of Shi and Eberhart inertia weights [7]. In 1998, the authors suggested a technique to dynamically adapt the inertia weight using a fuzzy controller [8]. Then, researchers, in [9], demonstrated that the application of a constriction factor is required for the PSO algorithm to converge.In the same framework and from another analytical view, PSO algorithms have been successfully used in a multitude of industrial applications and are readily trapped in local optima. To solve such issue, several improvements have been made: enhancing particle displacement adjustment parameters and improving topological structures, heterogeneous updating rules, and population grouping using multiple swarms. In [10], Liang and Sughantan have proposed a dynamic multiswarm particle swarm optimizer (DMSPSO) wherein the entire population is split in numerous swarms by applying clustering strategies to regroup these swarms and ensure the exchange of information between them. In [11], Yen and Leong have suggested two strategies, namely, swarm growth strategy and swarm decline strategy, which enable swarms to distribute information with the best individuals in the population. In addition, in [12], a cooperative method of PSO, which divides the population into four subswarms and affects significantly the convergence of the algorithm, has been introduced. In [13], Li et al. proposed an improved adaptive holonic PSO algorithm that did not consider connections between particles and used a clustering strategy. Obviously, the introduced algorithm enhanced the search efficiency. In [14], two optimization examples were solved using an improved optimization algorithm (OIPSO): optimization of resource constraints with the shortest project duration and optimization of resource leveling at fixed duration. Through these last two, it was proven that the developed algorithm allowed improving the optimization effect of PSO and accelerating the optimization speed. Similarly, in 2014, a new metaheuristic search algorithm has been proposed by Ngaam [15]; the population was partitioned into 4 subswarms: 2 basic subswarms for exploitation search as well as adaptive subswarm and the exploration subswarm. The main advantage of MsPSO is the possibility of using easily the tuning parameters, which improves the search performance of the algorithm. Despite the reported improvements, they cannot always meet the requirements of applications in various areas. Because of the importance of inertia weight as a tuning parameter used to ensure convergence and improve accuracy, Zdiri et al. [16] have performed a comparative study of the different inertia weight tuning strategies and they have also shown that the adaptive inertia weight is the best technique that allows obtaining greater precision. In this context and to assess the MsPSO algorithm performance, a new modified version, named multiswarm particle swarm optimization algorithm based on adaptive inertia weight (A-MsPSO), is proposed in this study. The main contribution of this article is resumed as follows: the population is split into 4 subswarms, each of which represents a single PSO. In each subswarm, we use an adaptive inertia weight strategy which oversees the search situation and adapts according to the evolutionary state of each particle. Adaptation is carried out, at each iteration, by a mechanism defined by the best overall aptitude (Gbest). The latter is determined from the best overall aptitude of each subswarm. This takes place with 2 constant acceleration coefficients applied to enhance the local search capacity of each subswarm and that of global exploration and speed of convergence. The remaining of this paper is structured as follows: Section 2 presents the PSO algorithm. MsPSO algorithm is defined in Section 3. Section 4 presents the introduced optimization algorithm (A-MsPSO). In Section 5, the relation between the position and the velocity of the particles in each subswarm of A-MsPSO is well explained. Section 6 presents the simulation results and shows the efficiency of the introduced algorithm by comparing it with other approaches. Section 7 suggests the future work possibilities. Finally, Section 8 concludes this work.


## 2. Particle Swarm Optimization

### 2.1. Definition

At the outset of the algorithm, the particles of the swarm are randomly distributed in the search space where each particle has a randomly defined movement speed and position. Thereafter, it is able, at each instance, to(i)
Evaluate its current position and memorize the best one it has reached so far and the fitness function value in this position.(ii)
Communicate with the neighboring particles, obtain from each of them its own best performance, modify its speed according to best solutions, and consequently, define the direction of the next displacement. The strategy of particle displacement is explained in Figure1.Figure 1
Schematic representation of the particle movement in a swarm.


### 2.2. Formalization

Theith particle displacement between iteration t and iteration t + 1 is formulated analytically by the two following relations of velocity and position, respectively,(1)vit+1=wvit+c1r1xpbest−xit+c2r2xgbest−xit,(2)xit+1=xit+vit+1,where w is a constant called inertia weight; xpbest denotes the best historical personal position determined by the ith particle; xgbest corresponds to the best overall position found by the population; c1 designates the cognitive parameter; c2 is the social parameter; and r1, r2 are generated randomly by a uniform distribution in 0,1.


### 2.3. PSO Improvement History

To improve the PSO algorithms, several modifications have been proposed. They can be grouped into three categories.


#### 2.3.1. Association of PSO with Other Search Operators

In intelligent systems, hybridization consists in combining the desirable properties of different methods aimed at minimizing their weaknesses. Multiple hybrid PSO systems have been used in specific application contexts. In what follows, we briefly review some of these approaches. Among the techniques widely used to increase PSO performance, we can mention hybridization with evolutionary algorithms (AE) and particularly genetic algorithms (GA). Angeline [17] utilized a tournament selection mechanism whose purpose is to replace the position and speed of each poorly-performing particle with those of the best performing particle. This movement in space improves performance in 3 test functions used, yet it has moved away from the classical PSO. Moreover, Zhang and Xie used, in [18], different techniques in tandem in their PSO with differential evolution (DEPSO). The canonical PSO and DE (differential evolution) operators were utilized in alternate generations. Obviously, hybridization was efficiently applied in some test functions and the obtained results indicate that DEPSO can improve the PSO efficiency in solving larger dimensional problems. Liu and Abraham, in [19], hybridized a turbulent PSO (TPSO) with a fuzzy logic controller. The TPSO was based on the idea that particles stuck in a suboptimal solution, which led to the premature convergence of PSO. The velocity parameters were, then, adaptively regulated during optimization using a fuzzy logic extension. The efficiency of the suggested method in solving high and low dimensionality problems was tested, with positive results in both cases. In particular, when the performance of canonical PSO deteriorated considerably in the face of problems with strong dimensionality, the TPSO and FATPSO were slightly affected. Ying Tan described a PSO algorithm based on a cloning process (CPSO) inspired by the natural immune system in [20]. The method is characterized by its quick convergence and ease of implementation. The applied algorithm increased the area adjacent to the potential solution and accelerated the evolution of the swarm by cloning the best individual in the following generations, resulting in greater optimization capacity and convergence. To diversify the population, Xiao and Zuo [21] adopted a multipopulation technique, using each subpopulation, to a separate peak. Then, they utilized a hybrid DEPSO operator in order to determine the optima in each one. Moving peaks benchmark tests yielded much lower average offline error than competing solutions. Chen et al. integrated the PSO and CS (cuckoo search), in [22], to develop PSOCS, where cuckoos near excellent solutions interacted with each other, migrated slowly towards the ideal solutions and led by the global bests in PSO. In [23], a new crow swarm optimization (CSO) algorithm, that combines PSO with CSA (crow search algorithm) and allows people to explore unknown locations while being guided by a random individual, was suggested. CSO algorithm was tested on several benchmark functions. The suggested CSO’s performance was measured in terms of optimization efficiency and global search capability and all of that are vastly enhanced over either PSO or CSA.


#### 2.3.2. Improvement of the Topological Structure

There are two original versions of PSO: the classic local ring version (Lbest) and the global star version (Gbest). In addition to these two original models, several topologies were proposed. The majority of these enhanced the PSO algorithm’s performance. A variety of topologies were introduced and improved from static topologies to dynamic ones. Some of the widely used and successful topologies are presented below.(1) Static Topologies. The best known static versions are the global version and the classic local version. In the latter, the swarm converges more slowly than in Gbest, but can probably locate the global optimum. In general, using the Lbest model reduces the risks of premature convergence of the algorithm, which affects positively the performance of the algorithm, particularly for multimodal problems. We present some more recent examples as follows:(2) Four Clusters [24]. The four-cluster topology uses four groups of particles linked together by several gateways. From a sociological point of view, this topology resembles four isolated communities. In each community, a few particles can communicate with a particle belonging to another community. The particles are divided into four groups, each of which consists of five particles and exchanges information with the other three groups. Each gateway is linked to only one group. This topology is characterized by the number of edges that indirectly connect two particles. The diameter of the graph is three, which means that information can be transmitted from one particle to another by borrowing a maximum of three edges.(3) Wheel or Focal [25]. In this topology, all particles are isolated from each other and they are related only to the focal particle. Moreover, information is communicated by this particle that uses this data to adapt its trajectory.In [26], the authors developed a new improved algorithm (FGLT-PSO) named fusion global-local topology particle swarm optimization that simultaneously utilizes local and global topologies in PSO to get out of local optima. This proposal enhanced the PSO algorithm performance in terms of solution precision and convergence velocity. The most important studies dealing with this network are represented in Figure 2.Figure 2
Different types of topologies of a particle swarm: (a) ring; (b) ray; (c) star; and (d) four clusters.
(a)(b)(c)(d)(4) Dynamic Topologies. The authors tried to set up dynamic topologies having better performance than static topologies by changing their structures from one iteration to another. This change allows particles to alter their travel paths so that they can escape local optima. In the following part, we will discuss a set of dynamic topologies:Fitness-distance ratio [27]: it is a variation of PSO based on a special social network; it was inspired by the observation of animal behavior. This social network does not use a specific geometric structure to effect communication between particles. In fact, the position and speed of a particle are updated by dimension. For each dimension, the particle has only one informant, which prevents several informants from communicating information that cancels each other out. The choice of this informant, named nbest (best nearest), is based on two criteria: (i) it has to be close to the current particle and (ii) it must have visited a position with better fitness.Bastos-Filho [28]: in this structure, the swarm is divided into groups or clans of particles. Each clan, containing a fixed number of fully-connected particles, admits a leader. The authors added the possibility of the particles migration from one clan to another and, at each iteration, the particle with the best location was the leader.


#### 2.3.3. Control of the PSO Algorithm Parameters

Many recent works focusing on optimization have shown that the best performance can be obtained only by selecting the adequate adjustment parameters [29, 30]. In PSO, two acceleration coefficients and the inertia weight are the three main factors affecting the algorithm performance. They affect also the orientation of the particle on its future displacement in order to guarantee a balance between local research and global research. Their control prevents the swarm from exploding and makes convergence easier.The inertia weight allows defining the exploration capacity of each particle in order to improve its convergence. Great value equals great range of motion and, therefore, global exploration. A weak value is synonymous to low range of motion and, thus, local exploration. Therefore, fixing this factor amounts to finding a compromise between local exploration and global exploration. The size of the inertia weight influences the size of explored hyperspace and no value can guarantee convergence towards the optimal solution. Like the performance of other swarm algorithms, that of PSO can be improved by selecting the right parameters. Thus, Clerc [31] gave some general guidelines for choosing the right combination. The acceleration coefficients were first used by Kennedy and Eberhart. The social parameter they utilized is the same as the cognitive parameter with a constant value. Although several enhancements were made, the combination of the parameters w, c1, and c2 is the only solution that makes the adjustment of the balance between the phases of intensification and diversification of the research process possible. For example, in [32], a new optimization strategy for particle swarms (APSO) was formulated; the coefficients are equal to 2.0 and adjusted adaptively according to the evolutionary state (exploitation, exploration, convergence, and jumping out). It allows the automated adjustment of acceleration coefficients and inertia weight during runtime, resulting in faster convergence and better search efficiency. PSO presents a major problem called premature convergence. This problem can lead to the algorithm stagnation in local optimum. In fact, much effort has been made by scientists and researchers to enhance the performance of this algorithm. In 2014 [15], a new algorithm, called multiswarm particle swarm optimization based on the distribution of population into four subswarms cooperating through a specific strategy, has been proposed by Ngaam. In the following section, we will present MsPSO in detail.


## 2.1. Definition

At the outset of the algorithm, the particles of the swarm are randomly distributed in the search space where each particle has a randomly defined movement speed and position. Thereafter, it is able, at each instance, to(i)
Evaluate its current position and memorize the best one it has reached so far and the fitness function value in this position.(ii)
Communicate with the neighboring particles, obtain from each of them its own best performance, modify its speed according to best solutions, and consequently, define the direction of the next displacement. The strategy of particle displacement is explained in Figure1.Figure 1
Schematic representation of the particle movement in a swarm.


## 2.2. Formalization

Theith particle displacement between iteration t and iteration t + 1 is formulated analytically by the two following relations of velocity and position, respectively,(1)vit+1=wvit+c1r1xpbest−xit+c2r2xgbest−xit,(2)xit+1=xit+vit+1,where w is a constant called inertia weight; xpbest denotes the best historical personal position determined by the ith particle; xgbest corresponds to the best overall position found by the population; c1 designates the cognitive parameter; c2 is the social parameter; and r1, r2 are generated randomly by a uniform distribution in 0,1.


## 2.3. PSO Improvement History

To improve the PSO algorithms, several modifications have been proposed. They can be grouped into three categories.


### 2.3.1. Association of PSO with Other Search Operators

In intelligent systems, hybridization consists in combining the desirable properties of different methods aimed at minimizing their weaknesses. Multiple hybrid PSO systems have been used in specific application contexts. In what follows, we briefly review some of these approaches. Among the techniques widely used to increase PSO performance, we can mention hybridization with evolutionary algorithms (AE) and particularly genetic algorithms (GA). Angeline [17] utilized a tournament selection mechanism whose purpose is to replace the position and speed of each poorly-performing particle with those of the best performing particle. This movement in space improves performance in 3 test functions used, yet it has moved away from the classical PSO. Moreover, Zhang and Xie used, in [18], different techniques in tandem in their PSO with differential evolution (DEPSO). The canonical PSO and DE (differential evolution) operators were utilized in alternate generations. Obviously, hybridization was efficiently applied in some test functions and the obtained results indicate that DEPSO can improve the PSO efficiency in solving larger dimensional problems. Liu and Abraham, in [19], hybridized a turbulent PSO (TPSO) with a fuzzy logic controller. The TPSO was based on the idea that particles stuck in a suboptimal solution, which led to the premature convergence of PSO. The velocity parameters were, then, adaptively regulated during optimization using a fuzzy logic extension. The efficiency of the suggested method in solving high and low dimensionality problems was tested, with positive results in both cases. In particular, when the performance of canonical PSO deteriorated considerably in the face of problems with strong dimensionality, the TPSO and FATPSO were slightly affected. Ying Tan described a PSO algorithm based on a cloning process (CPSO) inspired by the natural immune system in [20]. The method is characterized by its quick convergence and ease of implementation. The applied algorithm increased the area adjacent to the potential solution and accelerated the evolution of the swarm by cloning the best individual in the following generations, resulting in greater optimization capacity and convergence. To diversify the population, Xiao and Zuo [21] adopted a multipopulation technique, using each subpopulation, to a separate peak. Then, they utilized a hybrid DEPSO operator in order to determine the optima in each one. Moving peaks benchmark tests yielded much lower average offline error than competing solutions. Chen et al. integrated the PSO and CS (cuckoo search), in [22], to develop PSOCS, where cuckoos near excellent solutions interacted with each other, migrated slowly towards the ideal solutions and led by the global bests in PSO. In [23], a new crow swarm optimization (CSO) algorithm, that combines PSO with CSA (crow search algorithm) and allows people to explore unknown locations while being guided by a random individual, was suggested. CSO algorithm was tested on several benchmark functions. The suggested CSO’s performance was measured in terms of optimization efficiency and global search capability and all of that are vastly enhanced over either PSO or CSA.


### 2.3.2. Improvement of the Topological Structure

There are two original versions of PSO: the classic local ring version (Lbest) and the global star version (Gbest). In addition to these two original models, several topologies were proposed. The majority of these enhanced the PSO algorithm’s performance. A variety of topologies were introduced and improved from static topologies to dynamic ones. Some of the widely used and successful topologies are presented below.(1) Static Topologies. The best known static versions are the global version and the classic local version. In the latter, the swarm converges more slowly than in Gbest, but can probably locate the global optimum. In general, using the Lbest model reduces the risks of premature convergence of the algorithm, which affects positively the performance of the algorithm, particularly for multimodal problems. We present some more recent examples as follows:(2) Four Clusters [24]. The four-cluster topology uses four groups of particles linked together by several gateways. From a sociological point of view, this topology resembles four isolated communities. In each community, a few particles can communicate with a particle belonging to another community. The particles are divided into four groups, each of which consists of five particles and exchanges information with the other three groups. Each gateway is linked to only one group. This topology is characterized by the number of edges that indirectly connect two particles. The diameter of the graph is three, which means that information can be transmitted from one particle to another by borrowing a maximum of three edges.(3) Wheel or Focal [25]. In this topology, all particles are isolated from each other and they are related only to the focal particle. Moreover, information is communicated by this particle that uses this data to adapt its trajectory.In [26], the authors developed a new improved algorithm (FGLT-PSO) named fusion global-local topology particle swarm optimization that simultaneously utilizes local and global topologies in PSO to get out of local optima. This proposal enhanced the PSO algorithm performance in terms of solution precision and convergence velocity. The most important studies dealing with this network are represented in Figure 2.Figure 2
Different types of topologies of a particle swarm: (a) ring; (b) ray; (c) star; and (d) four clusters.
(a)(b)(c)(d)(4) Dynamic Topologies. The authors tried to set up dynamic topologies having better performance than static topologies by changing their structures from one iteration to another. This change allows particles to alter their travel paths so that they can escape local optima. In the following part, we will discuss a set of dynamic topologies:Fitness-distance ratio [27]: it is a variation of PSO based on a special social network; it was inspired by the observation of animal behavior. This social network does not use a specific geometric structure to effect communication between particles. In fact, the position and speed of a particle are updated by dimension. For each dimension, the particle has only one informant, which prevents several informants from communicating information that cancels each other out. The choice of this informant, named nbest (best nearest), is based on two criteria: (i) it has to be close to the current particle and (ii) it must have visited a position with better fitness.Bastos-Filho [28]: in this structure, the swarm is divided into groups or clans of particles. Each clan, containing a fixed number of fully-connected particles, admits a leader. The authors added the possibility of the particles migration from one clan to another and, at each iteration, the particle with the best location was the leader.


### 2.3.3. Control of the PSO Algorithm Parameters

Many recent works focusing on optimization have shown that the best performance can be obtained only by selecting the adequate adjustment parameters [29, 30]. In PSO, two acceleration coefficients and the inertia weight are the three main factors affecting the algorithm performance. They affect also the orientation of the particle on its future displacement in order to guarantee a balance between local research and global research. Their control prevents the swarm from exploding and makes convergence easier.The inertia weight allows defining the exploration capacity of each particle in order to improve its convergence. Great value equals great range of motion and, therefore, global exploration. A weak value is synonymous to low range of motion and, thus, local exploration. Therefore, fixing this factor amounts to finding a compromise between local exploration and global exploration. The size of the inertia weight influences the size of explored hyperspace and no value can guarantee convergence towards the optimal solution. Like the performance of other swarm algorithms, that of PSO can be improved by selecting the right parameters. Thus, Clerc [31] gave some general guidelines for choosing the right combination. The acceleration coefficients were first used by Kennedy and Eberhart. The social parameter they utilized is the same as the cognitive parameter with a constant value. Although several enhancements were made, the combination of the parameters w, c1, and c2 is the only solution that makes the adjustment of the balance between the phases of intensification and diversification of the research process possible. For example, in [32], a new optimization strategy for particle swarms (APSO) was formulated; the coefficients are equal to 2.0 and adjusted adaptively according to the evolutionary state (exploitation, exploration, convergence, and jumping out). It allows the automated adjustment of acceleration coefficients and inertia weight during runtime, resulting in faster convergence and better search efficiency. PSO presents a major problem called premature convergence. This problem can lead to the algorithm stagnation in local optimum. In fact, much effort has been made by scientists and researchers to enhance the performance of this algorithm. In 2014 [15], a new algorithm, called multiswarm particle swarm optimization based on the distribution of population into four subswarms cooperating through a specific strategy, has been proposed by Ngaam. In the following section, we will present MsPSO in detail.


## 2.3.1. Association of PSO with Other Search Operators

In intelligent systems, hybridization consists in combining the desirable properties of different methods aimed at minimizing their weaknesses. Multiple hybrid PSO systems have been used in specific application contexts. In what follows, we briefly review some of these approaches. Among the techniques widely used to increase PSO performance, we can mention hybridization with evolutionary algorithms (AE) and particularly genetic algorithms (GA). Angeline [17] utilized a tournament selection mechanism whose purpose is to replace the position and speed of each poorly-performing particle with those of the best performing particle. This movement in space improves performance in 3 test functions used, yet it has moved away from the classical PSO. Moreover, Zhang and Xie used, in [18], different techniques in tandem in their PSO with differential evolution (DEPSO). The canonical PSO and DE (differential evolution) operators were utilized in alternate generations. Obviously, hybridization was efficiently applied in some test functions and the obtained results indicate that DEPSO can improve the PSO efficiency in solving larger dimensional problems. Liu and Abraham, in [19], hybridized a turbulent PSO (TPSO) with a fuzzy logic controller. The TPSO was based on the idea that particles stuck in a suboptimal solution, which led to the premature convergence of PSO. The velocity parameters were, then, adaptively regulated during optimization using a fuzzy logic extension. The efficiency of the suggested method in solving high and low dimensionality problems was tested, with positive results in both cases. In particular, when the performance of canonical PSO deteriorated considerably in the face of problems with strong dimensionality, the TPSO and FATPSO were slightly affected. Ying Tan described a PSO algorithm based on a cloning process (CPSO) inspired by the natural immune system in [20]. The method is characterized by its quick convergence and ease of implementation. The applied algorithm increased the area adjacent to the potential solution and accelerated the evolution of the swarm by cloning the best individual in the following generations, resulting in greater optimization capacity and convergence. To diversify the population, Xiao and Zuo [21] adopted a multipopulation technique, using each subpopulation, to a separate peak. Then, they utilized a hybrid DEPSO operator in order to determine the optima in each one. Moving peaks benchmark tests yielded much lower average offline error than competing solutions. Chen et al. integrated the PSO and CS (cuckoo search), in [22], to develop PSOCS, where cuckoos near excellent solutions interacted with each other, migrated slowly towards the ideal solutions and led by the global bests in PSO. In [23], a new crow swarm optimization (CSO) algorithm, that combines PSO with CSA (crow search algorithm) and allows people to explore unknown locations while being guided by a random individual, was suggested. CSO algorithm was tested on several benchmark functions. The suggested CSO’s performance was measured in terms of optimization efficiency and global search capability and all of that are vastly enhanced over either PSO or CSA.


## 2.3.2. Improvement of the Topological Structure

There are two original versions of PSO: the classic local ring version (Lbest) and the global star version (Gbest). In addition to these two original models, several topologies were proposed. The majority of these enhanced the PSO algorithm’s performance. A variety of topologies were introduced and improved from static topologies to dynamic ones. Some of the widely used and successful topologies are presented below.(1) Static Topologies. The best known static versions are the global version and the classic local version. In the latter, the swarm converges more slowly than in Gbest, but can probably locate the global optimum. In general, using the Lbest model reduces the risks of premature convergence of the algorithm, which affects positively the performance of the algorithm, particularly for multimodal problems. We present some more recent examples as follows:(2) Four Clusters [24]. The four-cluster topology uses four groups of particles linked together by several gateways. From a sociological point of view, this topology resembles four isolated communities. In each community, a few particles can communicate with a particle belonging to another community. The particles are divided into four groups, each of which consists of five particles and exchanges information with the other three groups. Each gateway is linked to only one group. This topology is characterized by the number of edges that indirectly connect two particles. The diameter of the graph is three, which means that information can be transmitted from one particle to another by borrowing a maximum of three edges.(3) Wheel or Focal [25]. In this topology, all particles are isolated from each other and they are related only to the focal particle. Moreover, information is communicated by this particle that uses this data to adapt its trajectory.In [26], the authors developed a new improved algorithm (FGLT-PSO) named fusion global-local topology particle swarm optimization that simultaneously utilizes local and global topologies in PSO to get out of local optima. This proposal enhanced the PSO algorithm performance in terms of solution precision and convergence velocity. The most important studies dealing with this network are represented in Figure 2.Figure 2
Different types of topologies of a particle swarm: (a) ring; (b) ray; (c) star; and (d) four clusters.
(a)(b)(c)(d)(4) Dynamic Topologies. The authors tried to set up dynamic topologies having better performance than static topologies by changing their structures from one iteration to another. This change allows particles to alter their travel paths so that they can escape local optima. In the following part, we will discuss a set of dynamic topologies:Fitness-distance ratio [27]: it is a variation of PSO based on a special social network; it was inspired by the observation of animal behavior. This social network does not use a specific geometric structure to effect communication between particles. In fact, the position and speed of a particle are updated by dimension. For each dimension, the particle has only one informant, which prevents several informants from communicating information that cancels each other out. The choice of this informant, named nbest (best nearest), is based on two criteria: (i) it has to be close to the current particle and (ii) it must have visited a position with better fitness.Bastos-Filho [28]: in this structure, the swarm is divided into groups or clans of particles. Each clan, containing a fixed number of fully-connected particles, admits a leader. The authors added the possibility of the particles migration from one clan to another and, at each iteration, the particle with the best location was the leader.


## 2.3.3. Control of the PSO Algorithm Parameters

Many recent works focusing on optimization have shown that the best performance can be obtained only by selecting the adequate adjustment parameters [29, 30]. In PSO, two acceleration coefficients and the inertia weight are the three main factors affecting the algorithm performance. They affect also the orientation of the particle on its future displacement in order to guarantee a balance between local research and global research. Their control prevents the swarm from exploding and makes convergence easier.The inertia weight allows defining the exploration capacity of each particle in order to improve its convergence. Great value equals great range of motion and, therefore, global exploration. A weak value is synonymous to low range of motion and, thus, local exploration. Therefore, fixing this factor amounts to finding a compromise between local exploration and global exploration. The size of the inertia weight influences the size of explored hyperspace and no value can guarantee convergence towards the optimal solution. Like the performance of other swarm algorithms, that of PSO can be improved by selecting the right parameters. Thus, Clerc [31] gave some general guidelines for choosing the right combination. The acceleration coefficients were first used by Kennedy and Eberhart. The social parameter they utilized is the same as the cognitive parameter with a constant value. Although several enhancements were made, the combination of the parameters w, c1, and c2 is the only solution that makes the adjustment of the balance between the phases of intensification and diversification of the research process possible. For example, in [32], a new optimization strategy for particle swarms (APSO) was formulated; the coefficients are equal to 2.0 and adjusted adaptively according to the evolutionary state (exploitation, exploration, convergence, and jumping out). It allows the automated adjustment of acceleration coefficients and inertia weight during runtime, resulting in faster convergence and better search efficiency. PSO presents a major problem called premature convergence. This problem can lead to the algorithm stagnation in local optimum. In fact, much effort has been made by scientists and researchers to enhance the performance of this algorithm. In 2014 [15], a new algorithm, called multiswarm particle swarm optimization based on the distribution of population into four subswarms cooperating through a specific strategy, has been proposed by Ngaam. In the following section, we will present MsPSO in detail.


## 3. Multiswarm Particle Swarm Optimization Algorithm (MsPSO)

### 3.1. General Description

This new optimization algorithm is an improved version of PSO. In this approach, the population is divided into four subswarms (an adaptive subswarm, an exploring subswarm, and two basic subswarms) to obtain the best management of the exploration and the exploitation of the research process in order to reach the best solution. The subswarm cooperation and communication model is shown in Figure3.Figure 3
Model of communication between the subswarms in MsPSO.S1 and S2 are the basic subswarms, whereas S3 and S4 are the adaptive and exploration subswarms, respectively. All the subswarms share information with each other and participate in the overall exploitation. Based on cooperation and information sharing, the particles in S3 use the speed messages and fitness values of the particles in S1 and S2 to refresh their speeds and positions. However, the speed of the particles in S4 is updated from the concentration of particles in S1, S2, and S3. To improve the PSO convergence rate, several techniques, modifying the inertia weights and/or the acceleration coefficients in the algorithm update equations, have been recently developed.


### 3.2. The Inertia Weight in the Literature

The inertia weight, which was first developed by Shi and Eberhart to ensure a balance between exploitation and exploration, controls the effect of the particle’s orientation on future displacement. The authors showed that, ifw∈0.8,1.2 convergence is faster with a reasonable number of iterations and a large value of w facilitates a global exploration, a small value will make it easier to explore locally. Several methods were suggested to solve PSO-based optimization problems by using an inertia weight in the particle velocity equation with some modifications. The forces changing the inertia weight can be classified into three groups.


#### 3.2.1. Constant Inertia Weight

In many introduced methods, PSO-based optimization problems were solved by utilizing a constant inertia weight in a modified particle velocity equation. In [33], for example, Shi and Eberhard used a random value of w to follow the optima.(3)w=0.5+rand2,where rand is generated randomly by a uniform distribution in 0,1.


#### 3.2.2. Time-Varying Inertia Weight

In this class, the inertia weight value is determined as a function of time using a specific number of iterations. These strategies can be nonlinear or linear and decreasing or increasing. In what follows, we present some of these methods that have an impact factor in the literature. For example, in [34], Xin and Chen introduced a decreasing linear inertia weight efficiently used to enhance the fine tuning characteristics of the PSO. Besides, in [35], the authors added a chaotic term to the weight of inertia of linear growth. Moreover, Kordestani and Meybodi [36] developed an oscillating triangular inertia weight (OTIW) to follow the optima in a dynamic environment.


#### 3.2.3. Adaptive Inertia Weight

Inertia weight techniques are employed in the third category to keep track of the search scenario and change the inertia weight value relying on the feedback parameters [37]. In [38], Rao and Arumugam employed the ratio between the best overall shape and the best local mean shape of the particles in order to calculate the inertia weight in each iteration and avoid the premature convergence towards the local minimum. The strategy of the adaptive inertia weight has been made available to improve its research capacity and control the diversity of the population by an adaptive adjustment of the inertia weight.


### 3.3. Limitations and Discussion of MsPSO Algorithm

The MsPSO algorithm presents three main adjustment parameters:w, c1, and c2 that affect significantly the convergence of the algorithm. The four subswarms use two constant acceleration coefficients and time-varying inertia weight. The MsPSO algorithm contains a time-varying inertia weight function given by(4)wi=0.9×numiter−0.5inumiter.Besides,c1 and c2, are adjusted to a constant value.(5)c1=c2=1.494.This category of functions can cause incorrect update of the particle velocity as no data about the evolving state showing the population diversity is identified or used. Therefore, the choice of the parameters can disturb the direction of the future displacement of each particle and degrade the algorithm performance in terms of computation time and convergence. To overcome the mentioned problems, we suggest, in this manuscript, a modified version of the MsPSO algorithm called adaptive multiswarm particle swarm optimization (A-MsPSO) where a mechanism is applied to adjust both the inertia weight and acceleration coefficient. The advantage of this mechanism is that it considers the evolutionary state of the particles of the four abovementioned subswarms. This enhancement made to introduce a process of exchange and cooperation between the subswarms to guarantee the best exploration and the most efficient exploitation of the search space.


## 3.1. General Description

This new optimization algorithm is an improved version of PSO. In this approach, the population is divided into four subswarms (an adaptive subswarm, an exploring subswarm, and two basic subswarms) to obtain the best management of the exploration and the exploitation of the research process in order to reach the best solution. The subswarm cooperation and communication model is shown in Figure3.Figure 3
Model of communication between the subswarms in MsPSO.S1 and S2 are the basic subswarms, whereas S3 and S4 are the adaptive and exploration subswarms, respectively. All the subswarms share information with each other and participate in the overall exploitation. Based on cooperation and information sharing, the particles in S3 use the speed messages and fitness values of the particles in S1 and S2 to refresh their speeds and positions. However, the speed of the particles in S4 is updated from the concentration of particles in S1, S2, and S3. To improve the PSO convergence rate, several techniques, modifying the inertia weights and/or the acceleration coefficients in the algorithm update equations, have been recently developed.


## 3.2. The Inertia Weight in the Literature

The inertia weight, which was first developed by Shi and Eberhart to ensure a balance between exploitation and exploration, controls the effect of the particle’s orientation on future displacement. The authors showed that, ifw∈0.8,1.2 convergence is faster with a reasonable number of iterations and a large value of w facilitates a global exploration, a small value will make it easier to explore locally. Several methods were suggested to solve PSO-based optimization problems by using an inertia weight in the particle velocity equation with some modifications. The forces changing the inertia weight can be classified into three groups.


### 3.2.1. Constant Inertia Weight

In many introduced methods, PSO-based optimization problems were solved by utilizing a constant inertia weight in a modified particle velocity equation. In [33], for example, Shi and Eberhard used a random value of w to follow the optima.(3)w=0.5+rand2,where rand is generated randomly by a uniform distribution in 0,1.


### 3.2.2. Time-Varying Inertia Weight

In this class, the inertia weight value is determined as a function of time using a specific number of iterations. These strategies can be nonlinear or linear and decreasing or increasing. In what follows, we present some of these methods that have an impact factor in the literature. For example, in [34], Xin and Chen introduced a decreasing linear inertia weight efficiently used to enhance the fine tuning characteristics of the PSO. Besides, in [35], the authors added a chaotic term to the weight of inertia of linear growth. Moreover, Kordestani and Meybodi [36] developed an oscillating triangular inertia weight (OTIW) to follow the optima in a dynamic environment.


### 3.2.3. Adaptive Inertia Weight

Inertia weight techniques are employed in the third category to keep track of the search scenario and change the inertia weight value relying on the feedback parameters [37]. In [38], Rao and Arumugam employed the ratio between the best overall shape and the best local mean shape of the particles in order to calculate the inertia weight in each iteration and avoid the premature convergence towards the local minimum. The strategy of the adaptive inertia weight has been made available to improve its research capacity and control the diversity of the population by an adaptive adjustment of the inertia weight.


## 3.2.1. Constant Inertia Weight

In many introduced methods, PSO-based optimization problems were solved by utilizing a constant inertia weight in a modified particle velocity equation. In [33], for example, Shi and Eberhard used a random value of w to follow the optima.(3)w=0.5+rand2,where rand is generated randomly by a uniform distribution in 0,1.


## 3.2.2. Time-Varying Inertia Weight

In this class, the inertia weight value is determined as a function of time using a specific number of iterations. These strategies can be nonlinear or linear and decreasing or increasing. In what follows, we present some of these methods that have an impact factor in the literature. For example, in [34], Xin and Chen introduced a decreasing linear inertia weight efficiently used to enhance the fine tuning characteristics of the PSO. Besides, in [35], the authors added a chaotic term to the weight of inertia of linear growth. Moreover, Kordestani and Meybodi [36] developed an oscillating triangular inertia weight (OTIW) to follow the optima in a dynamic environment.


## 3.2.3. Adaptive Inertia Weight

Inertia weight techniques are employed in the third category to keep track of the search scenario and change the inertia weight value relying on the feedback parameters [37]. In [38], Rao and Arumugam employed the ratio between the best overall shape and the best local mean shape of the particles in order to calculate the inertia weight in each iteration and avoid the premature convergence towards the local minimum. The strategy of the adaptive inertia weight has been made available to improve its research capacity and control the diversity of the population by an adaptive adjustment of the inertia weight.


## 3.3. Limitations and Discussion of MsPSO Algorithm

The MsPSO algorithm presents three main adjustment parameters:w, c1, and c2 that affect significantly the convergence of the algorithm. The four subswarms use two constant acceleration coefficients and time-varying inertia weight. The MsPSO algorithm contains a time-varying inertia weight function given by(4)wi=0.9×numiter−0.5inumiter.Besides,c1 and c2, are adjusted to a constant value.(5)c1=c2=1.494.This category of functions can cause incorrect update of the particle velocity as no data about the evolving state showing the population diversity is identified or used. Therefore, the choice of the parameters can disturb the direction of the future displacement of each particle and degrade the algorithm performance in terms of computation time and convergence. To overcome the mentioned problems, we suggest, in this manuscript, a modified version of the MsPSO algorithm called adaptive multiswarm particle swarm optimization (A-MsPSO) where a mechanism is applied to adjust both the inertia weight and acceleration coefficient. The advantage of this mechanism is that it considers the evolutionary state of the particles of the four abovementioned subswarms. This enhancement made to introduce a process of exchange and cooperation between the subswarms to guarantee the best exploration and the most efficient exploitation of the search space.


## 4. Adaptive Multiswarm Particle Swarm Optimization Algorithm (A-MsPSO)

### 4.1. General Description of A-MsPSO Algorithm

A-MsPOS algorithm is used in this paper to enhance the exploration and exploitation of the basic MsPSO that incorporates four subswarms, an adaptive inertia weight method, and two constant acceleration coefficients. The populations are split into four subswarms containing N particles in a search space ofD dimension where each subswarm runs a single PSO and searches a local optimum or, at the same time, a global optimum. Thus, from the best local individual of the four subswarms, the best global optimum can be obtained as follows: gbest=fgbestsub/sub=1,2,3,4. The particle settings guarantee the update of speed and position using their subswarms equation (the details are presented in subsection 4.2). The four subswarms maintain specific binding to enhance the search capability. The cooperation and communication between the subswarms are presented in Figure 4. The pseudocode of the introduced A-MsPSO is provided in Algorithm 1 and the organization chart is exposed in Figure 5. The A-MsPSO algorithm contains 4 subswarms and uses various cooperative strategies to broaden the exploration and exploitation of the MsPSO.Figure 4
Communication model in A-MsPSO.Figure 5
Organization chart of A-MsPSO.Algorithm 1:The proposed algorithm (A-MsPSO).
BeginSet the particle size of each subswarmInitialize the positions, velocities, inertia weight, and acceleration factorsDetermine each particle’s shape worthFind the Pbest in subswarmsS1, S2, and S3 and the gbest in the populationRepeat until you hit the maximum number{Calculate the velocitiesv1, v2, v3, and v4 in subswarms S1, S2, S3, and S4Update the positionsEvaluate the fitness of theith particleUpdategbestIf the guide condition is satisfiedIn each subswarm, use diversity guided convergence strategy.End if}End doReturngbest of the algorithmFigure6 describes the mechanism of exploring the new region where x1, x2, and x3 positions of the particles of three subswarms S1, S2, and S3 are randomly placed in the search space; they approach the new captured Gbest position. At time T, the velocities of each particle are defined randomly and their position, at time T + 1, is determined according to the equations update. Position x4 has the ability to explore a new area from which another space of unknown search is provided using the modification equation (12). The search solutions are obtained from a circle containing the overall optimum. Graphically, the old region and the new one are separated by an arc of a circle. Furthermore, the ith particle in S3 is modified according to the fitness values and the velocity of the particles in the basic subswarms. In S4, the ith particle changes its velocity depending on velocities of the particles in S1, S2, and S3.Figure 6
The cooperative evolutionary process in A-MsPSO.


### 4.2. A-MsPSO Search Behavior

The four subswarms collaborate and search for the optimum Gbest solution by using the best solutions of each subswarm. As shown in Figure4, the global optimum is defined as a function of the global optimum of the 4 subswarms. The expressions of the speed and position of the particles of S1 and S2 are given by the following equations:(6)vi1/2t+1=wavi1/2t+c1r1pi1/2−xi1/2t+c2r2gbest−xi1/2t,where r1 and r2 are randomly generated in 0,1, the superscript of (1/2) represents the basic subswarms; and wa is the adaptive inertia weight obtained by(7)wai=wmax+wmax−wmin×exi−1exi+1,where(8)xi=gbest−igbest+i,where gbest represents the best position shared by the entire population; wmin and wmax denote initial value and final one, respectively.The particles in the adaptive subswarmS3 adjust their flight directions to the base of best particles in S1 and S2. The speed in S3 is revised in accordance with the following equation:(9)vi3t+1=waγγ1vi1t+1+γγ2vi2t+1+vi3t+c1r3pi3−xi3t+c2r4gbest−xi3t,where r3 and r4 are randomly generated in 0,1.The fitness valuesγ1 and γ2 of the particles in S1 and S2 are declared in the following equation:(10)γ=γ1+γ2.The velocity update equation inS4 is written as follows:(11)vi4t+1=vi1t+1+vi2t+1−vi3t+1.The position ofS4 is updated by applying the following equation:(12)xi4t+1=α1xi4t+α2pi4+α3gbest+vi4t+1,with(13)α1+α2+α3=1,where(14)α1=16,α2=13,α3=12.The impact factorsα1, α2, and α3 are employed to control the effects of the information on the position of the particle in S4.In A-MsPSO, information is shared by all particles in various areas. The basic subswarms (S1 and S2) influence the velocity and position information for other particles in other swarms. In S3 with the current velocities and the fitness values, the particles follow the new search repertoire. Figure 4 depicts the cooperation between the subswarms.InS4, the update of the position and the speed are different from those in the other three subswarms and the particles adjust their speed according to the speeds and positions of S1, S2, and S3. The rules of cooperation are defined to ensure the communication between the subswarms in the search process.


## 4.1. General Description of A-MsPSO Algorithm

A-MsPOS algorithm is used in this paper to enhance the exploration and exploitation of the basic MsPSO that incorporates four subswarms, an adaptive inertia weight method, and two constant acceleration coefficients. The populations are split into four subswarms containing N particles in a search space ofD dimension where each subswarm runs a single PSO and searches a local optimum or, at the same time, a global optimum. Thus, from the best local individual of the four subswarms, the best global optimum can be obtained as follows: gbest=fgbestsub/sub=1,2,3,4. The particle settings guarantee the update of speed and position using their subswarms equation (the details are presented in subsection 4.2). The four subswarms maintain specific binding to enhance the search capability. The cooperation and communication between the subswarms are presented in Figure 4. The pseudocode of the introduced A-MsPSO is provided in Algorithm 1 and the organization chart is exposed in Figure 5. The A-MsPSO algorithm contains 4 subswarms and uses various cooperative strategies to broaden the exploration and exploitation of the MsPSO.Figure 4
Communication model in A-MsPSO.Figure 5
Organization chart of A-MsPSO.Algorithm 1:The proposed algorithm (A-MsPSO).
BeginSet the particle size of each subswarmInitialize the positions, velocities, inertia weight, and acceleration factorsDetermine each particle’s shape worthFind the Pbest in subswarmsS1, S2, and S3 and the gbest in the populationRepeat until you hit the maximum number{Calculate the velocitiesv1, v2, v3, and v4 in subswarms S1, S2, S3, and S4Update the positionsEvaluate the fitness of theith particleUpdategbestIf the guide condition is satisfiedIn each subswarm, use diversity guided convergence strategy.End if}End doReturngbest of the algorithmFigure6 describes the mechanism of exploring the new region where x1, x2, and x3 positions of the particles of three subswarms S1, S2, and S3 are randomly placed in the search space; they approach the new captured Gbest position. At time T, the velocities of each particle are defined randomly and their position, at time T + 1, is determined according to the equations update. Position x4 has the ability to explore a new area from which another space of unknown search is provided using the modification equation (12). The search solutions are obtained from a circle containing the overall optimum. Graphically, the old region and the new one are separated by an arc of a circle. Furthermore, the ith particle in S3 is modified according to the fitness values and the velocity of the particles in the basic subswarms. In S4, the ith particle changes its velocity depending on velocities of the particles in S1, S2, and S3.Figure 6
The cooperative evolutionary process in A-MsPSO.


## 4.2. A-MsPSO Search Behavior

The four subswarms collaborate and search for the optimum Gbest solution by using the best solutions of each subswarm. As shown in Figure4, the global optimum is defined as a function of the global optimum of the 4 subswarms. The expressions of the speed and position of the particles of S1 and S2 are given by the following equations:(6)vi1/2t+1=wavi1/2t+c1r1pi1/2−xi1/2t+c2r2gbest−xi1/2t,where r1 and r2 are randomly generated in 0,1, the superscript of (1/2) represents the basic subswarms; and wa is the adaptive inertia weight obtained by(7)wai=wmax+wmax−wmin×exi−1exi+1,where(8)xi=gbest−igbest+i,where gbest represents the best position shared by the entire population; wmin and wmax denote initial value and final one, respectively.The particles in the adaptive subswarmS3 adjust their flight directions to the base of best particles in S1 and S2. The speed in S3 is revised in accordance with the following equation:(9)vi3t+1=waγγ1vi1t+1+γγ2vi2t+1+vi3t+c1r3pi3−xi3t+c2r4gbest−xi3t,where r3 and r4 are randomly generated in 0,1.The fitness valuesγ1 and γ2 of the particles in S1 and S2 are declared in the following equation:(10)γ=γ1+γ2.The velocity update equation inS4 is written as follows:(11)vi4t+1=vi1t+1+vi2t+1−vi3t+1.The position ofS4 is updated by applying the following equation:(12)xi4t+1=α1xi4t+α2pi4+α3gbest+vi4t+1,with(13)α1+α2+α3=1,where(14)α1=16,α2=13,α3=12.The impact factorsα1, α2, and α3 are employed to control the effects of the information on the position of the particle in S4.In A-MsPSO, information is shared by all particles in various areas. The basic subswarms (S1 and S2) influence the velocity and position information for other particles in other swarms. In S3 with the current velocities and the fitness values, the particles follow the new search repertoire. Figure 4 depicts the cooperation between the subswarms.InS4, the update of the position and the speed are different from those in the other three subswarms and the particles adjust their speed according to the speeds and positions of S1, S2, and S3. The rules of cooperation are defined to ensure the communication between the subswarms in the search process.


## 5. Particle Paths: Relation between the Particles Position and Velocity

We examine theoretically, in this section, the trajectories of the particles, the convergence of the A-MsPSO algorithm, and the speed of the particles of each subswarm. Convergence is defined based on the following limit:(15)limt⟶∞xit=Pi,where xi is the position at time t of the ith particle and Pi corresponds to local or global optimum. The velocity and position update equations for the A-MsPSO with the adaptive inertia weight is applied as follows:(16)vt+1=wavt+c1r1pbest−xt+c2r2gbest−xt,xit+1=xit+vit+1.Authors, in [39], investigated the convergence of MsPSO and demonstrated that parameters setting affects the particle performance. The performed analysis shows that the relation between the particles in the 4 subswarms can be presented as a system to validate the convergence of the A-MsPSO algorithm.Applying (2)–(11), we get the following A-MsPSO system:(17)xt+1=Fxt+RE,where F is the system matrix and R is the constant matrix.xt=xt1,xt2,xt3,xt4,xt−11,xt−12,xt−13T and E=p1,p2,p3,p4,gT is the vector consisting of individual optimums and the single global optimum.The used symbols are written as follows:(18)φ1=v11+v12=c1r11+c2r12,φ2=v21+v22=c1r21+c2r22,φ3=v31+v32=c1r31+c2r32,where rij is the random number in 0,1, j=1,2, and i=1,2,3.The equation of a particle position inS1 is formulated as follows:(19)xt+11=xt1+vt+11=xt1+wavt+v11pt1−xt1+v12gt−xt1=xt1−v11xt1−v12xt1+waxt1−xt−11+v11pt1+v12gt=xt1−v11xt1−v12xt1+waxt1−waxt−11+v11pt1+v12gt=xt11+wa−φ1−waxt−11+v11pt1+v12gt.The equation of a particle position inS2 is written as follows:(20)xt+12=xt2+vt+12=xt2+wavt+v21pt2−xt2+v22gt−xt2=xt2−v21xt2−v22xt2+waxt2−xt−12+v21pt2+v22gt=xt1−v21xt2−v22xt2+waxt2−waxt−12+v21pt1+v22gt=xt21+wa−φ2−waxt−12+v21pt2+v22gt.In subswarmS3, the equation applied to obtain the particle position is written as follows:(21)xt+13=xt3+vt+13=xt3+γγ1vt+11+γγ2vt+12+vt3+v31pt3−xt3+v32gt−xt3=xt3+waγγ1xt1wa−φ1−waxt−11+v11pt1+v12gt+γγ2xt2wa−φ2−waxt−12+v21pt2+v22gt+vt3+v31pt3−xt3+v32gt−xt3=xt3+waxt3−xt−13+waγγ1xt1wa−φ1−waxt−11+v11pt1+v12gt+γγ2xt2wa−φ2−waxt−12+v21pt2+v22gt+v31pt3−φ3xt3+v32gt=xt31+wa−φ3−waxt−13+v31pt3+waγwa−φ1γ1xt1−wa2γγ1xt−11+waγv11γ1pt1+waγwa−φ2γ2xt2−wa2γγ2xt−12+waγv21γ2pt2+γwav22γ2+γwav12γ1+v32gt.The equation of a particle position inS4 is formulated as follows:(22)xt+14=α1xt4+α2pt4+α3gt+vt+14=α1xt4+α2pt4+α3gt+vt+11+vt+12−vt+13=α1xt4+α2pt4+α3gt+1−waγγ1v1+t1+1−waγγ2v1+t2−wa−φ3xt3+waxt−13−v31pt3−v32gt=α1xt4+α2pt4−wa−φ3xt3+waxt−13−v31pt3+1−waγγ1wa−φ1xt1−waxt−11+v11pt1+1−waγγ2wa−φ2xt2−waxt−12+v21pt2+1−waγγ1v11+1−waγγ2v22−v32+α3gt.Then, A-MsPSO system is obtained as shown inB1.According to the convergence analysis performed in [40], the particles of each subswarm converge to stable points defined by the limits given in the following equations:(23)limt⟶+∞x1=c12p1+c22gbest,limt⟶+∞x2=c12p2+c22gbest,limt⟶+∞x3=wac1p1+p2+c12p3+gbest2wac2+c22,limt⟶+∞x4=1−wac12p1+p2−c12p3+α2p4+gbest1−2wac2−c22+α3.


## 6. Experimental Study

This section includes three parts. In the first part, we present the test functions used in the experimental study. However, the second part shows the good performance of A-MsPSO algorithm through computational experiments, while a nonlinear system is used, in the last part, to validate the suggested method performance.


### 6.1. Test Function and Parameter Settings

To test the efficiency of A-MSPSO from the benchmark functions available in the literature, we chose 4 unimodal functions (F1, F2, F3, and F4) and 11 multimodal functions (from F5 to F15) [41]. The formulas and the properties employed in these functions are shown in Table 1. The utilized experimental machine has a third-generation i3 processor of 2.5 GHz and 128 GB of storage capacity and the applied programming language is MATLAB.Table 1
The main test functions.
#Test functionRangefminF1Sphere model−100,100D–450F2Schwefel’s problem 2.22−100,100D–450F3Schwefel’s problem 1.2−100,100D–450F4Schwefel’s problem 2.21−100,100D–450F5Generalized Rosenbrock’s function−100,100D–310F6Step function−100,100D390F7Quartic function0,600D–180F8Generalized Schwefel’s problem 2.26−32,32D–140F9Generalized Rastrigin’s function−5,5D–330F10Ackley’s function−5,5D–330F11Generalized Griewank function−0.5,0.5D90F12Generalized penalized functions−π,πD–460F13Schwefel’s function−5,5D–130F14Shekel’s foxholes function−100,100D–300F15Kowalik’s function−100,100D100


### 6.2. Results and Discussions

In this subsection, we compare the introduced method with the most widely used inertia weight strategies, on the one hand, and with other versions of PSO, on the other hand. In subsection6.2, several inertia weight strategies, such as random-MsPSO, constant-MsPSO, linear time-varying-MsPSO, and sigmoid-MsPSO, are analyzed on 15 test functions. In subsection 6.2, the A-MsPSO algorithm is compared to a number of PSO versions and to the basic MsPSO algorithm. Then, in subsection 6.2, the computational cost of the A-MsPSO algorithm is compared to that of the existing PSO variations. Finally, in subsection 6.2.1, the performance of this algorithm on Box and Jenkins gas furnace data is validated.


#### 6.2.1. Comparisons of Inertia Weight Diversity

The most intensively used inertia weight strategies and the A-MsPSO algorithm are generally applied to solve benchmark problems. Table2 lists the parameters used in each approach. All of the functions were tested on 30 dimensions. The maximum number of iterations was set to 2000, while the number of executions for each test was equal to 30. The performance of the algorithm was measured by the standard deviation and the mean values, as illustrated in Table 3. The values in bold indicate the best solution with a significance level of 5%. In this context, the A-MsPSO uses an adaptive parameter adjustment strategy based on the evolving state of each particle improving performance in terms of overall optimization and convergence velocity. This finding shows that the A-MsPSO has a better search capacity compared to the other search algorithms (random-MsPSO, constant-MsPSO, linear-time-varying-MsPSO, and sigmoid MsPSO) applied on 13 functions from F1 to F15, except F8 and F15.Table 2
Description of the various inertia weight adjustment mechanisms.
LabelReferenceInertia weight strategyAdaptation mechanismw1[18]w=0.5+rand/2Random inertia weightw2[7]w=cConstant inertia weightw3[41]wi=numiter−i/numiterwmin−wmax+wmaxLinear time-varyingw4[42]wi=wmin−wmax/1+e−u×k−n×gen+wmaxu=10loggen−2Sigmoid inertia weightw5[43]wi=wmax+wmax−wmin×exi−1/exi+1Adaptive inertia weightTable 3
The results obtained for 15 test functions.
Mean±STDMean±STDMean±STDFunctionsF1F2F3Constant1.421e−81±3.966e−814.665e−43±8.523e−439.964e−84±2.315e−83Random2.477e−175±0.00e+006.038e−86±1.877e−851.141e−168±0.00e+00LTV1.683e−234±0.00e+009.813e−119±2.032e−1186.820e−234±0.00e+00Sigmoid2.592e−82±8.077e−821.789e−42±4.530e−421.890e−83±4.247e−83A-MsPSO1.687e − 298 ± 0.0e + 004.697e − 179 ± 0.00e + 002.127e − 297 ± 0.00e + 00FunctionsF4F5F6Constant3.119e−43±7.072e−432.892e+01±4.115e−020±0Random6.813e−87±1.749e−862.890e+01±2.967e−020±0LTV4.507e−117±1.265e−1162.890e+01±4.076e−020±0Sigmoid7.593e−44±7.062e−442.890e+01±3.698e−020±0A-MsPSO1.010e − 177 ± 0.00e + 002.890e + 01 ± 1.932e − 020 ± 0FunctionsF7F8F9Constant3.073e−04±1.779e−04−11.521e−04±8.382e+040±0Random1.265e−04±9.7369e−05−10.030e−04±5.857e+040±0LTV1.240e−04±9.4534e−05−9.351e−04±4.951e+040±0Sigmoid2.683e−04±1.9508−04−11.474e−04±6.063e+040±0A-MsPSO1.569e−04±1.4101−04−7.952e−04±3.244e+040±0FunctionsF10F11F12Constant8.8818e−16±0.00e+000±05.713e−01±1.417e−01Random8.8818e−16±0.00e+000±0.3.990e−01±1.319e−01LTV8.8818e−16±0.00e+000±03.051e−01±4.993e−02Sigmoid8.8818e−16±0.00e+000±05.318e−01±2.313e−01A-MsPSO8.8818e − 16 ± 00 ± 05.573e − 01 ± 1.152e − 01FunctionsF13F14F15Constant2.816e+00±2.195e−019.989e−01±2.220e−1610.003e−04±3.200e−05Random2.219e+00±1.307e−019.989e−01±1.282e−168.105e−04±3.374e−04LTV2.761e+00±3.044e−019.989e−01±0.00e+008.103e−04±3.373e−04Sigmoid2.903e+00±1.866e−019.989e−01±2.866e−168.304e−04±3.497e−04A-MsPSO2.093e + 00 ± 2.251e − 031.494e + 00 ± 1.282ee − 1610.006e − 04 ± 5.151e − 05In Figure7, the vertical axis and the horizontal axis represent the best fitness and the number of iterations, respectively. Each curve demonstrates a shift in the shape of each algorithm based on a predetermined number of iterations. The final point of each curve represents the global optimum.Figure 7
The algorithm’s convergence curves for some problems.The proposed algorithm has significantly faster convergence when applied on the functionsF1, F2, F3, F4, F5, F6, F9, and F11 than when applied on the functions F7, F8, F10, F12, F13, F14, and F15. However, the ability to jump out of the local optimum is improved. The curves in the functions F7, F10, F12, F14, and F15 remain stable after 200 iterations; this shows that the swarm is trapped to a local minimum, and the A-MsPSO method loses its global search capability.


#### 6.2.2. Comparison of the Introduced Algorithm with Other PSO Algorithms

The proposed A-MsPSO was compared to six other variants of PSO including LW-PSO, CLWPSO, SIW-APSO, MSCPSO, NPSO, and standard MsPSO. The parameters of the algorithms are listed in Table4 according to their references. The versions of PSO were applied to the 6 test functions (F1, F2, F3, F4, F5, and F6) with 30 dimensions. The maximum number of iterations was set to 1000 and 30 independent tests of each algorithm were performed. The results of standard deviation (STD) and mean values (Mean) are listed in Table 5 and the best are in bold. From this table, we notice that, on the 4 reference functions (F1, F2, F3, and F4), the results obtained by A-MsPSO are better than those provided by the other PSO variants. They show that the search method with an adaptive parameter adjustment strategy based on the evolutionary state of each and every particle is the most efficient in improving the PSO algorithm performance in solving optimization functions compared to the other.Table 4
List of reference algorithms used in our comparison.
AlgorithmNameReferenceLW-PSO“PSO with linear inertia weight”[7]MSCPSO“Multiswarm self-adaptive and cooperative particle swarm optimization”[12]MsPSO“Multiswarm PSO”[15]SIW-APSO“Self-inertia weight adaptive particle swarm optimization”[44]NPSO“Novel particle swarm optimization algorithm for global optimization”[45]CLWPSO“Comprehensive learning particle swarm optimizer”[46]Table 5
Comparison of mean and STD obtained by eight algorithms.
Mean±STDMean±STDMean±STDFunctionsF1F2F3LW-PSO7.67e−06±2.97e−067.30e−02±6.02e−026.83e−01±5.43e−01MSCPSO5.29e−23±3.69e−231.74e−012±3.76e−127.51e+00±1.26e+01MsPSO3.85e−104±8.07e−1042.02e−54±6.01e−542.03e−102±6.43e−102SIW-APSO1.18e−146±2.65e−145−3.87e+04±2.56e+012.56e−05±6.98e−04NPSO5.66e−141±9.8e−1412.41e−129±4.170e−1291.890e−83±4.247e−83CLWPSO4.46e−14±1.73e−142.10e+01±2.98e+000±0A-MsPSO2.59e − 166 ± 0.00e + 004.39e − 87 + 1.36e − 868.086e − 168 ± 0.00e + 00FunctionsF4F5F6LW-PSO1.93e−03±4.42e−041.69e−03±6.21e−041.19e−06±5.57e−07MSCPSO1.19e−10±4.79e−100±00±0MsPSO2.21e−54±5.76e−542.89e+01±3.63e−020±0SIW-APSO1.16e−17±4.67e−132.96e−01±4.57e−012.32e−01±6.41e−01NPSO0±00±04.49e104±7.78e−104CLWPSO3.14e−10±4.64e−143.45e−07±1.94e−074.85e−10±3.63e−10A-MsPSO5.96e − 85 ± 1.88e − 842.89e+01±1.80e−020.00e+00±0.00e+00


#### 6.2.3. Computational Cost

The CPU time was used to compare the computational efficiency of the PSO variants; the computational efficiency of each algorithm on the test functionf was computed using the following formula:(24)H=TefTtotf×100%,where Te is the time of computing the benchmark function algorithm f, and Ttot denotes the cumulative time of all the function algorithms f. Figure 8 shows the percentage of the total computational time by four PSO variants (A-MsPSO, S-MsPSO, R-MsPSO, and L-MsPSO) on fifteen benchmark functions. R-MsPSO is the most time consuming algorithm out of twelve reference functions and L-MsPSO is the fastest. The A-MsPSO algorithm ranks second out of twelve functions, first in F1 and third in F12 and F14. The comparison of S-MsPSO and A-MsPSO demonstrates that the latter is the longest, on 13 reference functions, and the standard MsPSO is the fastest. This observation means that the computational complexity of introduced algorithm increases by using an adaptive inertia weight.Figure 8
Comparison of the computational time with the fifteen benchmark functions from (F1) to (F15).


### 6.3. Box–Jenkins Gas Furnace Data

Box and Jenkins gas oven dataset is a benchmark widely used as a standard test in fuzzy modeling and identification techniques. It consists of 296 pairs of observations in/outs [u (k); y (k)] k from 1 to 296. Methane and air were combined to form a mixture of CO2-containing gases. The input u (k) of the furnace corresponds to the flow of methane and the y (k) output is the concentration of CO2 percentage in the outgoing gases.Step 1: the experiment was conducted on 296 pairs of data. The population size in the four subswarms was set to 6, the number of iterations was 50, the rule number was 3, the acceleration coefficients were fixed at 1.494 (5), and the inertia weight was chosen as declared previously according to equations (7) and (8). The model obtained for the candidate characteristic variables is visualized in Figure 9. The A-MsPSO algorithm had a performance index of 0.106, which is the best compared to those of the other algorithms indicated in Table 6.Step 2: the primary 148 data pairs were employed as training data to verify the robustness of A-MsPSO, while the remaining 148 data pairs were utilized as test data to predict the performance. The comparison of the performance of A-MsPSO and that of the real system is presented in Figures10 and 11, which display the proposed system approximation and generalization potential (A-MsPSO). The MSE values for the training and testing processes are 0.058 and 0.146, respectively, as shown in Table 7. Experimental results reveal that A-MsPSO has good precision and high powerful generalization ability in system modeling of Box–Jenkins gas furnace dataset.Figure 9
Modeling using 296 pairs of data from the Box–Jenkins gas furnace: (a) actual value and output from A-MsPSO and (b) error between A-MsPSO output and actual value.
(a)(b)Figure 10
Comparison of A-MsPSO and the real system over 148 testing data. (a) The A-MsPSO output and the real values. (b) Error between the A-MsPSO output and the real values.
(a)(b)Table 6
Step 1: different results on the Box–Jenkins gas furnace dataset.
ModelReferenceNumber of inputsNumber of rulesMSECheung et al. (2014)[13]230.110Box and Jenkins (1990)[47]6—0.202Sugeno and Yasukawa (1993)[48]360.190Evsukoff et al. (2002)[49]2900.090Sugeno and Yasukawa (1993)[50]660.075Tsekouras et al. (2005)[51]240.148Bagis et al. (2008)[52]220.161Zdiri et al. (2020)[16]230.106The values in bold indicate the parameters and the results of our algorithm.Table 7
Step 2: different results on the Box–Jenkins gas furnace dataset.
ModelReferenceNumber of rulesMSETraining dataTesting dataCheung et al. (2014)[13]30.0590.152Lin and Cunningham (1995)[53]50.0710.261Kim et al. (1997)[54]60.0340.244Li et al. (2012)[55]30.01590.126Tsekouras (2005)[56]60.0220.236Li et al. (2013)[57]30.0150.147Zdiri et al. (2020)[16]30.0580.146The values in bold indicate the parameter and the results of training data and testing data of our algorithm.


## 6.1. Test Function and Parameter Settings

To test the efficiency of A-MSPSO from the benchmark functions available in the literature, we chose 4 unimodal functions (F1, F2, F3, and F4) and 11 multimodal functions (from F5 to F15) [41]. The formulas and the properties employed in these functions are shown in Table 1. The utilized experimental machine has a third-generation i3 processor of 2.5 GHz and 128 GB of storage capacity and the applied programming language is MATLAB.Table 1
The main test functions.
#Test functionRangefminF1Sphere model−100,100D–450F2Schwefel’s problem 2.22−100,100D–450F3Schwefel’s problem 1.2−100,100D–450F4Schwefel’s problem 2.21−100,100D–450F5Generalized Rosenbrock’s function−100,100D–310F6Step function−100,100D390F7Quartic function0,600D–180F8Generalized Schwefel’s problem 2.26−32,32D–140F9Generalized Rastrigin’s function−5,5D–330F10Ackley’s function−5,5D–330F11Generalized Griewank function−0.5,0.5D90F12Generalized penalized functions−π,πD–460F13Schwefel’s function−5,5D–130F14Shekel’s foxholes function−100,100D–300F15Kowalik’s function−100,100D100


## 6.2. Results and Discussions

In this subsection, we compare the introduced method with the most widely used inertia weight strategies, on the one hand, and with other versions of PSO, on the other hand. In subsection6.2, several inertia weight strategies, such as random-MsPSO, constant-MsPSO, linear time-varying-MsPSO, and sigmoid-MsPSO, are analyzed on 15 test functions. In subsection 6.2, the A-MsPSO algorithm is compared to a number of PSO versions and to the basic MsPSO algorithm. Then, in subsection 6.2, the computational cost of the A-MsPSO algorithm is compared to that of the existing PSO variations. Finally, in subsection 6.2.1, the performance of this algorithm on Box and Jenkins gas furnace data is validated.


### 6.2.1. Comparisons of Inertia Weight Diversity

The most intensively used inertia weight strategies and the A-MsPSO algorithm are generally applied to solve benchmark problems. Table2 lists the parameters used in each approach. All of the functions were tested on 30 dimensions. The maximum number of iterations was set to 2000, while the number of executions for each test was equal to 30. The performance of the algorithm was measured by the standard deviation and the mean values, as illustrated in Table 3. The values in bold indicate the best solution with a significance level of 5%. In this context, the A-MsPSO uses an adaptive parameter adjustment strategy based on the evolving state of each particle improving performance in terms of overall optimization and convergence velocity. This finding shows that the A-MsPSO has a better search capacity compared to the other search algorithms (random-MsPSO, constant-MsPSO, linear-time-varying-MsPSO, and sigmoid MsPSO) applied on 13 functions from F1 to F15, except F8 and F15.Table 2
Description of the various inertia weight adjustment mechanisms.
LabelReferenceInertia weight strategyAdaptation mechanismw1[18]w=0.5+rand/2Random inertia weightw2[7]w=cConstant inertia weightw3[41]wi=numiter−i/numiterwmin−wmax+wmaxLinear time-varyingw4[42]wi=wmin−wmax/1+e−u×k−n×gen+wmaxu=10loggen−2Sigmoid inertia weightw5[43]wi=wmax+wmax−wmin×exi−1/exi+1Adaptive inertia weightTable 3
The results obtained for 15 test functions.
Mean±STDMean±STDMean±STDFunctionsF1F2F3Constant1.421e−81±3.966e−814.665e−43±8.523e−439.964e−84±2.315e−83Random2.477e−175±0.00e+006.038e−86±1.877e−851.141e−168±0.00e+00LTV1.683e−234±0.00e+009.813e−119±2.032e−1186.820e−234±0.00e+00Sigmoid2.592e−82±8.077e−821.789e−42±4.530e−421.890e−83±4.247e−83A-MsPSO1.687e − 298 ± 0.0e + 004.697e − 179 ± 0.00e + 002.127e − 297 ± 0.00e + 00FunctionsF4F5F6Constant3.119e−43±7.072e−432.892e+01±4.115e−020±0Random6.813e−87±1.749e−862.890e+01±2.967e−020±0LTV4.507e−117±1.265e−1162.890e+01±4.076e−020±0Sigmoid7.593e−44±7.062e−442.890e+01±3.698e−020±0A-MsPSO1.010e − 177 ± 0.00e + 002.890e + 01 ± 1.932e − 020 ± 0FunctionsF7F8F9Constant3.073e−04±1.779e−04−11.521e−04±8.382e+040±0Random1.265e−04±9.7369e−05−10.030e−04±5.857e+040±0LTV1.240e−04±9.4534e−05−9.351e−04±4.951e+040±0Sigmoid2.683e−04±1.9508−04−11.474e−04±6.063e+040±0A-MsPSO1.569e−04±1.4101−04−7.952e−04±3.244e+040±0FunctionsF10F11F12Constant8.8818e−16±0.00e+000±05.713e−01±1.417e−01Random8.8818e−16±0.00e+000±0.3.990e−01±1.319e−01LTV8.8818e−16±0.00e+000±03.051e−01±4.993e−02Sigmoid8.8818e−16±0.00e+000±05.318e−01±2.313e−01A-MsPSO8.8818e − 16 ± 00 ± 05.573e − 01 ± 1.152e − 01FunctionsF13F14F15Constant2.816e+00±2.195e−019.989e−01±2.220e−1610.003e−04±3.200e−05Random2.219e+00±1.307e−019.989e−01±1.282e−168.105e−04±3.374e−04LTV2.761e+00±3.044e−019.989e−01±0.00e+008.103e−04±3.373e−04Sigmoid2.903e+00±1.866e−019.989e−01±2.866e−168.304e−04±3.497e−04A-MsPSO2.093e + 00 ± 2.251e − 031.494e + 00 ± 1.282ee − 1610.006e − 04 ± 5.151e − 05In Figure7, the vertical axis and the horizontal axis represent the best fitness and the number of iterations, respectively. Each curve demonstrates a shift in the shape of each algorithm based on a predetermined number of iterations. The final point of each curve represents the global optimum.Figure 7
The algorithm’s convergence curves for some problems.The proposed algorithm has significantly faster convergence when applied on the functionsF1, F2, F3, F4, F5, F6, F9, and F11 than when applied on the functions F7, F8, F10, F12, F13, F14, and F15. However, the ability to jump out of the local optimum is improved. The curves in the functions F7, F10, F12, F14, and F15 remain stable after 200 iterations; this shows that the swarm is trapped to a local minimum, and the A-MsPSO method loses its global search capability.


### 6.2.2. Comparison of the Introduced Algorithm with Other PSO Algorithms

The proposed A-MsPSO was compared to six other variants of PSO including LW-PSO, CLWPSO, SIW-APSO, MSCPSO, NPSO, and standard MsPSO. The parameters of the algorithms are listed in Table4 according to their references. The versions of PSO were applied to the 6 test functions (F1, F2, F3, F4, F5, and F6) with 30 dimensions. The maximum number of iterations was set to 1000 and 30 independent tests of each algorithm were performed. The results of standard deviation (STD) and mean values (Mean) are listed in Table 5 and the best are in bold. From this table, we notice that, on the 4 reference functions (F1, F2, F3, and F4), the results obtained by A-MsPSO are better than those provided by the other PSO variants. They show that the search method with an adaptive parameter adjustment strategy based on the evolutionary state of each and every particle is the most efficient in improving the PSO algorithm performance in solving optimization functions compared to the other.Table 4
List of reference algorithms used in our comparison.
AlgorithmNameReferenceLW-PSO“PSO with linear inertia weight”[7]MSCPSO“Multiswarm self-adaptive and cooperative particle swarm optimization”[12]MsPSO“Multiswarm PSO”[15]SIW-APSO“Self-inertia weight adaptive particle swarm optimization”[44]NPSO“Novel particle swarm optimization algorithm for global optimization”[45]CLWPSO“Comprehensive learning particle swarm optimizer”[46]Table 5
Comparison of mean and STD obtained by eight algorithms.
Mean±STDMean±STDMean±STDFunctionsF1F2F3LW-PSO7.67e−06±2.97e−067.30e−02±6.02e−026.83e−01±5.43e−01MSCPSO5.29e−23±3.69e−231.74e−012±3.76e−127.51e+00±1.26e+01MsPSO3.85e−104±8.07e−1042.02e−54±6.01e−542.03e−102±6.43e−102SIW-APSO1.18e−146±2.65e−145−3.87e+04±2.56e+012.56e−05±6.98e−04NPSO5.66e−141±9.8e−1412.41e−129±4.170e−1291.890e−83±4.247e−83CLWPSO4.46e−14±1.73e−142.10e+01±2.98e+000±0A-MsPSO2.59e − 166 ± 0.00e + 004.39e − 87 + 1.36e − 868.086e − 168 ± 0.00e + 00FunctionsF4F5F6LW-PSO1.93e−03±4.42e−041.69e−03±6.21e−041.19e−06±5.57e−07MSCPSO1.19e−10±4.79e−100±00±0MsPSO2.21e−54±5.76e−542.89e+01±3.63e−020±0SIW-APSO1.16e−17±4.67e−132.96e−01±4.57e−012.32e−01±6.41e−01NPSO0±00±04.49e104±7.78e−104CLWPSO3.14e−10±4.64e−143.45e−07±1.94e−074.85e−10±3.63e−10A-MsPSO5.96e − 85 ± 1.88e − 842.89e+01±1.80e−020.00e+00±0.00e+00


### 6.2.3. Computational Cost

The CPU time was used to compare the computational efficiency of the PSO variants; the computational efficiency of each algorithm on the test functionf was computed using the following formula:(24)H=TefTtotf×100%,where Te is the time of computing the benchmark function algorithm f, and Ttot denotes the cumulative time of all the function algorithms f. Figure 8 shows the percentage of the total computational time by four PSO variants (A-MsPSO, S-MsPSO, R-MsPSO, and L-MsPSO) on fifteen benchmark functions. R-MsPSO is the most time consuming algorithm out of twelve reference functions and L-MsPSO is the fastest. The A-MsPSO algorithm ranks second out of twelve functions, first in F1 and third in F12 and F14. The comparison of S-MsPSO and A-MsPSO demonstrates that the latter is the longest, on 13 reference functions, and the standard MsPSO is the fastest. This observation means that the computational complexity of introduced algorithm increases by using an adaptive inertia weight.Figure 8
Comparison of the computational time with the fifteen benchmark functions from (F1) to (F15).


## 6.2.1. Comparisons of Inertia Weight Diversity

The most intensively used inertia weight strategies and the A-MsPSO algorithm are generally applied to solve benchmark problems. Table2 lists the parameters used in each approach. All of the functions were tested on 30 dimensions. The maximum number of iterations was set to 2000, while the number of executions for each test was equal to 30. The performance of the algorithm was measured by the standard deviation and the mean values, as illustrated in Table 3. The values in bold indicate the best solution with a significance level of 5%. In this context, the A-MsPSO uses an adaptive parameter adjustment strategy based on the evolving state of each particle improving performance in terms of overall optimization and convergence velocity. This finding shows that the A-MsPSO has a better search capacity compared to the other search algorithms (random-MsPSO, constant-MsPSO, linear-time-varying-MsPSO, and sigmoid MsPSO) applied on 13 functions from F1 to F15, except F8 and F15.Table 2
Description of the various inertia weight adjustment mechanisms.
LabelReferenceInertia weight strategyAdaptation mechanismw1[18]w=0.5+rand/2Random inertia weightw2[7]w=cConstant inertia weightw3[41]wi=numiter−i/numiterwmin−wmax+wmaxLinear time-varyingw4[42]wi=wmin−wmax/1+e−u×k−n×gen+wmaxu=10loggen−2Sigmoid inertia weightw5[43]wi=wmax+wmax−wmin×exi−1/exi+1Adaptive inertia weightTable 3
The results obtained for 15 test functions.
Mean±STDMean±STDMean±STDFunctionsF1F2F3Constant1.421e−81±3.966e−814.665e−43±8.523e−439.964e−84±2.315e−83Random2.477e−175±0.00e+006.038e−86±1.877e−851.141e−168±0.00e+00LTV1.683e−234±0.00e+009.813e−119±2.032e−1186.820e−234±0.00e+00Sigmoid2.592e−82±8.077e−821.789e−42±4.530e−421.890e−83±4.247e−83A-MsPSO1.687e − 298 ± 0.0e + 004.697e − 179 ± 0.00e + 002.127e − 297 ± 0.00e + 00FunctionsF4F5F6Constant3.119e−43±7.072e−432.892e+01±4.115e−020±0Random6.813e−87±1.749e−862.890e+01±2.967e−020±0LTV4.507e−117±1.265e−1162.890e+01±4.076e−020±0Sigmoid7.593e−44±7.062e−442.890e+01±3.698e−020±0A-MsPSO1.010e − 177 ± 0.00e + 002.890e + 01 ± 1.932e − 020 ± 0FunctionsF7F8F9Constant3.073e−04±1.779e−04−11.521e−04±8.382e+040±0Random1.265e−04±9.7369e−05−10.030e−04±5.857e+040±0LTV1.240e−04±9.4534e−05−9.351e−04±4.951e+040±0Sigmoid2.683e−04±1.9508−04−11.474e−04±6.063e+040±0A-MsPSO1.569e−04±1.4101−04−7.952e−04±3.244e+040±0FunctionsF10F11F12Constant8.8818e−16±0.00e+000±05.713e−01±1.417e−01Random8.8818e−16±0.00e+000±0.3.990e−01±1.319e−01LTV8.8818e−16±0.00e+000±03.051e−01±4.993e−02Sigmoid8.8818e−16±0.00e+000±05.318e−01±2.313e−01A-MsPSO8.8818e − 16 ± 00 ± 05.573e − 01 ± 1.152e − 01FunctionsF13F14F15Constant2.816e+00±2.195e−019.989e−01±2.220e−1610.003e−04±3.200e−05Random2.219e+00±1.307e−019.989e−01±1.282e−168.105e−04±3.374e−04LTV2.761e+00±3.044e−019.989e−01±0.00e+008.103e−04±3.373e−04Sigmoid2.903e+00±1.866e−019.989e−01±2.866e−168.304e−04±3.497e−04A-MsPSO2.093e + 00 ± 2.251e − 031.494e + 00 ± 1.282ee − 1610.006e − 04 ± 5.151e − 05In Figure7, the vertical axis and the horizontal axis represent the best fitness and the number of iterations, respectively. Each curve demonstrates a shift in the shape of each algorithm based on a predetermined number of iterations. The final point of each curve represents the global optimum.Figure 7
The algorithm’s convergence curves for some problems.The proposed algorithm has significantly faster convergence when applied on the functionsF1, F2, F3, F4, F5, F6, F9, and F11 than when applied on the functions F7, F8, F10, F12, F13, F14, and F15. However, the ability to jump out of the local optimum is improved. The curves in the functions F7, F10, F12, F14, and F15 remain stable after 200 iterations; this shows that the swarm is trapped to a local minimum, and the A-MsPSO method loses its global search capability.


## 6.2.2. Comparison of the Introduced Algorithm with Other PSO Algorithms

The proposed A-MsPSO was compared to six other variants of PSO including LW-PSO, CLWPSO, SIW-APSO, MSCPSO, NPSO, and standard MsPSO. The parameters of the algorithms are listed in Table4 according to their references. The versions of PSO were applied to the 6 test functions (F1, F2, F3, F4, F5, and F6) with 30 dimensions. The maximum number of iterations was set to 1000 and 30 independent tests of each algorithm were performed. The results of standard deviation (STD) and mean values (Mean) are listed in Table 5 and the best are in bold. From this table, we notice that, on the 4 reference functions (F1, F2, F3, and F4), the results obtained by A-MsPSO are better than those provided by the other PSO variants. They show that the search method with an adaptive parameter adjustment strategy based on the evolutionary state of each and every particle is the most efficient in improving the PSO algorithm performance in solving optimization functions compared to the other.Table 4
List of reference algorithms used in our comparison.
AlgorithmNameReferenceLW-PSO“PSO with linear inertia weight”[7]MSCPSO“Multiswarm self-adaptive and cooperative particle swarm optimization”[12]MsPSO“Multiswarm PSO”[15]SIW-APSO“Self-inertia weight adaptive particle swarm optimization”[44]NPSO“Novel particle swarm optimization algorithm for global optimization”[45]CLWPSO“Comprehensive learning particle swarm optimizer”[46]Table 5
Comparison of mean and STD obtained by eight algorithms.
Mean±STDMean±STDMean±STDFunctionsF1F2F3LW-PSO7.67e−06±2.97e−067.30e−02±6.02e−026.83e−01±5.43e−01MSCPSO5.29e−23±3.69e−231.74e−012±3.76e−127.51e+00±1.26e+01MsPSO3.85e−104±8.07e−1042.02e−54±6.01e−542.03e−102±6.43e−102SIW-APSO1.18e−146±2.65e−145−3.87e+04±2.56e+012.56e−05±6.98e−04NPSO5.66e−141±9.8e−1412.41e−129±4.170e−1291.890e−83±4.247e−83CLWPSO4.46e−14±1.73e−142.10e+01±2.98e+000±0A-MsPSO2.59e − 166 ± 0.00e + 004.39e − 87 + 1.36e − 868.086e − 168 ± 0.00e + 00FunctionsF4F5F6LW-PSO1.93e−03±4.42e−041.69e−03±6.21e−041.19e−06±5.57e−07MSCPSO1.19e−10±4.79e−100±00±0MsPSO2.21e−54±5.76e−542.89e+01±3.63e−020±0SIW-APSO1.16e−17±4.67e−132.96e−01±4.57e−012.32e−01±6.41e−01NPSO0±00±04.49e104±7.78e−104CLWPSO3.14e−10±4.64e−143.45e−07±1.94e−074.85e−10±3.63e−10A-MsPSO5.96e − 85 ± 1.88e − 842.89e+01±1.80e−020.00e+00±0.00e+00


## 6.2.3. Computational Cost

The CPU time was used to compare the computational efficiency of the PSO variants; the computational efficiency of each algorithm on the test functionf was computed using the following formula:(24)H=TefTtotf×100%,where Te is the time of computing the benchmark function algorithm f, and Ttot denotes the cumulative time of all the function algorithms f. Figure 8 shows the percentage of the total computational time by four PSO variants (A-MsPSO, S-MsPSO, R-MsPSO, and L-MsPSO) on fifteen benchmark functions. R-MsPSO is the most time consuming algorithm out of twelve reference functions and L-MsPSO is the fastest. The A-MsPSO algorithm ranks second out of twelve functions, first in F1 and third in F12 and F14. The comparison of S-MsPSO and A-MsPSO demonstrates that the latter is the longest, on 13 reference functions, and the standard MsPSO is the fastest. This observation means that the computational complexity of introduced algorithm increases by using an adaptive inertia weight.Figure 8
Comparison of the computational time with the fifteen benchmark functions from (F1) to (F15).


## 6.3. Box–Jenkins Gas Furnace Data

Box and Jenkins gas oven dataset is a benchmark widely used as a standard test in fuzzy modeling and identification techniques. It consists of 296 pairs of observations in/outs [u (k); y (k)] k from 1 to 296. Methane and air were combined to form a mixture of CO2-containing gases. The input u (k) of the furnace corresponds to the flow of methane and the y (k) output is the concentration of CO2 percentage in the outgoing gases.Step 1: the experiment was conducted on 296 pairs of data. The population size in the four subswarms was set to 6, the number of iterations was 50, the rule number was 3, the acceleration coefficients were fixed at 1.494 (5), and the inertia weight was chosen as declared previously according to equations (7) and (8). The model obtained for the candidate characteristic variables is visualized in Figure 9. The A-MsPSO algorithm had a performance index of 0.106, which is the best compared to those of the other algorithms indicated in Table 6.Step 2: the primary 148 data pairs were employed as training data to verify the robustness of A-MsPSO, while the remaining 148 data pairs were utilized as test data to predict the performance. The comparison of the performance of A-MsPSO and that of the real system is presented in Figures10 and 11, which display the proposed system approximation and generalization potential (A-MsPSO). The MSE values for the training and testing processes are 0.058 and 0.146, respectively, as shown in Table 7. Experimental results reveal that A-MsPSO has good precision and high powerful generalization ability in system modeling of Box–Jenkins gas furnace dataset.Figure 9
Modeling using 296 pairs of data from the Box–Jenkins gas furnace: (a) actual value and output from A-MsPSO and (b) error between A-MsPSO output and actual value.
(a)(b)Figure 10
Comparison of A-MsPSO and the real system over 148 testing data. (a) The A-MsPSO output and the real values. (b) Error between the A-MsPSO output and the real values.
(a)(b)Table 6
Step 1: different results on the Box–Jenkins gas furnace dataset.
ModelReferenceNumber of inputsNumber of rulesMSECheung et al. (2014)[13]230.110Box and Jenkins (1990)[47]6—0.202Sugeno and Yasukawa (1993)[48]360.190Evsukoff et al. (2002)[49]2900.090Sugeno and Yasukawa (1993)[50]660.075Tsekouras et al. (2005)[51]240.148Bagis et al. (2008)[52]220.161Zdiri et al. (2020)[16]230.106The values in bold indicate the parameters and the results of our algorithm.Table 7
Step 2: different results on the Box–Jenkins gas furnace dataset.
ModelReferenceNumber of rulesMSETraining dataTesting dataCheung et al. (2014)[13]30.0590.152Lin and Cunningham (1995)[53]50.0710.261Kim et al. (1997)[54]60.0340.244Li et al. (2012)[55]30.01590.126Tsekouras (2005)[56]60.0220.236Li et al. (2013)[57]30.0150.147Zdiri et al. (2020)[16]30.0580.146The values in bold indicate the parameter and the results of training data and testing data of our algorithm.


## 7. Future Work Possibilities

In future work, the proposed approach can be applied in several applications to (i) encrypt images using a 7D hyperchaotic map [30]; (ii) adjust the hyperparameters of a 4D chaotic map [58]; and (iii) optimize the parameters of 5D hyperchaotic map using A-MsPSO to perform encryption and decryption processes [29]; (iv) it may also be used in photovoltaic water pumping applications [59]. In addition, future research will focus on a thorough examination of A-MsPSO’s applications in increasingly complicated practical optimization situations in order to deeply analyze the attributes and evaluate its performance.Figure 10
Comparison of A-MsPSO and the real system over 148 training data. (a) The A-MsPSO output and the real values. (b) Error between the A-MsPSO output and the real values.
(a)(b)


## 8. Conclusion

An adaptive inertia weight approach was utilized in this paper to present a modified version of the multiswarm particle swarm optimization algorithm. Based on the comparison of four methods for establishing inertia weight strategies in the MsPSO algorithm, the ability of the A-MsPSO algorithm to optimize the issues with a bigger search space may be demonstrated by comparing experimental results in higher attributes. It was concluded that the MsPSO algorithm with adaptive inertia weight strategy is the best for greater accuracy, according to the results obtained by the performed tests. Theoretically, the four subswarms of A-MsPSO can also converge towards their own position of stable equilibrium. Furthermore, the experimental findings show that the suggested algorithm is capable of producing a robust model with improved generalization ability. We expect that this study will be extremely useful to researchers and will inspire them to develop good solutions to solve optimization problems using the A-MsPSO algorithm.(25)F=1+wa−φ1000−wa0001+wa−φ2000−wa0waγwa−φ1γ1waγwa−φ2γ21+wa−φ30−wa2γγ1−wa2γγ2−waΓaΓbφ3−waα1ΓcΓd−wa100000001000000010000,with Γa=1−waγ/γ1wa−φ1, Γb=1−waγ/γ2wa−φ2, Γc=−wa1−waγ/γ1, and Γd=−wa1−waγ/γ2.(26)R=v11000v120v2100v22waγγ1v11waγγ2v21v310waγγ1v12+waγγ2v22+v32γ1−waγγ1v11γ2−waγγ2v21−v31α2γ1−waγγ1v12+γ2−waγγ2v22−v32+α3000000000000000(27)xt+11xt+12xt+13xt+14xt1xt2xt3=1+wa−φ1000−wa0001+wa−φ2000−wa0waγwa−φ1γ1waγwa−φ2γ21+wa−φ30−wa2γγ1−wa2γγ2−waΓaΓbφ3−waα1ΓaΓd−wa100000001000000010000xt1xt2xt3xt4xt−11xt−21xt−31+(28)v11000v120v2100v22waγγ1v11waγγ2v21v310waγγ1v12+waγγ2v22+v32γ1−waγγ1v11γ2−waγγ2v21−v31α2γ1−waγγ1v12+γ2−waγγ2v22−v32+α3000000000000000pt1pt2pt3pt4gtB1.



---

*Source: 4194263-2021-10-12.xml*