# Adaptive Multi-Level Search for Global Optimization:
## 3 Brief Review of Particle Swarm Optimization (PSO)  
Particle swarm optimization (PSO), inspired from swarm behaviors in nature such as fish and bird schooling, was developed by [PERSON] and [PERSON] [32]. Since then, PSO has attracted a lot of attention and been developed as a main representative form of swarm intelligence. PSO has been applied to many areas, such as image and video analysis applications, engineering designs and scheduling applications, classification and data mining, etc. [33]. There are at least twenty PSO variants as well as hybrid algorithms obtained by combining PSO with other existing algorithms, which are also becoming increasingly popular [34; 35; 36]. To integrate PSO with MAM to find the global optimum, adaptive multi-level search is proposed in this paper. PSO is employed for the global-level exploration in the first step. A number of particles are first placed in the search space of the optimization problem with initial positions and velocities. However, the particles can fly over the entire design space not only determined by the individual and collective knowledge of positions from the global-level search but also based on the \"local\" information of each particle. Here, the \"local\" information means the local-level exploitation in the second step. In the neighborhood of each particle, an adaptive metamodel is constructed using MAM in Section 2, which replaces the original optimization problem by a sequence of mathematical approximations that use much simpler objective and constraint functions. Hence, the critical information about individual constraint functions is kept and this leads to the improved accuracy of metamodels. During the process of metamodel building, each particle is endowed with a horizon in the surrounding region and then is refined with the current individual position to boost the possibility of finding an optimal position. Eventually, the swarm as a whole, similar to a flock of birds collectively foraging for food  
Figure 1: Flow chart of the mulitpoint approximation method (MAM). while each bird brilliantly and directly finds the most tasty food within the limited horizon, has the ability to move toward to a global optimum. Each particle in PSO represents a point in the design space of an optimization problem with an associated velocity vector. In each iteration of PSO, the velocity vector is updated by using a linear combination of three terms shown in Equation (10). The first term called inertia or momentum reflects a memory of the previous flight direction and prevents the particle from changing directions thoroughly. The second term, called the cognitive component, describes the tendency of particles returning to the previously found best positions. The last term, called the social component, quantifies the group norm or standard that should be attained. In other words, each particle tends to move toward the position of the current global best \(gbest\) and the location of the individual best \(pbest\) while moving randomly [33]. The aim is to find the global best among all the current best solutions until the objective no longer improves or a certain number of iterations are reached. The standard iteration procedure of PSO is formulated as follows:  
\[ \begin{split}\mathbf{V}_{i}^{t+1}&=\omega\mathbf{V}_{i}^{ t}+\alpha\mathbf{\epsilon}_{1}(\mathbf{pbest}_{i}^{t}-\mathbf{x}_{i}^{t})+\beta\mathbf{ \epsilon}_{2}(\mathbf{gbest}_{i}^{t}-\mathbf{x}_{i}^{t})\\ \mathbf{x}_{i}^{t+1}&=\mathbf{x}_{i}^{t}+\mathbf{V}_{i}^{t+1} \end{split}  \tag{10}\]  
where \(\omega\) is the parameter called inertial weight, \(t\) is the current iteration number, \(\alpha\) and \(\beta\) are parameters called acceleration coefficients, and \(\epsilon_{1}\) and \(\epsilon_{2}\) are two homogeneously distributed random vectors generated within the interval \([0,1)\), respectively.