# Local Coupled Extreme Learning Machine Based on Particle Swarm Optimization
## 2 Theoretical Background
### Particle Swarm Optimization  
In 1995, a particle swarm methodology was proposed for nonlinear function optimization by [PERSON] and [PERSON] [16], which was called the PSO algorithm. It belongs to a population-based, heuristic optimization algorithm. The PSO algorithm is simple, easy to be realized and has a fast convergence rate. It has been widely applied in the fields of scientific research and engineering application [20].  
As a swarm-based algorithm, the particles of the PSO algorithm may flow through the searching space depending on the best position information of their own and their neighbors'. The initial values of the particles in the population are set randomly [24].  
In the PSO algorithm, suppose \(D\) is the dimension of searching space and \(\widehat{N}\) is the number of particles, respectively. Then, \(x_{i}^{t}\) and \(v_{i}^{t}\) are denoted by the current position and the current velocity of the \(i\)th particle at iteration \(t\), respectively [25]. Therefore, the new velocity and the particle position in the next iterative time are described as:  
\[v_{i}^{k}(t+1)=w\cdot v_{i}^{k}(t)+c_{1}\cdot rand()(p_{i}^{k}(t)-x_{i}^{k}(t))+ c_{2}\cdot rand()(g_{i}^{k}(t)-x_{i}^{k}(t)) \tag{6}\]  
\[x_{i}^{k}(t+1)=x_{i}^{k}(t)+v_{i}^{k}(t+1)\ \ 1\leq i\leq\hat{N},1\leq k\leq D \tag{7}\]  
where \(w\) denotes the inertia weight. \(c_{1}\) and \(c_{2}\) stand for the different acceleration coefficient, respectively. \(rand()\) denotes a constant value in the interval \([0,1]\) and is set randomly. \(p_{i}^{k}\) is the best position of the \(i\)th particle in the search stage at present, \(g_{i}^{k}\) represents the global best position, which constitutes the best position found in the population at present.  
In the PSO algorithm, the initial parameter \(w\) plays the role of balancing the global search and the local search. Therefore, in order to ensure higher exploring ability in the early iteration and fast convergence speed in the last part iteration, \(w\) is not a constant and can be expressed as a nonlinear function of time [17; 26]:  
\[w(t)=w_{max}-iter\times\frac{(w_{max}-w_{min})}{max\_iter} \tag{8}\]  
where \(w_{max}\) and \(w_{min}\) are the initial and terminal values of inertia weight in the iteration process, respectively. The parameter \(max\_iter\) is the maximum iteration number of the algorithm and \(iter\) is the current iteration time of the algorithm.  
In addition, in order to enhance the global search in the early part iteration, to encourage the particles to converge to the global optimal solution and to improve the convergence speed in the final iteration period [27], the acceleration parameters \(c_{1}\) and \(c_{2}\) are described as:  
\[c_{1}=(c_{1 min}-c_{1 max})\frac{iter}{max\_iter}+c_{1 max} \tag{9}\]  
\[c_{2}=(c_{2 max}-c_{2 min})\frac{iter}{max\_iter}+c_{2 min} \tag{10}\]  
where \(c_{1 max}\) and \(c_{1 min}\), \(c_{2 max}\) and \(c_{2 min}\) are constants. Based on the Equation (6), the searching ability of the cognitive and social components can be changed by changing the values of \(c_{1}\) and \(c_{2}\), which can improve the convergence rate of the PSO algorithm.