# Identification of Wiener Box-Jenkins Model for Anesthesia Using Particle Swarm Optimization
## 3 System Identification
### Iterative Optimization Using PSO  
PSO optimization algorithm (PSO), proposed by [PERSON] and [PERSON] [26], is one of the meta-heuristic based optimization approaches that gained a lot of interest in the literature with many successful applications [26; 27; 28]. One major advantage of meta-heuristic algorithms is that they are derivative-free. This is in contrast to classical optimization techniques such as gradient descent or Newton methods, which require first and sometimes second derivatives of the predictor with respect to the parameter vector.  
In PSO algorithm, a set or population of candidate solutions, called particles, are considered in at each iteration in an attempt to explore the search space. Based on the information gathered in each iteration, the current solutions are updated, and a new population is created. The best global solution at each iteration is recorded. The algorithm terminates when the improvement in the best solution is below some specified tolerance, or if the maximum number of iterations is reached.  
The PSO algorithm, illustrated in Figure 2, can be summarized as follows [35]:  
1. Generate, randomly, a population of initial parameter vectors, solutions, or _particles_, \(\theta_{i}^{(0)}\), for \(i=1,2,...,N_{p}\), where \(N_{p}\) is the population size. Each entry in the vector \(\theta_{i}^{(0)}\) is initialized to a random number generated from a uniform distribution in the interval \([\theta_{Min},\theta_{Max}]\) where \(\theta_{Min}\) and \(\theta_{Max}\) denote the lower and upper bounds, respectively, of the parameters. Additionally, initialize the velocity, \(V_{i}^{(0)}\), for each particle in the population, to zero.
2. Evaluate the cost function given by (14) for each particle \(\theta\).
3. Update the best value \(\mathit{pbest}_{i}\) for each particle \(i\) by comparing the current cost of each particle with its previous best value.  
4. Update the population's best value \(gbest\).
5. Update the velocity of each particle using the following formula: \[\begin{split} V_{i}^{(t+1)}=wV_{i}^{(t)}+& c_{1}r_{1}^{(t)}(pbest_{i}-\theta_{i}^{(t)})\\ +& c_{2}r_{2}^{(t)}(gbest-\theta_{i}^{(t)})^{\prime} \end{split}\] (16) where \(V_{i}^{(t+1)}\) and \(V_{i}^{(t+1)}\) denotes the velocity of particle \(i\) at iterations \(t+1\) and \(t\), respectively. The parameters \(r_{1}\), and \(r_{2}\) are two random numbers, and \(c_{1}\) and \(c_{2}\) are two positive constants called the _personal_ and _global_ acceleration coefficients. The parameter \(w\), called the inertia weight, controls the effect of the current particle solution on its next solution. The inertia weight is damped in each iteration using the following formula: \[w=w\zeta_{damp},\] (17) where \(\zeta_{damp}\) denotes the damping ratio of the inertia coefficient.
6. Update each particle position using the following formula: \[\theta_{i}^{(t+1)}=\theta_{i}^{(t)}+V_{i}^{(t+1)}\] (18)
7. If the change in _gbest_ is less than some given small tolerance or the maximum number of iterations is reached, the algorithm terminates; otherwise, go to step 2). The settings for the parameters of the PSO algorithm to be used in this paper are listed in Table 1. More information about the selection the PSO parameters were explained in [36].  
Figure 2: Flowchart of the PSO optimization algorithm.