## 4 Selection of Computational Intelligence and Reinforcement Learning Solutions
### Particle Swarm Optimization and Quantum-Inspired Particle Swarm Optimization  
PSO is another metaheuristic consisting of a particle swarm that moves in the search space of an optimization problem and is attracted by global optima. PSO evaluates the objective function at each particle's current location in a process known as Swarm Intelligence. Each member of the swarm can be a candidate solution, and the way to aggregate the information between all members is the key to better guiding the search in the subsequent steps [52]. Through the years, PSO has drawn much attention thanks to its ease of implementation and the relatively small number of parameters that have to be tuned to obtain a very good balance between exploration and exploitation. However, finding the proper value for each parameter is not a trivial assignment, and many research efforts have been made to reach the current state-of-the-art [53; 54].  
More specifically, PSO is a relatively simple algorithm, inspired by the behavior of bird flocks [50], based on the exploration of the search space by a swarm of \(M\) interacting particles. At each iteration \(t\), each particle \(i\) has:  
* A current position vector \(X_{i}(t)=(X_{i,1}(t),\ldots,X_{i,N}(t))\), whose components represent the decision variables of the problem;
* A velocity vector \(V_{i}(t)=(V_{i,1}(t),\ldots,V_{i,N}(t))\), which captures the movement of the particles;
* A particle attractor \(P_{i}(t)=(P_{i,1}(t),\ldots,P_{i,N}(t))\), representing the \"highest\" (best) position that the particle has encountered so far.  
The particle movement is, therefore, governed by the following equations:  
\[\begin{split} V_{i,j}(t+1)=& V_{i,j}(t)+\\ & C_{1}*r_{ij}(t)*(P_{i,j}(t)-X_{i,j}(t))+\\ & C_{2}*R_{i,j}(t)*(G_{j}(t)-X_{i,j}(t))\\  X_{i,j}(t+1)=& X_{i,j}(t)+V_{i,j}(t+1)\end{split} \tag{2}\]  
in which \(G(t)\) is the swarm attractor, representing the \"highest\" (best) position that the entire swarm has encountered so far, \(r_{i,j}(t)\) and \(R_{i,j}(t)\) are random sequences uniformly sampled in \((0,1)\), and \(C_{1}\) and \(C_{2}\) are constants.  
A well-known evolution of PSO is represented by QPSO. In QPSO, all particles have a quantum behavior instead of the classical Newtonian random walks considered by classical PSO. QPSO significantly simplifies the configuration process of PSO, reducing the number of configuration parameters required to only one, i.e., the \(\alpha\) contraction-expansion parameter [55], which represents a single \"knob\" that enables adjusting the balance between the local and global search of the algorithm. Furthermore, QPSO has been demonstrated to overcome the weaknesses of PSO in the resolution of several benchmarks [56], which instead tends to be stuck in local minima. The steps required to implement QPSO are illustrated in Algorithm 2.  
```
1:procedureQPSO(target_function,conf)
2:\(particles\gets initialize\_random\_swarm(conf)\)
3:\(iteration\gets 1\)
4:\(best\gets null\)
5:repeat
6:for all\(p\) in \(particles\)do
7:\(p.val\gets target\_function(p.pos)\)
8:if\(p.val>p.bestval\)then
9:\(p.bestpos\gets p.pos\)
10:\(p.bestval\gets p.val\)
11:endif
12:if\(p.val> swarm\_bestval\)then
13:\(swarm\_bestpos\gets p.pos\)
14:\(swarm\_bestval\gets p.val\)
15:endif
16:endfor
17:\(mean\_bestpos\leftarrow\frac{1}{M}\sum_{i=1}^{M}particles[i].bestpos\)
18:\(\vec{\phi}\gets rand()\)
19:\(\vec{u}\gets rand()\)
20:\(s\gets rand()\)
21:\(attr\leftarrow\vec{\phi}*p.bestpos+(1-\vec{\phi})*swarm\_bestpos\)
22:\(\delta\gets a*|p.pos-mean\_bestpos|*\ln(1/\vec{u})\)
23:\(p.pos\gets attr+sign(rand()-0.5)*\delta\)
24:\(iteration\gets iteration+1\)
25:until\(iteration>conf.max\_iterations\)
26:return\(swarm\_bestpos,swarm\_bestval\)
27:endprocedure
```  
**Algorithm 2** Quantum-inspired Particle Swarm Optimization.