# A Feature Construction Method That Combines Particle Swarm Optimization and Grammatical Evolution
## 2 The Proposed Method
### The Used PSO Method  
The main steps for this algorithm are outlined in detail in Algorithm 1. ```
1. Initialization Step. (a) Set \(\text{iter}=0\). (b) Set \(m\) as the total number of particles. (c) Set \(\text{iter}_{\text{max}}\) as the maximum number of iterations allowed. (d) Initialize randomly the positions \(\overrightarrow{p_{1}},\overrightarrow{p_{2}},...,\overrightarrow{p_{m}}\) for the particles. For the grammatical evolution, every particle is a vector of randomly selected integers. (e) Initialize randomly the velocities \(\overrightarrow{u_{1}},\overrightarrow{u_{2}},...,\overrightarrow{u_{m}}\). For the current work, every vector of velocities is a series of randomly selected integers in the range \(\bigl{[}u_{\text{min}},u_{\text{max}}\bigr{]}\). In the current work, \(u_{\text{min}}=-5\) and \(u_{\text{max}}=5\). (f) For\(i=1..m\), set \(\overrightarrow{b_{i}}=\overrightarrow{p_{i}}\). The vector \(\overrightarrow{b_{i}}\) denotes the best located position of particle \(\overrightarrow{p_{i}}\). (g) Set \(\overrightarrow{p}_{\text{best}}=\arg\min_{i\in\text{L}..m}f\bigl{(} \overrightarrow{p_{i}}\bigr{)}\). 2. Termination Check Step. If \(\text{iter}\geq\text{iter}_{\text{max}}\) then go to step 8. 3. For\(i=1..m\), do the following: (a) Compute the velocity \(\overrightarrow{u_{i}}\) as a combination of the vectors \(\overrightarrow{u_{i}}\), \(\overrightarrow{p_{i}}\), \(\overrightarrow{b_{i}}\), and \(\overrightarrow{p}_{\text{best}}\). (b) Set the new position for the particle to \(\overrightarrow{p_{i}}=\overrightarrow{p_{i}}+\overrightarrow{u_{i}}\). (c) Calculate the fitness \(f(p_{i})\) for particle \(p_{i}\) using the procedure described in Section 2.3. (d) If\(f\bigl{(}\overrightarrow{p_{i}}\bigr{)}\leq f\bigl{(}\overrightarrow{b_{i}} \bigr{)}\), then \(\overrightarrow{b_{i}}=\overrightarrow{p_{i}}\). 4. End For. (e) Set \(\overrightarrow{p}_{\text{best}}=\arg\min_{i\in\text{L}..m}f\bigl{(} \overrightarrow{p_{i}}\bigr{)}\). (f) Set \(\text{iter}=\text{iter}+1\). (f) Goto step 2. (f) Test step. Apply the mapping function of the best particle \(\overrightarrow{p}_{\text{best}}\) to the test set of the problem, and apply a machine learning model obtaining the corresponding test error. ```  
**Algorithm 1** The base PSO algorithm executed in one processing unit. The above calculates at every iteration the new position of the particle \(i\) using  
\[\overrightarrow{p_{i}}=\overrightarrow{p_{i}}+\overrightarrow{u_{i}} \tag{3}\]  
In most cases, the new velocity could be a linear combination of the previously computed velocity and the corresponding vectors for the best values \(\overrightarrow{b_{i}}\) and \(\overrightarrow{p}_{\text{best}}\), and it can be defined as follows:  
\[\overrightarrow{u_{i}}=\omega\overrightarrow{u_{i}}+r_{1}c_{1}\Bigl{(} \overrightarrow{b_{i}}-\overrightarrow{p_{i}}\Bigr{)}+r_{2}c_{2}\bigl{(} \overrightarrow{p}_{\text{best}}-\overrightarrow{p_{i}}\bigr{)} \tag{4}\]where the following are true:  
1. The variables \(r_{1},\ r_{2}\) are random numbers defined in \([0,1]\). 2. The constants \(c_{1},\ c_{2}\) are defined in the range \([1,2]\). 3. The variable \(\omega\), commonly called the inertia, was suggested by [PERSON] and [PERSON] [76]. In the original paper, they proposed the idea that large values for the inertia coefficient can lead to a better exploration of the search space, while smaller values of the coefficient lead to the method being concentrated around regions likely to contain the global minimum. Hence, in their work, the value of the inertia factor generally started with large values and decreased with repetition. In the current work, the inertia value was computed through the following equation: \[\omega=0.5+\frac{r}{2}\] (5)  
The variable \(r\) is a a random number with \(r\in[0,1]\). This inertia calculation was proposed in [104].