# A Feature Construction Method That Combines Particle Swarm Optimization and Grammatical Evolution
## 3 Experiments
### Experimental Results  
In order to give greater credibility to the experiments carried out, the method of tenfold cross validation was incorporated for every experimental dataset. Every experiment was repeated 30 times, using different seeds for the random generator each time. All the used code was implemented in ANSI C++ using the OPTIMUMS programming library for optimization purposes, which is freely available at [[https://github.com/itsoulos/OPTIMUMS/](https://github.com/itsoulos/OPTIMUMS/)]([https://github.com/itsoulos/OPTIMUMS/](https://github.com/itsoulos/OPTIMUMS/))(accessed on 10 July 2023). For the classification datasets, the average classification error as measured in the test set is reported, while for the regression datasets, the average regression error is reported. Here, for the term classification error, we mean the percentage of patterns in the test set that were classified into a different class than expected. Also, in every table, an additional column, \"AVERAGE\", was added to show the average classification or regression error for the corresponding datasets. The values for the experimental parameters are shown in Table 2.  
In all techniques, the same parameter sets and the same random numbers were used in order to have a fair comparison of the experimental results. In the first phase, the proposed technique will generate new artificial features from the existing ones with the help of a technique guided by the partnership of particle swarm optimization and grammatical evolution. In the second phase of the technique, these features will be used to modify the original test set, to which any machine learning method can now be applied. In the second phase of the current work, two different techniques will be used: an RBF neural network and an artificial neural network, which will be trained using a genetic algorithm. This is performed in order to establish the potential of the proposed procedure and improve the performance of both simple machine learning models and complex models. The proposed technique that created artificial features was compared on the same datasets against a series of well-known methods from the relevant literature: \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Meaning** & **Value** \\ \hline \(m\) & Particles or chromosomes & 200 \\ \hline \(H\) & Number of hidden nodes & 10 \\ \hline iter\({}_{\text{max}}\) & Maximum number of iterations & 200 \\ \hline \(L_{\text{max}}\) & Limit used in penalty calculation & 100 \\ \hline \(\lambda\) & Penalty factor & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The values for every parameter used in the experiments. A genetic algorithm with \(m\) chromosomes, denotes as GENETIC in the experimental tables, is used to train an artificial neural network with \(H\) hidden nodes. After termination of the genetic algorithm, the local optimization method BEFOGS is applied to the best chromosome of the population. The radial basis function (RBF) network [140] with \(H\) processing nodes. The optimization method Adam [141], which is used to train an artificial neural network with \(H\) hidden nodes. The Prop optimization method [142; 143], which is used to train an artificial neural network with \(H\) hidden nodes. The NEAT method (Revolution of Augmenting Typologies) [144]. The experimental results using the above methods on the classification datasets are shown in Table 3, and the results for the regression datasets are illustrated in Table 4.