# A Feature Construction Method That Combines Particle Swarm Optimization and Grammatical Evolution
## 3 Experiments
### Experimental Results
\begin{table}
\begin{tabular}{c c c c c c} \hline
**DATASET** & **GENETIC** & **RBF** & **ADAM** & **PROP** & **NEAT** \\ \hline Appendicitis & 18.10\% & 12.23\% & 16.50\% & 16.30\% & 17.20\% \\ \hline Australian & 32.21\% & 34.89\% & 35.65\% & 36.12\% & 31.98\% \\ \hline Balance & 8.97\% & 33.42\% & 7.87\% & 8.81\% & 23.14\% \\ \hline Bands & 35.75\% & 37.22\% & 36.25\% & 36.32\% & 34.30\% \\ \hline Dermatology & 30.58\% & 62.34\% & 26.14\% & 15.12\% & 32.43\% \\ \hline Hayes Roth & 56.18\% & 64.36\% & 59.70\% & 37.46\% & 50.15\% \\ \hline Heart & 28.34\% & 31.20\% & 38.53\% & 30.51\% & 39.27\% \\ \hline House Votes & 6.62\% & 6.13\% & 7.48\% & 6.04\% & 10.89\% \\ \hline Ionosphere & 15.14\% & 16.22\% & 16.64\% & 13.65\% & 19.67\% \\ \hline Liver disorder & 31.11\% & 30.84\% & 41.53\% & 40.26\% & 30.67\% \\ \hline Lymphography & 23.26\% & 25.31\% & 29.26\% & 24.67\% & 33.70\% \\ \hline Mammography & 19.88\% & 21.38\% & 46.25\% & 18.46\% & 22.85\% \\ \hline Parkinson’s & 18.05\% & 17.42\% & 24.06\% & 22.28\% & 18.56\% \\ \hline Puma & 32.19\% & 25.78\% & 34.85\% & 34.27\% & 34.51\% \\ \hline Pop failures & 5.94\% & 7.04\% & 5.18\% & 4.81\% & 7.05\% \\ \hline Regions2 & 29.39\% & 38.29\% & 29.85\% & 27.53\% & 33.23\% \\ \hline Suharto & 34.86\% & 32.19\% & 34.04\% & 34.90\% & 34.51\% \\ \hline Segment & 57.72\% & 59.68\% & 49.75\% & 52.14\% & 66.72\% \\ \hline BC & 8.56\% & 7.27\% & 35.35\% & 21.57\% & 12.88\% \\ \hline Wine & 19.20\% & 31.41\% & 29.40\% & 30.73\% & 25.43\% \\ \hline Z\_F\_S & 10.73\% & 13.16\% & 47.81\% & 29.28\% & 38.41\% \\ \hline ZOE\_NF\_S & 8.41\% & 9.02\% & 47.43\% & 6.43\% & 43.75\% \\ \hline ZONE\_S & 2.60\% & 4.03\% & 11.99\% & 27.27\% & 5.44\% \\ \hline ZOO & 16.67\% & 21.93\% & 14.13\% & 15.47\% & 20.27\% \\ \hline
**AVERAGE** & **22.94\%** & **26.78\%** & **30.24\%** & **24.60\%** & **28.63\%** \\ \hline \end{tabular}
\end{table} 
Table 3: Average classification error for the classification datasets using the well-known methods. The results using the proposed method and for the construction of two, three, and four artificial features are presented in Tables 5 and 6. The RBF column represents the experimental results in which, after the construction of the artificial features, an RBF network with \(H\) processing nodes was applied on the modified dataset. Also, the column marked \"GENETIC\" in Tables 5 and 6 stands for the results obtained by the application of a genetic algorithm with \(m\) chromosomes to the modified dataset when the feature creation procedure was finished. The experimental results are of great interest, as one can see from their careful study that the proposed technique was able to significantly reduce the error in the corresponding test sets. Especially in the case of regression problems, the reduction in error was, on average, greater than 50%. Moreover, the usage of a neural network trained by a genetic algorithm on the modified datasets gave clearly better results than the use of an RBF neural network, especially in the classification datasets. An additional test was performed for the regression datasets, where the number of particles in the PSO algorithm increased from 100 to 400, and the results are graphically illustrated in Figure 3.  
Judging from the results, we can observe that the selection of 200 particles in the experimental results was an optimal choice and a compromise between the speed and efficiency of the method, as adding another 200 particles to the particle swarm optimization did not significantly improve the efficiency of the proposed method. Moreover, a graphical comparison between the genetic algorithm when applied to artificial datasets and the genetic algorithm when applied to the original classification datasets is given in Figure 4. The same graphical comparison is also shown for the RBF model in Figure 5, and in these figures, the ability of the proposed method to drastically reduce the learning error through the construction of artificial features is evident.