# A Feature Construction Method That Combines Particle Swarm Optimization and Grammatical Evolution
## 4 Conclusions  
A hybrid technique that utilizes a particle swarm optimizer and a feature creation method using grammatical evolution was introduced here. The proposed method can identify possible dependencies between the original features and can also reduce the number of required features to a limited number. Also, the method can remove from the set of features those features that may not contribute to the learning of the dataset by some machine learning model. In addition, to make learning more efficient, the values of the generated features are bounded within a value interval using penalty factors. The constructed features are evaluated in terms of their effectiveness with the help of a fast machine learning model such as the RBF network, even though other more effective models could also be used. Among the advantages of the proposed procedure is the fact that it does not require any prior knowledge of the dataset to which it will be applied, and furthermore, the procedure is exactly the same whether it is a data classification problem or a data fitting problem. The particle swarm optimization method was used for the production of the characteristics, as it has been proven by the relevant literature to be an extremely efficient technique with a limited number of parameters that must be defined by the user. The current work was applied on an extended series of widely used datasets from various fields and was compared against some machine learning models on the same datasets. From the experimental results, it was seen that the proposed technique dramatically improved the performance of traditional learning techniques when applied to artificial features. The proposed two-stage technique generated artificial features in the first stage guided by the particle swarm optimization technique, and in the second stage, either a neural network trained by a genetic algorithm or an RBF network was used in the modified test set. In both cases, the improvement from artificial feature generation in the control error was significant for each learning model. This improvement reached an average of 30% for data classification and 50% for data fitting problems. In fact, in many cases, the improvement in the test error exceeded 75%. Moreover, the method appears to be quite robust, since increasing the number of particles in the particle swarm optimization method did not appear to significantly reduce the average error in the test sets. Furthermore,  
Figure 7: The comparison between the proposed method for constructing 2, 3, and 4 artificial features (blue color) and several Welkin classification methods (black color) on 24 different datasets revealed that the proposed method demonstrated superior performance with statistical significance. The results were obtained using the Wilcox on a sandbank test. Across all 24 datasets, the proposed method consistently outperformed the Welkin classification methods regarding the classification error. increasing the number of features constructed did not seem to have a dramatic effect on the performance of the method, which means that the method was able to achieve good generalization results even with a limited number of features, which in turn led to greatly reducing the number of dimensions of the original problem. Future work on the method may include the use of parallel techniques for feature construction to drastically reduce the required execution time. I.G.T. and A.T. conceived the idea and the methodology, and I.G.T. implemented the corresponding software. I.G.T. conducted the experiments, employing objective functions as test cases, and provided the comparative experiments. A.T. performed the necessary statistical tests. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable.