# A Feature Construction Method That Combines Particle Swarm Optimization and Grammatical Evolution
## 1 Introduction
Examples of techniques proposed to tackle this problem are the weight sharing methods [53; 54], methods that reduce the number of parameters of the model (pruning methods) [55; 56], weight elimination [57; 58; 59], weight decaying methods [60; 61], dropout methods [62; 63], the Sarprop method [64], and positive correlation methods [65]. Recently, a variety of papers have proposed methods to handle the overfitting problem in various cases, such as the usage of genetic algorithms for training data selection in RBF networks [66], the evolution of RBF models using genetic algorithms for rainfall prediction [67], and pruning decision trees using genetic algorithms [68; 69]. This paper recommends a two-phase method for data classification or regression problems. In the first phase, a global optimization method directs the production of artificial features from the existing ones with the help of grammatical evolution [70]. Grammatical evolution is a variation of genetic programming where the chromosomes are production rules of the target BNF grammar, and it has been used successfully in a variety of applications, such as music composition [71], economics [72], symbolic regression [73], robotics [74], and caching algorithms [75]. The global optimization method used in this work is the particle swarm optimization (PSO) method [76; 77; 78]. The PSO method was selected as the optimization method due to its simplicity and the small number of parameters that should be set. Also, the PSO method has been used in many difficult problems in all areas of the sciences, such as problems that arise in physics [79; 80], chemistry [81; 82], medicine [83; 84], and economics [85]. Furthermore, the PSO method was successfully applied recently in many practical problems such as flow shop scheduling [86], the successful development of electric vehicle charging strategies [87], emotion recognition [88], robotics [89], the optimal design of a brace-viscous damper and pendulum tuned mass damper [90], application to high-dimensional expensive industrial problems [91], and RFID readers [92]. The generated artificial features are nonlinear combinations of the original ones, and any machine learning model can be used to effectively estimate their dynamics. In the present implementation, the RBF network was used since it is a widely tested machine learning model, but also because its training is much faster compared with other models. In the second phase, the best features obtained from the first phase are also used to modify the test set of the objective problem, and a machine learning method can be used to estimate the error in the control set. The idea of creating artificial features using grammatical evolution was first introduced in the paper by [PERSON] et al. [93], and it has been successfully applied on a series of problems, such as spam identification [94], fetal heart classification [95], epileptic oscillations [96], the construction of COVID-19 predictive models [97], and performance and early drop prediction for higher education students [98]. Feature selection using neural networks has been also proposed in a series of papers, such as the work of [PERSON] and [PERSON] [99] or the work of [PERSON] et al. [100]. Moreover, [PERSON] utilized a simulated annealing approach [101] to select the most important features for classification datasets. Also, [PERSON] et al. [102] developed a genetic algorithm that produces features using an entropy-based fitness function. The rest of this article is organized as follows. In Section 2, the steps of the proposed method are fully described. In Section 3, the used experimental datasets as well as the results obtained by the incorporation of the proposed method are outlined. Finally, in Section 4, some conclusions are listed.