## 2 Methods
### Feature Selection
#### 2.3.2 Wrapper Method  
As illustrated in the first part of the results section and explained in the discussion section, the previously used benchmark filter method [14; 15] suffers from several crucial  
Figure 2: Dimensionality reduction and most relevant feature subset selection: (a) Filter method and (b) Wrapper method. drawbacks. To address these drawbacks, the invented feature selection algorithm presented in this study builds on the idea of the wrapper method, shown in Figure 2b. The wrapper method structure is as follows: the feature subset selection algorithm searches for the \"best possible\" subset using a selected ML algorithm itself as part of the feature subset evaluating function [23, 25]. In this structure, the selected ML algorithm is trained on the dataset, usually partitioned into internal training and validation sets, with different subsets of features removed from or added to the data. The feature subset with the highest evaluation is chosen as the final set on which to train the final ML model. In this arrangement, the ML algorithm in the wrapper structure is often considered to be a black box and is consequently evaluated during the training stage, either only on various validation sets or ultimately on an independent testing set that was not used during the training process. Although this approach gives better performance than the filter method, it is computationally expensive and, more importantly, prone to overfitting. In the established algorithm, we resolve the over-fitting of the wrapper method, which originates primarily from the bias of the training data, with two major advances. First, the black-box framework is discarded, and an optimization algorithm is embedded into the learning structure of the ML algorithm, rendering the feature selection process part of the ML model development. Second, the fitness (cost) function of the optimization algorithm is built such that the optimized model is independent of the training results. As the feature domain is discontinuous, the specified optimization problem requires a guided random search technique. As a result, a Genetic Algorithm (GA) was chosen as an optimization algorithm. By natural design, the GA explores the population of points in the given domain using probabilistic and global heuristic search [29]. This makes optimization resilient to local minima/maxima, enabling investigation of the importance of features and their combination across the entire feature domain [23]. Furthermore, GA performs well when the fitness function, which is an objective function used to direct genetic programming towards an optimal design solution, is complex and defined as a mixed (discrete and continuous) multi-objective problem, as it is in this optimization process, described by the fitness function \(FF\) in Equations (1)-(4). \[\underset{FF_{1},\,\,FF_{2},\,\,FF_{3}}{\text{argmin}}\,\,FF = FF_{1}+FF_{2}+FF_{3} \tag{1}\] \[\underset{ACC_{train}}{\text{argmin}}\,\,FF_{1} = \frac{1}{ACC_{train}}\] (2) \[\underset{m}{\text{argmin}}\,\,FF_{2} = a\frac{1}{m}\] (3) \[\underset{||\boldsymbol{w_{m}}||}{\text{argmin}}\,\,FF_{3} = b\frac{1}{||\boldsymbol{w_{m}}||}\, \tag{4}\]  
where \(m\) is the subset's number of features, \(a\) and \(b\) are weight parameters, and \(||\boldsymbol{w_{m}}||\) is the width of the separating plane in \(m^{th}\) dimension. The fitness function \(FF\) was created with the objective to minimize the subset of features while striving to achieve three goals. First, \(FF_{1}\) ensures the model's accuracy with respect to the training set. Second, the \(FF_{2}\) term ensures that the total number of features was minimized. Ultimately, the \(FF_{3}\) term has been established to direct the ML model to a state with the greatest potential for generalization without relying on the training set as feedback. The idea behind the design of the \(FF_{3}\) function is based on the geometric concept in the SVM structure and the notion of separating distance between two sets of classes [7, 22, 23, 30, 31, 32].