## 4 Discussion  
The results of this investigation clearly highlight the weaknesses of the (benchmark) filter method. Its main drawback is highlighted in Figure 4, where no clear association between the relevance of the removed features and the model's performance trend can be identified. In Figure 4a,c, no discernible impact of dimensionality reduction on the model's accuracy can be seen until it is reduced to nearly 80 features, and thereafter, the model's overall performance begins to decline. Figure 4b illustrates the same pattern, except in a more dramatic manner, where the dimensionality of the model has little effect on the model's performance until it is reduced to fewer than 24 features. One might claim that the Linear SVM at about 70 features and the RBF SVM at 19 features achieve their optimal performance; however, there is no perceptible information-gain-trend before or after these peak outcomes that can be identified with the filter method's dimensionality reduction. Figure 4d shows some of the agreements between the dimensionality of the model and its performance, with the overall accuracy (red) of the model showing an upward rise, achieving its best performance with about 160 features. However, as with the other three ML models, while the features are excluded sequentially from the data, it is unclear how to establish the threshold point for rankings, which must be defined by the user in order to include only the necessary features and exclude the redundant ones. As a consequence, the only way the interaction between the filter method and the ML models can be measured is through the performance of the model at the training or the testing stage, which essentially results in a more data-biased classification model, providing no clear indication that the generalization of the model was improved through dimensionality reduction using the filter method. Unlike the filter method, the ML model using the embedded wrapper approach and GA optimization, generated far more satisfying results. As shown in Figure 5, we were able to create a feature-set optimization algorithm that guides the development of the ML model to the best possible state, with a maximum accuracy of 100%. The fit-value (fitness function), represented by the solid black line in Figure 5, demonstrates a consistent relationship with the model's training (dotted blue), testing (dotted red) accuracy, and decreased dimensionality, represented by a solid green line with its scale on the right side of the same plot. Consequently, given that the fitness function (1)-(4) of the optimization was entirely independent of the testing data and only partly reliant on the training data (2), it is clear that the maximum accuracy was achieved solely as a result of the ML model's decreased dimensionality and increased generalization. The high generalization and maximum precision are evidently attributable to the innovative use of the mathematical framework of the SVM (5), (6) on which \(FF_{3}\) (4) was designed, something that can only be incorporated using the wrapper method. Further  
Figure 5: Feature Optimization Results (left \(y\)-scale) of the Embedded Genetic Algorithm Wrapper Method and the Dimension Reduction (right \(y\)-scale). Data: \(N=56\)\((p=32,n=24),\ R=38\)\((p=23,n=15),\ E=18\)\((p=9,n=9)\). more, the strong results can be due to the combination of features investigated in the whole feature domain. In comparison to the filter method and certain wrapper methods, we were able to integrate the features' dependencies on each other into the feature-selection cycle. This was accomplished thanks to the probabilistic and heuristic structure of the GA. The feature dependencies also became evident when the chosen number of chromosomes, one of the GA hyper parameters, was selected to be small (\(<\)300). In this scenario, the fitness function's optimum fit-value would be obtained after just a few epochs (\(<\)100), because it would settle around local minima with a feature combination close to the starting one.