# Analyzing Physics-Inspired Metaheuristic Algorithms in Feature Selection with K-Nearest-Neighbor
## 3 Results and Discussion
### Comparison with Other Methods from the Literature  
In this section, the top three physics-inspired metaheuristic algorithms are compared with the state of the art. For this comparison, results from other KNN-metaheuristic communications reported by [PERSON] et al. [37] were chosen. The metaheuristics chosen for comparison draw their metaphor inspiration from various sources. For example, the Grey Wolf Optimizer (GWO) and Whale Optimization Algorithm (WOA) may be classified as mammal inspired, whereas moth Flame Optimization (MFO) and the Butterfly Optimization Algorithm (BFO) are insect inspired. Similarly, Harris Hawk Optimization (HHO) and the Marine Predator Algorithm (MPA) are inspired by preying behavior seen in nature. Additionally, results based on popular ML algorithms such as Naive Bayes, Logistic Regression, Random Forest, Support Vector Machine (SVM), K-NN, Decision Tree, and Stochastic Gradient Descent (SGD) are also compared, along with their principal component analysis (PCA)-enhanced versions. In terms of classification accuracy (Table 7), in two out of the three datasets compared, EO outperformed all the other methods. In fact, for the breast cancer dataset and ionosphere dataset, EO was on average 12.75% and 7.12% better, respectively, than the metaheuristics presented in [37]. In the sonar dataset, too, EO and SCA were within 2.5% of the best solution reported in [37]. Additionally, when compared with the ML algorithms, the EO solution for the breast cancer dataset was on average 17.89% better. An average superiority of 5.68% was seen for EO when compared with the PCA-ML methods reported in [38]. The average features selected for the breast cancer, ionosphere, and sonar datasets by the various metaheuristics are reported in Table 8. It can be observed that the feature reductions by the current physics-inspired metaheuristics are much higher. For the breast cancer, ionosphere, and sonar datasets, the average percent feature reduction achieved by the three physics-inspired algorithms was 85.11%, 88.24%, and 84.89%, respectively, and for the metaheuristic algorithms from [37], it was only 67.62%, 64.71%, and 67.14%, respectively. Thus, from the comprehensive comparisons shown so far, it is clear that the current KNN hybridized physics-inspired metaheuristic algorithms (especially EO, SCA, and HGSO) are superior to those reported in the literature. Moreover, it is seen that even solutions by hybridized ML algorithms (for example, by dimensionality reduction techniques such as PCA) were inferior to current solutions. This is worth highlighting, since the current wrapper methods are much simpler in terms of computational complexity as compared to the PCA-hybridized ML methods. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset** & **Stats** & **SA** & **GSA** & **SCA** & **ASO** & **HCSO** & **EO** \\ \hline \multirow{3}{*}{DS1} & Avg. fitness rank & 5 & 3 & 6 & 1 & 4 & 2 \\  & Avg. accuracy rank & 5 & 3 & 6 & 1 & 4 & 2 \\  & AFS rank & 6 & 1 & 4 & 3 & 2 & 4 \\ \hline \multirow{3}{*}{DS2} & Avg. fitness rank & 6 & 5 & 3 & 2 & 4 & 1 \\  & Avg. accuracy rank & 6 & 5 & 3 & 2 & 4 & 1 \\  & AFS rank & 4 & 5 & 1 & 6 & 3 & 2 \\ \hline \multirow{3}{*}{DS3} & Avg. fitness rank & 6 & 1 & 5 & 3 & 4 & 2 \\  & Avg. accuracy rank & 6 & 1 & 5 & 3 & 4 & 2 \\  & AFS rank & 4 & 4 & 2 & 6 & 1 & 3 \\ \hline \multirow{3}{*}{DS4} & Avg. fitness rank & 6 & 4 & 3 & 5 & 2 & 1 \\  & Avg. accuracy rank & 5 & 4 & 3 & 5 & 2 & 1 \\  & AFS rank & 6 & 4 & 1 & 5 & 2 & 3 \\ \hline \multirow{3}{*}{DS5} & Avg. fitness rank & 6 & 5 & 1 & 4 & 3 & 1 \\  & Avg. accuracy rank & 6 & 5 & 1 & 4 & 3 & 1 \\  & AFS rank & 6 & 5 & 2 & 4 & 3 & 1 \\ \hline \multirow{3}{*}{D6} & Avg. fitness rank & 6 & 3 & 4 & 1 & 5 & 2 \\  & Avg. accuracy rank & 6 & 3 & 4 & 1 & 5 & 2 \\ \cline{1-1}  & AFS rank & 6 & 5 & 2 & 4 & 1 & 3 \\ \hline \multirow{3}{*}{D56} & Avg. Rank & 5.61 & 3.66 & 3.11 & 3.33 & 3.11 & **1.88** \\ \cline{1-1}  & & & & & & & \\ \end{tabular}
\end{table} 
Table 6: Average rank of all averages (Fitness, accuracy, and AFS).