## 4 Methodology
### Evaluation Function  
In FS, the best subset of features is the one that maximizes the classification accuracy with a lower number of features. The fitness function \(f(\cdot)\) can be defined as follows:  
\[f(X)=max(Acc(X)+w_{f}\times(1-\frac{L_{f}}{L_{t}})) \tag{9}\]  
where \(Acc\) is the classification accuracy under the candidate solution \(X\), and \(w_{f}\in(0,1]\) is a factor used to scale and adjust the contribution of a mathematical term of the given equation. In other words, \(w_{f}\) is a factor used in math to reduce the weight of some mathematical term in a given equation. When \(w_{f}\) is closer to \(0\), it reduces the credits of the corresponding term, while a \(w_{f}\) closer to \(1\) maintains its full influence. \(L_{f}\) is the number of features in the current solution, and \(L_{t}\) is the total number of features. Note that, in our work, \(L_{t}=d\) since we have \(d\) features in total. For example, assume a dataset with \(40\) attributes, \(w_{f}=1\), two solutions achieved, with \(97\%\) and \(97\%\) accuracies, respectively. The first one has \(20\) features, and the second one has \(30\) features. Then, the \(f\) of the first one is \(97.5\%\), and the \(f\) of the second one is \(97.25\%\), which means that first solution is better than the second one. The first one is better since it yields the same accuracy as the second one but with \(10\) fewer features.  
In this work, we started by using the K-Nearest Neighbor (KNN) classifier with \(10\)-fold cross-validation. KNN is one of the easiest classifiers with effective performance (accuracy) [115]. KNN is applicable to handling different tasks like classification, regression, and searching problems. It is known as a lazy algorithm [116; 117], which means that no calculations are conducted in the training phase until the point of prediction, which means that no training phase is associated with KNN. It starts by computing the distances when it reaches the first point in the testing phase to determine which class label the current data point refers to by calculating the similarity (distance) between each data point and all other data points that are booked for the training portion. Then, upon a predefined K value, KNN decides which class the data point (tuple) refers to.  
KNN uses a set of distance functions such as the Euclidean distance (ED), Mahalaobis distance, Manhattan distance (MD) [118], Earth movers distance, Chebyshev distance, and Canberra distance. We set K to 3, and we used an MD measurement in this work. Recall that MD is defined as follows:  
\[dis(X,Y)=\sum_{i=1}^{d}|x_{i}-y_{i}| \tag{10}\]  
where \(dis(X,Y)\) is the distance between \(X,Y\) data vectors (data points). Note that \(|x_{i}-y_{i}|\) is the absolute subtraction between the two vectors of features. Here, two important factors that have to be taken in consideration when utilizing KNN classification algorithm are bias and variance. This is because the learning error rate increases (bias increases) by increasing K, but the testing error may be decreased (variance decreases), and vice versa.  
We emphasize that we also considered other classifiers in our work to test the effectiveness of our proposed FS approach, including logistic regression, XGBoost, and Naive Bayes.