## 3 Methodology
### Experimental Setup  
This section discusses the experimental setup of the comparative performance analysis of FS methods as depicted in Figure 1. The experimental setup can be described based on three major steps.  
**Step 1**: At first, four FFR methods as presented in Table 1 are applied to the original full software defect datasets. In this case, each of the FFR methods (IG, RFA, GR, and CV based on Ranker Search method) assesses and ranks the software features based on their respective characteristics. \(\log_{2}N\) features were selected from the ranked list provided by each of the FFR methods. Specifically, the first six features were selected from the resulting rank list of the FFR methods. The \(\log_{2}N\) (where N is number of features in each software defect datasets) was adopted according to the work of [PERSON] et al. [27] which indicated that it is better to cull \(\log_{2}N\) features; many empirical studies have followed this procedure [26; 28; 29; 32]. According to Table 4, the value for \(N\) in this study is (\(N>=21\)) and this is due to the choice of different software defect datasets for the experiment. In the end, reduced datasets in terms of attributes are generated.  
**Step 2**: Secondly, the two FSS methods in Table 2 and the respective search methods in Table 3 are applied to the original full datasets. The two FSS (CFS, CNS) methods with seven search methods (BFS, BAT, FS, AS, GSS, PSOS, and GS) making fourteen FSS methods to apply on the dataset. Each of  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Classifiers** & **Description** & **Parameter Setting** \\ \hline Naïve Bayes (NB) & A Bayes Theorem based & NumDecimalPlaces = 2; \\  & classification technique. & NumAttrEval = Normal Dist. \\ \hline Decision Tree (DT) & A Tree-based classification & Confidence factor = 0.25; \\  & technique. & MinNumObj = 2 \\ \hline  & & Kernel = PolyKernel (E = 1.0; \\  & A Function-based classification & C = 250007); lambda = 0.01; \\  & technique. & Quadratic penalty = BFGS \\  & & Optimization \\ \hline K Nearest Neighbor (KNN) & An Instance learning-based classification technique. & K = 1; NNNSearch = LinearNNSearch (based on \\  & & Euclidean Distance) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Classification Algorithms.  
Figure 1: Experimental Setup.  
the FSS methods generates a subset of features that have high predictive potentials. The respective parameter settings of the search methods as depicted in Table 3 were used. As in the case of FFR methods, reduced datasets are also generated at the end of this step.  
**Step 3**: This step involves the prediction process. That is the application of classifiers in Table 4 on the software defect dataset (Filtered datasets from Step 1 and 2). Four classifiers (NB, DT, LR, and KNN) were applied on both the reduced (Filtered datasets from Step 1 and 2) and full datasets. The essence of this step is to show the efficacy of reduced software features in SDP. Each of the experiments was carried out using the 10-folds cross-validation (CV) method. This is to avoid the issue of biases and overfitting of the prediction models and also to reduce the effect of class imbalance which is one of the data quality problems in data mining [29; 37]. In addition, due to the random nature of the search methods of the FSS methods, each experiment involving FSS methods were performed 10 times to validate the result. Eventually, a total of 2900 ((4 FFR methods \(\times\) 5 datasets \(\times\) 4 classifiers) + (14 FSS methods \(\times\) 5 datasets \(\times\) 4 classifiers \(\times\) 10 runs) + (4 classifiers \(\times\) 5 datasets)) distinct experiments were carried out in this study.