## 5 Empirical Study
### Data Preprocessing  
Data preprocessing is an essential task to prepare and transform the data into a suitable form for machine learning models. Handling missing values is one of the important preprocessing steps. Different methods were proposed to handle the missing values, such as mean imputation, median imputation, removing missing values, and nearest neighbor imputation. Mean imputation substitutes missing values with the mean of all known values of the column. It is one of the most commonly used techniques and has been shown to be efficient with supervised machine learning models [44]. Thus, in this study, we utilized the mean imputation to deal with the missing values.  
Dealing with high-dimensional data is a challenging task before we build machine learning models. Feature selection can deal with this issue by selecting the most relevant features [45]. There are three main feature selection categories: filter, wrapper, and embedded methods. Filtering methods are faster than the other methods because they filter the attributes independently from the classification model. Filtering methods filter the attributes based on a specific metric such as chi-square, information gain, or gain ratio. Information gain calculates the dependency between independent attributes (features) and dependent attributes (target variable). It can be calculated as follows:  
\[\text{Information Gain}(Y,X)=E(Y)-E(Y|X) \tag{5}\]  
where:  
* \(E(Y)\) is the entropy of the target variable (\(Y\)).
* \(H(Y\,|\,X)\) is the conditional entropy of \(Y\) given \(X\).  
However, feature selection using information gain suffers from bias towards features that have more distinct values. Thus, the gain ratio was proposed to decrease the bias in information gain by taking into account intrinsic information. The gain ratio is a normalized  
\begin{table}
\begin{tabular}{c c c c c} \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**Number of**} & \multirow{2}{*}{**Number of Samples**} & \multicolumn{2}{c}{**Number of**} & \multirow{2}{*}{**Imbalance Ratio**} \\  & **Individual Features** & & & **Defective Samples** & \\ \hline CM1 [39] & 37 & 327 & 42 & 6.78 \\ PC2 [39] & 36 & 745 & 16 & 45.56 \\ MC1 [39] & 38 & 1988 & 46 & 42.21 \\ MC2 [39] & 39 & 125 & 44 & 1.84 \\ JM1 [39] & 21 & 7782 & 1672 & 3.65 \\ Eclipse-2.0 [40] & 31 & 6729 & 975 & 5.90 \\ Tomcat [43] & 20 & 1162 & 77 & 14.09 \\ Prop-43 [43] & 20 & 11,679 & 341 & 33.24 \\ Zxing [41] & 51 & 399 & 83 & 2.38 \\ Lucene [42] & 17 & 691 & 64 & 9.79 \\ \hline \end{tabular}
\end{table}
Table 3: Datasets description.  
version of information gain. Equations (6) and (7) present the equations of gain ratio and intrinsic information, respectively.  
\[\text{Gain ratio}(Y,X)=\frac{\text{Information Gain (Y,X)}}{\text{Intrinsic value (X)}} \tag{6}\]  
\[\text{Instrinsic value (X)}=-\sum\frac{|S_{i}|}{S}\times log_{2}\frac{|S_{i}|}{S} \tag{7}\]  
where:  
* \(|S|\) is the number of all values of the feature X.
* \(|S_{i}|\) is the count of each distinct value of the feature X.  
In this study, we utilized gain ratio to select the most relevant features which was used previously in the literature with ensemble learning in the field of software defect prediction [21].