## 4 Experimental Design and Setup
### Datasets
#### 4.1.3 Dataset Analysis  
The primary objective of the dataset analysis is to detect any attributes that contain values that might cause problems when building any prediction models and produce skewed or misleading results. The descriptive statics of these metrics are presented in Supplementary A in the reproduction package. The minimum values are zero or one in all datasets. In contrast, the maximum values are 575 in CLOC metric in antlr4 dataset, 662 in TNM metric in junit dataset, 11,272 in LLOC metric in MapDB dataset, 1160 in LOC metric in mcMMO dataset, 1390 in WarningInfo metric in mct dataset, 179 in NII metric in oryx dataset, and 1104 in WarningMinor metric in titan dataset. Thus, there was a considerable difference between the minimum and maximum values in all datasets. For this reason, normalization was applied, which will be described in the next section. Another important finding was that the average values of WarningInfo metrics were high in all datasets, ranging from 8 to 15. The remaining metrics have different values of descriptive static, which suggests that the datasets have varying characteristics. \begin{table}
\begin{tabular}{l l} \hline \hline
**Category** & **Metrics** \\ \hline \hline \multicolumn{2}{l}{Cohesion} & \multicolumn{1}{l}{LCOM5} \\ \hline \hline \multicolumn{2}{l}{Complexity} & NL, NLE and WMC \\ \hline \hline \multicolumn{2}{l}{Coupling} & CBO, CBOI, NII, NOI and RFC \\ \hline \hline \multicolumn{2}{l}{Documentation} & AD, CD, CLOC, DLOC, PDA, PUA, TCD, and TCLOC \\ \hline \multicolumn{2}{l}{Inheritance} & DIT, NOA, NOC, NOD and NOP \\ \hline \hline \multirow{2}{*}{Size} & LOC, LLOC, NA, NG, NLA, NLG, NLM, NLPA, NLPM, NLS, NM, NPA, NPM, NS, NOS, TLOC, TLLOC, TNA, TNG, TNLA, TNLG, TNLM, TNLPM, TNLS, TNM, TNPA, TNPM, TNS and TNOS \\ \hline \multicolumn{2}{l}{Code duplication} & CCL, CCO, CC, CI, CLC, CLLC, LDC and LLDC \\ \hline \hline \multicolumn{2}{l}{Warning} & WarningBlocker, WarningCritical, WarningInfo, WarningMajor and WarningMinor \\ \hline \hline \multirow{8}{*}{Rules} & Android Rules, Basic Rules, Bruce Rules, Clone Implementation Rules, Clone Metric Rules, Code Size Rules, Cohesion Metric Rules, Comment Rules, Complexity Metric Rules, Controversial Rules, Coupling Metric Rules, Coupling Rules, Design Rules, Documentation Metric Rules, Empty Code Rules, Finalizer Rules, Import Statement Rules, Inheritance Metric Rules, JJERE Rules, JUnit Rules, Jakara Commons Logging Rules, Java Logging Rules, JavaRules, MigratingToUnit Rules, Migration Rules, Migration13 Rules, Migration14 Rules, Migration15 Rules, Naming Rules, Optimization Rules, Security Code Guideline Rules, Size Metric Rules, Strict Exception Rules, String and StringBuffer Rules, Type Resolution Rules, Unnecessary and Unused Code Rules and \\ \multicolumn{2}{l}{Vulnerability Rules} \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Metrics used as independent variables and their category. Firstly, we drop metrics that have zero values (Supplementary B in the reproduction package), and therefore cannot be used as predictors (see [PERSON] et al. [29]). In total, 21 metrics were removed from some of the datasets, and 7 metrics were removed from all datasets (see summary metrics removed in the Supplementary A in the reproduction package). Secondly, we calculate the Spearman correlation, which is a well-known statistical measurement to compute the strength (i.e., strong or weak) and the direction (i.e., positive or negative) of relationship between two ranked variables [30]. Spearman correlation was selected in this study because it is appropriate for the monotonic relationship, which indicates that if one variable increases, the other variable either increases or decreases. From this, we create a correlation matrix to identify the dependence between variables (Supplementary B in the reproduction package).