## 5 The Results of the Experiment
### Impact of Software Metrics-RQ2  
The basic analysis of changes in metrics was done first. Here, all the metric changes in all previous releases were grouped by software metrics. Table 8 shows the descriptive statistic values of changes in software metrics grouped by the change of MINC. Even though the changes are small, the sample size of classes analysed in the experiment was sufficient that the statistical test could be applied for comparison of two independent groups. As the data of software metric changes are not normally distributed (Shapiro-Wilk returned p <0.001 for every metric in both groups/MINC changes), a non-parametric Mann-Whitney U test was used to compare the software metric changes. The groups of classes compared were two: The classes where MINC rose (Up) and the classes where MINC fell (Down). The results of the Mann-Whitney U test showed that there are statistically significant differences between the two groups of classes in all changes in software metric (p <0.001 for all of six software metric changes). The small changes in means show that, in most cases, the software metric was not  
Figure 7: Classification accuracy (**a**) and F-score (**b**) metrics of prediction of the change in MINC with the addition of historical changes for both classifiers. Dotted lines present logarithmic trends. changed at all. This shows that data have high kurtosis, with values of metric changes concentrated around 0 (no change). The latter is primarily a consequence of the fact that only a part of the classes that are subject of modification (e.g., bug fix or modified functionality) is changed in a single release. This imbalance in independent values could provide a challenge in using it in the prediction model, as will be demonstrated in the following sections. Next, the ranking of features analysis was performed to assess the impact of individual software metrics to the prediction of MINC change. Table 9 depicts the results of the assessment of how particular software metrics used in the experiment contribute to the software maintainability prediction model throughout the iterations of adding historic measurement data. The software metrics were ranked by the strength of contribution, with 1 indicating the strongest and 6 indicating the weakest contribution of the software metrics to the prediction model. For the analysis, there are attached reference values of the software metrics' contribution to the model in row N-0 of Table 9. Reference values in N-0 contain measurements of C&K software metrics and measurement of the MINC metric all in the same version, and not software metric versions from the past and MINC from the future. In this layer, the C&K measurement values and values of MINC metric are directly dependent, namely, the values of the MINC metric are calculated based on the software metric measurements. That is the main reason why the measurements of layer N-0 were excluded from the experimental data, and serves only as a reference in comparison of the results of the experiment. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Software** & **MINC** & **N** & **Mean** & **\(\Delta\) Mean** & **Std. Dev.** & **Min** & **Max** & **Kurtosis** & **Mann-Whitney U** \\ \hline \multirow{3}{*}{CBO} & Down & 12,5802 & 0.0543 & \multirow{3}{*}{0.0098} & 1.5246 & \multirow{3}{*}{-316.0} & \multirow{3}{*}{55.0} & \multirow{3}{*}{15,223} & U = 20407429629 \\  & Up & 94,248 & 0.0641 & & & & & \\  & Down & 125,802 & 0.0009 & & & & & \\  & Up & 94,248 & 0.0005 & & & & & \\  & Down & 125,802 & 0.0004 & & & & & \\  & Up & 94,248 & 0.0006 & & & & & \\  & Up & 94,248 & 0.0005 & & & & & \\  & Down & 125,802 & 0.0004 & & & & & \\  & Up & 94,248 & 0.0006 & & & & & \\  & Down & 125,802 & 0.0075 & & & & & \\  & Up & 94,248 & 0.0062 & & & & & \\  & Down & 125,802 & 0.0789 & & & & & \\  & Up & 94,248 & 0.0935 & & & & & \\  & Down & 125,802 & 0.1807 & & & & & \\  & Up & 94,248 & 0.1942 & & & & & \\ \hline \hline \end{tabular}
\end{table}