## 5 The Results of the Experiment
### Use Case on One Software Repository
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & **CBO** & **DIT** & **LCOM** & **NSUB** & **RFC** & **TCC** & **MINC** \\ \hline Average & 11.06 & 1.41 & 0.24 & 0.30 & 11.81 & 24.04 & 117.20 \\ Median & 11.00 & 1.00 & 0.11 & 0.00 & 10.00 & 20.00 & 118.45 \\ St. Dev. & 7.77 & 0.52 & 0.31 & 0.63 & 7.52 & 18.18 & 11.40 \\ Min & 1.0 & 1.0 & 0.0 & 0.0 & 1.0 & 1.0 & 91.92 \\ Max & 44.0 & 3.0 & 1.0 & 3.0 & 32.0 & 82.0 & 142.44 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Descriptive statistics about C&K software metrics and MINC of the _httpcore_ project source code files used in the analysis. The project, which is hosted on GitHub, has in total 31 code contributors, and six of them with the contribution of more than 1000 lines of code. A MINC change prediction model can be built with only the historical data of this project, so no general patterns about the codebase development were extracted, but only project specifics were considered. Again, random forest, with 5000 classification trees, is trained in steps where more historical changes (more changes between versions) are considered. The data are split with 10-fold cross-validation in each step, and the results are averaged across folds. Figure 8 shows that the prediction of MINC change of the model built only on this project is better than the prediction ability of the general model. The best model (N-15) produced CART and Random Forest models with accuracies 0.78 and 0.87 with F-score of 0.74 and 0.86 respectively, versus the general models (from the previous Section), with accuracies of 0.58 both times and F-scores of 0.53 and 0.55 for CART and random forest. Again, this is logical, as only project-specific patterns are extracted and not the general ones. Figure 8: Classification accuracy (**a**) and F-score (**b**) metrics of prediction of the change in MINC with the addition of historical changes for both classifiers for project _httpcore._ Dotted lines present the performance of the best general model. The classification metrics' accuracy and average F-score of this project-specific model are far better in later steps than the classification metrics of the general models. This supports the hypothesis that focusing on one project can produce better results than trying to extract the general knowledge. There is a small drop of classification performance after N-15, which could be attributed to too wide data (too many features and too few instances), which could be mitigated with feature selection or data over-sampling. The bad classification performance of earlier steps (from N-1 to N-5) only supports our hypothesis that including more historical changes from the past improves the future MINC change prediction significantly. Of course, the good performance using this approach on other projects individually would not be guaranteed. The information needed for future MINC change prediction is not always reflected in source code, and, consequently, software metrics, as in this project. Naturally, these project-specific models are over-fitted to this project, and would perform worse on any other project than the general model. With this approach, the MINC change prediction model must be built individually for every project. This could not be usable and doable for every software project development process (small teams with limited hardware capabilities and machine learning knowledge), but can still be incorporated as a static analysis done overnight on a dedicated system, similar to overnight testing of committed changes. The analysis for this experiment was done on a computer with Intel i7-4790 CPU @ 3.6 GHz with four cores, 24 GB of RAM memory.