## 4 Experimental Design and Setup
### Datasets  
In this empirical study, we used seven publicly available datasets containing details of source code metrics and refactoring collected from a total of 37 subsequent releases of systems from seven open-source Java systems located in GitHub [27]. These are combined into one manually validated dataset for each of the seven systems [13]. We select these datasets particularly because many research studies have explored the utilization of prediction models in software maintainability. Most of these studies have predicted change maintenance effort using CHANGE metric and maintainability index, but little progress has been made in predicting change-proneness in software maintainability [1]. In addition, these datasets are the only datasets available on the PROMISE repository and suitable for predicting change-proneness. To the best of the author's knowledge, these datasets are considered the newest datasets in software maintainability prediction and have not been utilized in previous studies to predict change-proneness. Table 3 provides a summary of the datasets used in this study, which includes the dataset name, number of classes, number of releases, time interval, and URL [13]. The datasets were initially collected to investigate the impact of code refactoring (changes made to the structure of the internal source code which do not affect the functionality or external behavior of the code [8]) on maintainability, and the original datasets contain source code metrics including refactoring metrics, along with a score for maintainability at both method and class levels [13] We removed some metrics to convert the refactoring datasets into ones that are suitable to predict change-proneness at the class level. To achieve this, we performed three steps:  
* **Step 1. ** We removed the relative maintainability index (RMI), which is the dependent variable in the original datasets, because it is a derived variable (from some of the independent variables which will cause any machine learning model to just learn the function that defines it) and also not an accurate reflection of maintenance effort. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Dataset Name** & **\# Classes** & **\# Release** & **Time Interval** & **URL** \\ \hline antfr4 & 436 & 5 & 21 January 2013–22 January 2015 & [[https://github.com/antlr/antfr4](https://github.com/antlr/antfr4)]([https://github.com/antlr/antfr4](https://github.com/antlr/antfr4)) (accessed on 4 June 2020) \\ \hline junit & 657 & 8 & 13 April 2012–28 December 2014 & [[https://github.com/junit-team/junit](https://github.com/junit-team/junit)]([https://github.com/junit-team/junit](https://github.com/junit-team/junit)) (accessed on 4 June 2020) \\ \hline MapDB & 439 & 6 & 1 April 2013–20 June 2015 & [[https://github.com/jankotel/MapDB](https://github.com/jankotel/MapDB)]([https://github.com/jankotel/MapDB](https://github.com/jankotel/MapDB)) (accessed on 4 June 2020) \\ \hline mcMMO & 301 & 5 & 24 June 2012–29 March 2014 & [[https://github.com/mcMMO-Dev/mcMMO](https://github.com/mcMMO-Dev/mcMMO)]([https://github.com/mcMMO-Dev/mcMMO](https://github.com/mcMMO-Dev/mcMMO)) (accessed on 4 June 2020) \\ \hline mct & 2162 & 3 & 30 June 2012–27 September 2013 & [[https://github.com/nasa/mct](https://github.com/nasa/mct)]([https://github.com/nasa/mct](https://github.com/nasa/mct)) (accessed on 4 June 2020) \\ \hline oryx & 536 & 4 & 11 November 2013–10 June 2015 & [[https://github.com/cloudera/oryx](https://github.com/cloudera/oryx)]([https://github.com/cloudera/oryx](https://github.com/cloudera/oryx)) (accessed on 4 June 2020) \\ \hline titan & 1486 & 6 & 7 September 2012–13 February 2015 & [[https://github.com/thinkaurelius/titan](https://github.com/thinkaurelius/titan)]([https://github.com/thinkaurelius/titan](https://github.com/thinkaurelius/titan)) (accessed on 4 June 2020) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary of the datasets. * **Step 2.