# Software Risk Prediction: Systematic Literature Review on Machine Learning Techniques
## 5 Results and Discussions
### Answers to the Research Questions
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Category** & **Sample Size in Dataset** & **No. PS** & **Percentage** \\ \hline Small & Less than 100 & 7 & 43.75\% \\ \hline Medium & 100 to 200 & 2 & 12.50\% \\ \hline Large & More than 200 & 4 & 25.00\% \\ \hline Unknown & Unknown Size & 3 & 18.75\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Category of datasets based on the number of projects. Figure 3: Average number of SRPM studies. RQ06: What detecting techniques are employed in the development of SRPM models? Detection techniques, which are employed in the development of SRPM models, are divided into two major categories: (1) Classification model and (2) Regression model. Some other techniques were also applied, those not for detection but for descriptive analyses. Figure 5 shows the statistical description of these techniques:  
RQ07: How many of the SRPM studies employ the ML approach? As our main concern in this study is software risk prediction using machine learning, we need to know how many of the studies used a machine learning approach. The ratio of the ML approach studies can be seen in Figure 6.  
Figure 4: The ratio of Machine learning and Statistical approaches. Figure 5: Detecting techniques and other techniques used in the PS. RQ08: What are the various performance metrics used in SRPM studies? To evaluate the performance of the prediction models, performance metrics are used. There are several performance metrics that are accessible for evaluation, in general. These performance metrics are also used in the realm of SRPM research to assess and compare the findings obtained using various prediction approaches. The following are the key performance indicators and their descriptions:  
**Correctly Classified Instances:**: The sum of True Positive (_TP_) and True Negative (_TN_) refers to correctly classified instances. **Incorrectly Classified Instances:**: The sum of False Positive (_FP_) and False Negative (_FN_) refers to incorrectly correctly classified instances. **Accuracy:**: The number of correctly classified instances out of all the instances is known as accuracy. Accuracy can be expressed as:  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{1}\]
**Precision:**: The precision is measured as the ratio of the number of correctly classified positive instances to the total number of positive instances. Precision can be expressed as:  
\[Precision=\frac{TP}{TP+FP} \tag{2}\]
**Recall:**: The recall is obtained by dividing the total number of positive instances by the number of positive instances correctly classified as Positive. Recall can be expressed as follows:  
\[Recall=\frac{TP}{TP+FN} \tag{3}\]
**F-Measure:**: F-Measure is a method of combining precision and recall into a single measure that incorporates both. F-Measure can be expressed as:  
\[F\text{-}Measure=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{4}\]  
Figure 6: How many studies used machine learning approaches. **Receiver Operating Characteristic (ROC):** A graphical way to evaluate the performance of a classifier is a receiver operating characteristic (ROC) analysis. It evaluates a classifier's performance using two statistics: true positive rate and false positive rate [53]. **Mean Absolute Error (MAE):** The Mean Absolute Error (MAE) is a regression model assessment indicator. The MAE of a model with respect to a test set is the average of all individual prediction errors on all instances in the test set [54]. The discrepancy between the real value and the expected value for each instance is called a prediction error [55]. Mean Absolute Error (MAE) can be expressed as  
Figure 3:0  
where,  
\(\hat{y}_{i}=\) predicted value,  
\(y_{i}=\) true value,  
\(n=\) total number of instances. **Mean Squared Error (MSE):** Model evaluation metric Mean Squared Error(MSE) is frequently used with regression models. The MSE of a model with respect to a test set is the average of all squared prediction errors in the test set.