## 2 Materials and Methods
### Machine Learning Algorithms
#### 2.3.4 Performance and Error Scales Analysis  
In these algorithms, different success criteria are used in the performance analysis. These are precision, sensitivity, F-measure and ROC criteria. ROC criteria are Receiver Operating Characteristics. A graphical representation is used to evaluate the diagnostic ability of a binary classification model by plotting the True Positive Rate (Sensitivity) against the False Positive Rate (1--Specificity) at various threshold settings. The curve helps visualize the trade-off between sensitivity and specificity, and the Area Under the Curve (AUC) is often used as a metric to summarize the model's performance, with a higher AUC indicating better discriminatory ability. While calculating the values of these criteria, the estimated and the available data are compared [25]. In the evaluation process, TP (true positive), TN (true negative), FP (false positive) and FN (false negative) values are used. The confusion matrix created with these values is manually generated to describe the performance of the classification model. The confusion matrix generated with this information is shown in Figure 3 below.  
Precision is described as the proportion of matching examples (true positives) out of all examples predicted to belong to a particular class.  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{6}\]  
Recall is described as the fraction of examples that were predicted to belong to a class concerning all of the examples that truly belong in the class.  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{7}\]  
The F-measure provides a single score that balances both the concerns of precision and recall in one number.  
\[\text{F}-\text{measure}=\frac{2\times\text{Recall}\times\text{Precision}}{ \text{Recall}+\text{Precision}} \tag{8}\]  
The ROC value is obtained from the curve developed to interpret the performance of the algorithm. Error measures of the model are mean square error (MAE), root mean square error (RMSE) and kappa statistics. The MAE measures the average size of errors in a set of predictions without regard to their direction. It measures the accuracy of continuous variables. In other words, the MAE is the average over the confirmation sample of the  
Figure 3: Confusion matrix.  
absolute values of the differences between the forecast and the corresponding observation. The RMSE is a quadratic evaluation rule that measures the average level of error. The difference between the estimate and the observed values is squared and then averaged over the sample. Finally, the square root of the average is obtained. As the errors are squared before being averaged, the RMSE gives a relatively high weight to large errors. This means that the RMSE is most useful when large errors are particularly undesirable. The MAE and RMSE can be used together to identify the variance of the errors in a set of predictions [26]. The RMSE is always greater than or equal to the MAE; the greater the difference, the greater the variance in individual errors in the sample. If the RMSE = MAE, then all errors are of the same size. The MAE and RMSE are formulated as Equations (9) and (10).  
\[\text{MAE}=\frac{1}{\text{N}}{\sum}_{i=1}^{\text{N}}{\left|{{x_{i,i}}-{x_{ \text{o,i}}}}\right|} \tag{9}\]  
\[\text{RMSE}=\sqrt{\frac{1}{\text{N}}{\sum}_{i=1}^{\text{N}}{\left({x_{i,i}-{x_ {\text{o,i}}}}\right)^{2}}} \tag{10}\]  
where \(x_{f,i}\) and \(x_{o,i}\) are the ith expectation and observation, respectively.  
The kappa is a type of the correlation coefficient. Similar to these coefficients, it can vary from \(-1\) to \(+1\), where \(0\) symbolizes the quantity of agreement that can be expected from arbitrary chance, and \(1\) signifies an excellent agreement between the raters. As with all correlation statistics, the kappa is a standardized value and hence is construed similarly across multiple research [27]. The kappa value is formulated as Equation (11).  
\[K=\frac{{p_{o}}-{p_{e}}}{{1-{p_{e}}}} \tag{11}\]