# Investigating the Applications of Machine Learning Techniques to Predict the Rock Brittleness Index
## 3 Modelling Process and Results
### Evaluation of the Developed Models
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Performance Index**} & \multicolumn{3}{c}{**RF**} & \multicolumn{3}{c}{**CHAID**} & \multicolumn{3}{c}{**RNN**} & \multicolumn{3}{c}{**SVM**} & \multicolumn{3}{c}{**ANN**} \\ \cline{2-13}  & \multicolumn{3}{c}{**TR**} & \multicolumn{3}{c}{**TE**} & \multicolumn{3}{c}{**TR**} & \multicolumn{3}{c}{**TE**} & \multicolumn{3}{c}{**TR**} & \multicolumn{3}{c}{**TE**} & \multicolumn{3}{c}{**TR**} & \multicolumn{3}{c}{**TE**} & \multicolumn{3}{c}{**TR**} & \multicolumn{3}{c}{**TE**} & \multicolumn{3}{c}{**TR**} & \multicolumn{3}{c}{**TE**} \\ \hline  & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** & **Value** & **Rank** \\ \hline \(R^{2}\) & 0.89 & 5 & 0.75 & 2 & 0.77 & 3 & 0.74 & 1 & 0.81 & 4 & 0.84 & 4 & 0.73 & 1 & 0.84 & 3 & 0.75 & 2 & 0.85 & 5 \\ RMSE & 1.08 & 5 & 1.75 & 2 & 1.99 & 3 & 1.8 & 1 & 1.33 & 4 & 1.44 & 3 & 1.57 & 1 & 1.42 & 4 & 1.50 & 2 & 1.39 & 5 \\ VMF (\%) & 86.84 & 5 & 80.5 & 2 & 78.31 & 3 & 75.1 & 1 & 80.62 & 4 & 86.1 & 3 & 72.62 & 1 & 87.5 & 5 & 74.00 & 2 & 87 & 4 \\ MAE & 0.91 & 5 & 1.36 & 1 & 1.14 & 4 & 1.35 & 2 & 1.16 & 3 & 1.15 & 4 & 1.32 & 1 & 1.06 & 5 & 1.29 & 2 & 1.16 & 3 \\ \(d20\)-index & 1.00 & 5 & 0.91 & 2 & 0.95 & 2 & 0.84 & 1 & 0.99 & 4 & 0.97 & 4 & 0.91 & 1 & 0.96 & 3 & 0.97 & 3 & 0.97 & 5 \\ \hline Sum of the ranks & TR & 25 & TE & 9 & TR & 15 & TE & 6 & TR & 19 & TE & 18 & TR & 5 & TE & 20 & TR & 11 & TE & 22 \\ \hline \multicolumn{13}{c}{**34**} & \multicolumn{3}{c}{**21**} & \multicolumn{3}{c}{**37**} & \multicolumn{3}{c}{**25**} & \multicolumn{3}{c}{**33**} \\ \hline \hline \multicolumn{13}{c}{Perfect \(R^{2}=1\) Perfect RMSE = 0: Perfect VMF = 100\%\_Perfect \(MAE=0\), \(d20\)-index = 1. Training dataset = TR; Testing dataset = TE} \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation of Models Developed Using Five Performance Indices. Typically, the higher lines show better models, particularly on the left side of the chart. To compare a model developed and the at-chance model, the area between a model and the red line can be used. In fact, this area identifies how much better a proposed model is compared to the at-chance model. Additionally, the area between a model proposed and the perfect model identifies where a proposed model can be improved. For training stage, it is shown that the perfect model has correctly identified 100% of the samples, which had the \(BI\) of greater than 16.458, at the percentile of 40%. The RF model was the closest follower of the perfect model and correctly identified 100% of the samples, which had the \(BI\) greater than 16.458, at the percentile of 50%. The weakest model was the CHAID which identified the hit at the percentile of 84%. For testing dataset, it can be seen that that the perfect model has correctly identified 100% of the samples, which had the \(BI\) of greater than 16.458, at the percentile of 35%. The perfect model was followed by the RF and ANN models which had correctly identified 100% of the samples, which had the \(BI\) of greater than 16.458, at the percentile of 41%. The KNN and ANN had the weakest performance and identified the hit at the percentile of 47%.