# Investigating the Applications of Machine Learning Techniques to Predict the Rock Brittleness Index
## 2 Methodology  
### Models Developed  
The models that were developed in this study are CHAID, RF, SVM, KNN, and ANN. The CHAID is from decision tree family which produce non-binary tree structure. This technique that was developed by [PERSON] [41] employs a chi-square test to produce multiple sequential combinations and splits, and finally a single decision tree. Typically, the decision tree techniques are susceptible to overfitting. However, the CHAID automatically prunes the tree to alleviate the overfitting phenomenon. Moreover, the CHAID generates a number of rule sets and each of these rule sets has a confidence level and accuracy.  
While the single-based decision trees are easy to implement and understand, these techniques are prone to result in different generalization behaviour with small changes to the training data [42]. Indeed, these techniques are viewed as unstable and high variance. The RF technique is extensively efficient to remedy the abovementioned shortcomings of single-based decision trees. This technique was developed by [PERSON] [43] and is an ensemble-based approach (Figure 1). The RF generates more accurate prediction results compared to single-based trees since it combines a huge number of single trees. It is worth pointing out that the RF enjoys a bagging approach to create each number of an ensemble from diverse datasets. This approach randomly chooses from the space of single decision trees and generates almost identical (low diversity) predictions.  
Other ML techniques such as KNN, SVM, and ANN are also powerful tools for classification and regression analysis in civil and mining problems [35]. KNN is an easy to implement, simple and effective data mining algorithm [44]. The basic theory behind the KNN is disce  
Figure 1: RF structure.  
samples (e.g., employing the distance functions) which has the nearest distance from unknown samples in the calibration dataset. Moreover, the KNN identifies the class of unknown samples among the \"\(k\)\" samples by calculating the average of the response variables [45]. Thus, the \"\(k\)\" plays an important role in the KNN performance [46].  
Concerning the SVM, this technique is capable to handle the high dimensional and linearly non-separable datasets [47; 48]. In addition, it can reduce the error for training and testing datasets, as well as the model complexity [49]. According to [PERSON] and [PERSON] [50], statistical learning theory is the basic theory behind the SVM. In addition, the performance of SVM is influenced by Kernel functions, such as linear, radial basis function, sigmoid and polynomial [51]. It is noteworthy to mention that the SVM aims to determine a perfect separation hyper plane that can distinguish the two classes [51]. The SVM regression aims to discover the largest margin. Figure 2 shows a typical structure of SVM.  
In terms of ANN, this technique is a kind of artificial intelligence which emulates some functions of an individual's mind. Typically, the ANN is tended to sort experiential knowledge [52]. This technique includes a series of layers, and each layer includes a sequence of neurons. These neurons in every layer are connected thorough weighted links to all neurons on the previous and following layers [52; 53; 54; 55].  
A positive weight reveals an excitatory association, whereas a negative weight reveals an inhibitory association. A typical ANN includes three layers, i.e., input layer, hidden layer, and an output layer [56; 57; 58; 59; 60]. This structure is shown in Figure 3 for more illustration.  
Figure 3: Typical three-layer neural network.  
Figure 2: Typical structure of SVM.