## 4 Machine Learning Methodology
### Algorithm Development  
The DT classifier was the algorithm selected to be trained based on the accuracy results collected in [20; 23; 25]. The software used to develop the classifier algorithm was MATLAB R2018b [35].  
The developed algorithm obtains an adjusted binary classification DT based on the input variables and responses [36; 37], in order to predict responses for new data. The obtained binary tree divides branching nodes based on the values of the attributes.  
Based on the predictors and responses explained in Sections 4.1 and 4.2, the dataset used in the algorithm development was as follows:  
\[\begin{split} T&=\left\{T_{1},T_{2},...,T_{m}\right\} \\ x_{i}&=\left\{x_{1},x_{2},...,x_{p}\right\}\\ S&=\left\{S_{1},S_{2},...,S_{m}\right\}\end{split} \tag{1}\]  
where \(T\) is the set of all observations, \(m\) is the number of observations, \(x_{i}\) is the set of values of a predictor, \(p\) is the number of predictors and \(S\) is the set of responses to be predicted; each response takes one of the three classes defined in Section 4.2.  
Figure 11: Responses used for algorithm training.  
The calculation procedure to develop the optimal DT, starting from the node \(t\) that contains the set of all observations \(T\), follows the next steps.  
1. Calculate the impurity of node \(t\).  
Gini's diversity index (\(I_{t}\)) measures the impurity of node \(t\) and can be written as:  
\[I_{t}=1-\sum_{i}p^{2}(i) \tag{2}\]  
where the sum is over the classes \(i\) at the node, and \(p(i)\) is the probability of class \(i\) at the node. A pure node (a node with just one class) has a Gini index of zero; otherwise, the Gini index is positive.  
2. Calculate the probability at node \(t\).  
The probability that an observation is in node \(t\) is given as:  
\[P(T)=\sum_{j\in T}w_{j} \tag{3}\]  
where \(w_{j}\) is the weight of observation \(j\) and \(T\) is the set of all observations at node \(t\). If no different weights are used, \(w_{j}=1/n\), where n is the sample size.  
3. Sort predictor elements in ascending order.  
Each ordered element of the predictor \(x_{i}\) can be a candidate to split the node.  
4. Calculate the impurity gain (\(\Delta I\)).  
In order to determine the best way to split node \(t\) using \(x_{i}\), the \(\Delta I\) over all splitting candidates is calculated. For all splitting candidates of \(x_{i}\), the algorithm splits node \(t\) into left (\(t_{L}\)) and right (\(t_{R}\)) nodes, each with its set of observations, \(T_{L}\) and \(T_{R}\), respectively. Then, the \(\Delta I\) is calculated as follows:  
\[\Delta I=P(T)I_{t}-P(T_{L})I_{t_{L}}-P(T_{R})I_{t_{R}} \tag{4}\]  
where \(P(T_{L})\) and \(P(T_{R})\) are the probabilities that an observation is at node \(t_{L}\) and \(t_{R}\), respectively, and \(I_{t_{L}}\) and \(I_{t_{R}}\) are the impurities at the child nodes.  
5. Selection of splitting node.  
The algorithm selects the splitting candidate that produces the largest \(\Delta I\).  
6. Once the splitting node has been selected, the child nodes (\(t_{L}\) and \(t_{R}\)) become parent nodes (node \(t\)). Then, the previous steps are recursively repeated to split the new parent nodes until pure nodes are achieved or the stopping rules are reached.  
Following the previous steps, a very deep DT can be generated with many small leaves, achieving a low training error, but the test error is usually high. To avoid deep growth, stopping rules should be imposed. The stopping rules used were:  
* The maximum number of decision splits was 50.
* The minimum number of branch node observations was 10.
* The minimum number of leaf node observations was one.