# Machine Learning Methods with Decision Forests for Parkinson's Detection
## 3 Decision Forests
### Random Forest (RF)  
In the year 2001, the Random Forest algorithm has been proposed as a robust decision forest by [PERSON] [26]. Since then, Random Forest attracted more than 63,000 citations in various areas of data science. The power of Random Forest lies with the way of establishing decision trees in the forests with a set of bootstrap samples. For a dataset of \(m\) dimensions, a random variable \(X=(X_{1},X_{2},\ldots\ldots,\ X_{m})^{T}\) is a set of input variables (also known as predictor variables) and \(Y\) is a real values response variable, where RF deduce a prediction function \(f(x)\) for correctly predicting \(Y\). During the prediction process the lowest error represented by a loss function \(l(Y,f(x))\) determines the closeness of \(f(x)\) with \(Y\). RF also has provision to penalizes \(f(x)\) for strong deviation from \(Y\).  
Random Forest has been designed to address both classification and regression problems. However, in line with the aim and objectives of this article the analysis has been restricted to the classification only. Therefore, from the classification point of view, the loss function \(l(Y,f(x))\) is the zero-one loss and is represented as:  
\[l(Y,f(x))=\left\{\begin{array}{rl}0,\ if\ Y=f(x)\\ 1\ otherwise\end{array}\right. \tag{1}\]In other words, the classification function \(f(x)\) for each response values of \(Y\) can be represented as:  
\[f(x)=\begin{array}{c}argmax\\ y\in Y\end{array}P(Y=y\mid X=x) \tag{2}\]  
which is nothing but the Bayes rule. Equation (2) determines the classification approach for single decision tree. However, for \(J\) number of decision trees (\(t=h_{1}(x),h_{1}(x),\ h_{1}(x),\ \ldots\ldots.h_{J}(x)\)), the classification function of Random Forest is:  
\[f(x)=\begin{array}{c}argmax\\ y\in Y\end{array}\sum_{j=1}^{J}I\big{(}y=h_{j}(x)\big{)} \tag{3}\]  
Here, \(I\) is the indicator function. It should be noted that the RF relies on the principle of recursive binary partitioning [58; 59; 60; 61]. The predictor space used to be partitioned on individual variable of \(X\). Concerning the classification, the splitting criteria for each nonterminal node is calculated through the Gini index and mathematically represented as:  
\[Q=\sum_{k\
eq k^{\prime}}^{K}\hat{p}_{k}\hat{p}_{k^{\prime}} \tag{4}\]  
where, \(K\) = Number of classes, \(\hat{p}_{k}\) = Contributed proportion of class \(k\) in the node \(\hat{p}_{k^{\prime}}\) The node \(\hat{p}_{k^{\prime}}\) can be estimated as:  
\[\hat{p}_{k^{\prime}}=\ \frac{1}{n}\sum_{i=1}^{n}I(y_{i}=k) \tag{5}\]  
[PERSON] et al. [58] derived Random Forest both for classification and regression. As the classification is the purpose of this research work, Equations (1)-(5) has been referred in the context of classification.  
During the splitting process, two child nodes are created on the left and right side which is further subjected to splitting like parent node. The process repeated toilling the terminal nodes are achieved (on stopping criteria). Each of the terminal nodes comes up with a predicted result. The predicted result at the terminal nodes for each observation is the maximum repeating class for that observation, which is nothing but the predicted class for concern observation. This principle is applicable to continuous variables. However, for categorical attributes, the final class is decided upon the voting of the prediction result of decision trees.