# Pattern Discovery in White Etching Crack Experimental Data Using Machine Learning Techniques
## 2 Data and Methods
### Random Forests  
In order to discover the pattern in the available data and correctly classify WEC risk level of a given oil composition using the percentages of its constituting compounds as input, a random forest (RF) model was developed. The available data of 679 oils including their respective constituents' percentages and their risk classification (high or low) were used to train and test the RF models. The random forest [31] model relies on the collective ability of multiple weak classifiers (decision trees) to learn to approximate a function. In this case, the desired function should output the risk level of a given oil composition (high or low) using the percentages of the 21 possible compounds contained in the oil as input variables. Since a random forest is no more than an ensemble of decision trees, Figure 1 illustrates how an example decision tree would classify a given oil based on its constituting compounds. Starting from the root of the tree at the top of the figure, a given oil either follows the left or right path depending on its percentage of \(\Theta\). It then follows the appropriate path depending on its percentage of c6 or c3 to the so-called leaves of the decision tree, illustrated as pie charts in Figure 1. After a number of oil compositions go from the root of the tree to one of the four leaves depending on their constituting compounds, each leave would have a ratio of high and low risk oil compositions as shown in the figure. This process is referred to as training the decision tree. In this example tree, 90% of the oil compositions that made it to the leftmost leaf are low risk oils. If a new oil composition with unknown risk level reaches the leftmost leaf, then the decision tree estimates with 90% probability that the new oil composition is low risk with respect to WECs. A random forest contains a number of such decision trees with different numbers of branches and different splitting criteria at each branch to collectively reach a more accurate classification. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Oil** & **c6** & **c11** & **c16** & **Risk** \\ \hline
195 & 0\% & 98\% & 2\% & Low \\
196 & 0.5\% & 99\% & 0.5\% & High \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Examples of oil compositions. To develop a random forest, some design parameters, so-called hyperparameters, need to be decided by the investigator in a process of tuning the RF to reach optimal performance. Some of the most influential hyperparameters on RF performance are [32]:  
* Sample size: the size of the sample selected from the total number of oils to be the training data for each tree in the random forest. Decreasing this value will most likely result in less accurate predictions by the individual trees. However, increasing this value can also result in overfitting, where the RF achieves significantly higher performance on the training data, but performs poorly on the test data, i.e., new oil compositions with unknown risk levels. * Number of tried features at each split (from here on referred to as ftry): the number of randomly selected candidate variables, in this case compound IDs, for each split in a given decision tree in the RF when growing it. A split in a decision tree is every point when a given oil either follows a right or left path. For example, in Figure 1, the first split is performed according to the percentage of c9 in the oil. If two variables are tried with an ftry = 2, then the variable that best splits high and low risk oil compositions is selected. For example, if c1 and c2 are tried and c1 results in a split with the right side of the split containing only high risk oils and the left side containing only low risk oils, and c2 results in a mixture of high and low risk oils on both sides of the split, then c1 is chosen. This is because the split according to the percentage of c1 in the oils, in this example, results in a purer separation of high and low risk oils compared to c2. If ftry is equal to 3, then three compound IDs are instead evaluated at each split.