# Pattern Discovery in White Etching Crack Experimental Data Using Machine Learning Techniques
## 2 Data and Methods
### Random Forests
Similar to sample size, decreasing ftry results in worse performance by the individual trees, but increasing it can result in overfitting. Much like the case with sample size, the right balance needs to be found where the highest performance by the RF is reached. * Node size: the minimum number of oils in a terminal node of any tree in the RF. Without going into more details, the typically used value for classification problems is 1, which was the value chosen for developing the RF in this investigation since it generally provides good results [32]. When attempted, increasing the node size did not lead to higher accuracy. [PERSON] et al. provide more details on random forest hyperparameters as well as some best practices for tuning RF models [32]. In addition, the pioneering paper by [PERSON] [31] provides more information about random forests. The number of trees in the random forest is also a design decision when developing a random forest. The degree of influence of this hyperparameter is controversial with the research consensus favoring setting it to a computationally feasible large number [32; 33]. In this investigation, increasing the number of trees above 500 trees did not lead to higher accuracy, so the number of trees was set to 500. In order to identify the optimal ftry and sample size values, hyperparameter tuning was performed by trying different combinations of the two hyperparameters and assessing the performance of the resulting random forests. Ultimately, the combination resulting in the random forest with the least classification error was selected. In case of ties, the parameters requiring less computational effort was  
Figure 1: Example decision tree. selected. Since there were only 21 compound IDs present in the data, ftry values could only range from 1 to 21. For the sample size, it was decided to try the range from 1 to 469 with steps of 26, since the training set contained 475 oils. All possible ftry values were tried. The combination resulting in the best performance was ftry = 5 and sample size = 53. This combination led to a 10-fold cross validation accuracy of 98.51%. The R package by [PERSON] et al. was utilized for tuning the hyperparameters of the random forest models in this paper [34]. It is worth noting that the dataset was initially split into a 70%, 30% split before tuning the RF. The division was performed randomly. The tuning was performed using only the 70% set (containing 476 oils). Ten-fold cross validation (CV) was used to estimate the error of each RF model. The benefit of this method is that it allows for testing the machine learning algorithm with the chosen hyperparameters on every available oil in the dataset. Therefore, this was the method of choice for validating the generalizability of all machine learning algorithms in this investigation. For a more detailed explanation of how 10-fold cross validation was applied in this investigation, the reader is referred to publication [28]. The following steps provide an overview of the analysis performed on the oil compositions using random forests:  
1. Splitting the 679 available oil compositions randomly into two smaller datasets: 70% of oils are selected as the training set and 30% are selected as the test set. 2. Hyperparameter tuning: different combinations of sample size and ftry are used to train a random forest model using the training set. Ten-fold cross validation is used to estimate the classification performance of each resulting random forest model. The combination resulting in the top performance is identified as the optimal combination. 3. Developing a tuned RF classifier: the optimal hyperparameter combination is used to develop an RF classifier, trained using the training set. 4. Testing the tuned RF classifier: the developed tuned RF classifier is tested on the test set to verify its accuracy on unseen data. 5. Reaching a more representative estimate of model accuracy: use the optimal hyperparameter pair to perform 10-fold cross validation on all 679 oils.