# BoxRF: A New Machine Learning Algorithm for Grade Estimation
## 3 Results  
The top 5 performances of the four combinations--(NR = 1, FH = 1), (NR = 1, FH = 2), (NR = 2, FH = 1), and (NR = 2, FH = 2)--are shown in Tables 4-7. The performances shown are R\({}^{2}\) (coefficient of determination), R\({}^{2}\_\)std, root mean square error (RMSE), RMSE_std, box dimensions, minimum number of samples in the box, and the number of trees. Each row of the tables represents median performance across seven iterations with the listed hyperparameters. R\({}^{2}\_\)std is the standard deviation of R\({}^{2}\) across the seven iterations. This \begin{table}
\begin{tabular}{l l} \hline
**Box Dimension** & **Dimension Combinations** \\ \hline Box Length (BL) & [50, 100], [100, 300], [300, 500] \\ Box Width (BW) & [50, 100], [100, 200] \\ Box Height (BH) & [25, 50], [50, 100], [100, 300] \\ \hline \end{tabular} Other hyperparameters of the algorithm and their options are shown in Table 2: Hyperparameter settings.  
\begin{tabular}{l l l} \hline
**Hyperparameter** & **Values** \\ \hline Prediction from first hit only (FH) & 1, 2 \\ Number of times to reuse sample (NR) & 1, 2 \\ Number of trees (T) & 3, 7, 11, 15 \\ Minimum number of samples in a box (MS) & 3, 5 \\ \hline \end{tabular} The dataset was split into training (70%) and testing (30%) subsets. Splitting was carried out in such a way as to ensure the subsets were similar. Table 3 shows the similarity between the train and test subsets at various percentiles.  
\end{table} 
Table 2: Hyperparameter settings. \begin{table}
\begin{tabular}{c c c c} \hline
**Percentile** & **Column** & **Train** & **Test** \\ \hline \multirow{4}{*}{10} & East & 435,431 & 435,430 \\  & North & 5,428,712 & 5,428,719 \\  & Elevation & 1017 & 1012 \\  & Grade & 0.04 & 0.04 \\ \hline \multirow{4}{*}{25} & East & 435,874 & 435,874 \\  & North & 5,429,580 & 5,429,577 \\  & Elevation & 1161 & 1160 \\  & Grade & 0.13 & 0.14 \\ \hline \multirow{4}{*}{50} & East & 436,464 & 436,464 \\  & North & 5,430,148 & 5,430,151 \\  & Elevation & 1303 & 1302 \\  & Grade & 0.32 & 0.32 \\ \hline \multirow{4}{*}{75} & East & 436,940 & 436,940 \\  & North & 5,430,536 & 5,430,533 \\  & Elevation & 1385 & 1385 \\  & Grade & 0.54 & 0.55 \\ \hline \multirow{4}{*}{90} & East & 437,695 & 437,693 \\  & North & 5,430,818 & 5,430,817 \\ \cline{1-1}  & Elevation & 1423 & 1423 \\ \cline{1-1}  & Grade & 0.80 & 0.80 \\ \hline \end{tabular}
\end{table} 
Table 3: Similarity of training and testing subsets. was computed to determine if the performances varied significantly from one random iteration to the next. RMSE_std similarly shows the standard deviation for the RMSE. Several things are observed:  
* The median R\({}^{2}\) performance is 0.75, which is almost identical to the best performance. This means that several combinations can yield a high performance. * The performances are tightly clustered around R\({}^{2}\) of 0.71 to 0.75, with NR = 1 and FH = 2 underperforming the most. This underperformance is not surprising, as NR =1 can result in boxes not populating well, while FH = 2 forces predictions to consider all boxes that contain the location to be predicted. * The standard deviations for both R\({}^{2}\) and RMSE are very low, suggesting that random iterations did not yield very different performances. In other words, for a given set of hyperparameters, BoxRF produces consistent results. * The top 50% of the reported highest performances had 11 trees. Except for one, all had a minimum sample size of three. No top-performing model had 15 trees. \begin{table}
\begin{tabular}{c c c c c c c c} \hline
**R\({}^{2}\)** & **R\({}^{2}\_\)std** & **RMSE** & **RMSE\_std** & **BL** & **BW** & **BH** & **MS** & **T** \\ \hline
0.751 & 0.000 & 0.170 & 0.000 & [50, 100] & [50, 100] & [50, 100] & 3 & 11 \\
0.750 & 0.000 & 0.170 & 0.000 & [100, 300] & [50, 100] & [100, 300] & 3 & 11 \\
0.750 & 0.000 & 0.170 & 0.000 & [100, 300] & [100, 200] & [50, 100] & 3 & 7 \\
0.750 & 0.000 & 0.170 & 0.000 & [300, 500] & [50, 100] & [100, 300] & 3 & 11 \\ \hline \end{tabular}
\end{table}