# Optimal Feature Set Size in Random Forest Regression
## 2 Feature Set Size (_mtry_)
### Random Forest Algorithm in Regression
The large fluctuation of RMSE means that the performance of the RF is severely affected by _mtry_ values. Moreover, the results clearly show that the optimal _mtry_ that achieves the smallest RMSE differs from the common choices in all four datasets. Second, in Figure 2, we explore a relative distance, which is defined as (optimal _mtry_--default \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Name** & **Observation** & **Feature** & **Source** \\ \hline cou & 3114 & 13 & [30] \\ cox & 462 & 255 & R Library _anet_ (Cox-2 Activity) \\ cps & 534 & 10 & [31] \\ dee & 696 & 13 & [31] \\ dia & 366 & 15 & [30] \\ eec & 1296 & 8 & UCI (Energy efficiency with Cooling) \\ eeh & 1296 & 8 & UCI (Energy efficiency with Heating) \\ ele & 10,000 & 12 & UCI (Electrical Grid Stability Simulated) \\ fac & 500 & 18 & UCI (Facebook metrics) \\ fam & 1318 & 22 & [32] \\ fat & 252 & 17 & [33] \\ fir & 517 & 8 & UCI (Forest Fires) \\ fis & 6806 & 24 & [34] \\ hat & 100 & 13 & [35] \\ hit & 322 & 18 & R Library _ISLR_ (Hitters) \\ ins & 5822 & 85 & UCI (Insurance Company Benchmark) \\ ist & 538 & 7 & UCI (ISTANBUL STOCK EXCHANGE) \\ lab & 2953 & 18 & [36] \\ lah & 200 & 16 & [37] \\ mdp & 1495 & 8 & R Library _COUNT_ (medpar) \\ med & 4406 & 21 & [38] \\ ozo & 366 & 12 & R Library _mlbench_ (Ozone) \\ pks & 5875 & 16 & UCI (Parkins Telemonitoring with total-UPDRS) \\ rat & 144 & 10 & [38] \\ rvm & 3874 & 14 & R Library _COUNT_ (rwm1984) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: \(Cont.\) \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Name** & **Observation** & **Feature** & **Source** \\ \hline sce & 113 & 10 & [39] \\ sms & 141 & 10 & [39] \\ sol & 1066 & 10 & UCI (Solar Flare with C-class) \\ spm & 395 & 30 & UCI (Student Performance with Math) \\ spp & 649 & 30 & UCI (Student Performance with Portuguese) \\ tre & 100 & 8 & [40] \\ tri & 186 & 60 & [41] \\ wag & 3380 & 17 & [42] \\ wqr & 1599 & 11 & UCI (Wine Quality with red) \\ wqw & 4898 & 11 & UCI (Wine Quality with white) \\ yac & 308 & 6 & UCI (Yacht Hydrodynamics) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The description of 56 datasets for regression problems. _mtry_)/_p_, to measure the difference between the optimal size and the default size. There are only four datasets where the default size is matched to the optimal size, and, in most cases, the default size fails to offer the best RF prediction accuracy. In addition, there are more datasets where the optimal size is larger than the default size than there are datasets in the opposite case. Last, Figure 3 compares the optimal size and the default size by using a relative RMSE, defined as \(log(\frac{\text{RMSE}}{\text{RMSE with default }mtry})\). The relative RMSE greater than 0 means that the optimal size achieves more accurate prediction than the default size. The RF model with the optimal size is obviously more accurate than that with the default size because almost all boxes are located above 0. The performance difference is huge when the optimal size is larger than the default size, but the difference is quite small when the optimal size is

Figure 1: Comparison of RMSE depending on different feature set sizes (_mtry_) on four representative datasets (fam, fis, hit, and med). Each point is the averaged RMSE over 100 replicates. Figure 2: Comparison between the optimal _mtry_ and the default _mtry_ (_p_/3). The _x_-axis and _y_-axis indicate 56 datasets and the relative distance, respectively. smaller than the default size. In summary, through the experimental results, we observed that: (1) the _mtry_ value has an important role in the RF prediction performance; (2) the default _mtry_ value cannot guarantee the best RF performance in regression problems; (3) the optimal _mtry_ value differed dataset to dataset; and (4) there is no clear pattern for the optimal size, thus it is difficult to guess the best _mtry_ value in advance. ### Rule for Optimal Feature Set Size

In this section, we further try to find a rule associated with the optimal _mtry_ using characteristics of datasets.