# Optimal Feature Set Size in Random Forest Regression
## 2 Feature Set Size (_mtry_)
### Random Forest Algorithm in Regression
### Comparison between the Proposed Search and Typical Grid Search

Figure 8 compares the estimated _mtry_ by the proposed search algorithm and the one by the typical grid search method in terms of prediction accuracy and computational cost. The typical grid search method can be considered as a special version of the proposed search algorithm, where the searches are always performed from \(mtry=1\) to \(p\) with the _Searchsize_ of \(p\). The results demonstrate that, while the RF prediction accuracies using the two estimated sizes are not different because almost all boxes are located near 0, the proposed search algorithm is much faster than the typical grid search method. ## 5 Conclusions

We focused on the influence of _mtry_ on the RF prediction in regression problems because most existing studies about _mtry_ have been primarily conducted focusing on classification problems. Using many datasets, a total of 56 real or artificial datasets, we carried out several experiments and observed that _mtry_ has a significant impact on the RF regression prediction. In addition, the default _mtry_ of \(p/3\) cannot guarantee the highest accuracy in regression. We also found that it is difficult to find a specific rule for estimating the best _mtry_ since the optimal size seems domain dependent. Fortunately, we found a relationship between the optimal size and characteristics of datasets. If the features of the dataset have similar relevance to each other, the optimal size tends to be relatively small, thus smaller _mtry_ values are preferred. In the opposite case, larger _mtry_ values are preferred. Figure 7: Comparison of RF prediction using the estimated size by the proposed search algorithm and the default size of \(p/3\). The \(x\)-axis and \(y\)-axis indicate 56 datasets and the relative RMSEs, respectively. Each box-plot is based on 100 relative RMSEs. We developed a search algorithm for estimating the optimal _mtry_ in RF regression, which is motivated from a method for RF classification [16]. The proposed search algorithm combines a typical grid search method and two unique concepts of \"_SearchDirection_\" and \"_SearchSize_\". The two additional concepts allow our proposed search algorithm to estimate the best _mtry_ more efficiently than the typical grid search method. In the experimental studies, we demonstrated that the estimated size by the proposed algorithm is close to the optimal size and produces better prediction accuracy than the default size. We also confirmed that the proposed algorithm provides similar prediction accuracy to the typical grid search method, but it is more efficient in terms of computational cost. Conceptualization, S.H. and H.K. ; Methodology, S.H. and H.K. ; Formal analysis, S.H. ; Supervision, H.K. ; Manuscript writing, S.H. and H.K. ; and Manuscript editing, S.H. and H.K. All authors have read and agreed to the published version of the manuscript. This work was supported by Basic Science Research program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science, and Technology (No. 2016R1 DIA1B02011696). Not applicable. Data availability StatementData used in this study are available at [[https://github.com/shan-stat/rf_reg](https://github.com/shan-stat/rf_reg)]([https://github.com/shan-stat/rf_reg](https://github.com/shan-stat/rf_reg)), (accessed on 12 March 2021). The authors declare no conflict of interest. ## References

* [PERSON] (2001) [PERSON] Random Forests. _Mach. Learn._**2001**, _45_, 5-32. [CrossRef]
* [PERSON] and [PERSON] (2001) [PERSON]; [PERSON]; [PERSON] _The Elements of Statistical Learning: Data Mining, Inference, and Prediction_; Springer: New York, NY, USA, 2001. * [PERSON] et al. (2013) [PERSON]; [PERSON]; [PERSON]; [PERSON] _An Introduction to Statistical Learning_; Springer: New York, NY, USA, 2013. * [PERSON] et al. (2017) [PERSON]; [PERSON]; [PERSON]; [PERSON] Cervical Cancer Diagnosis based on Random Forest. _Int. J. Perform. Eng._**2017**, 13. [CrossRef]
* [PERSON] et al.