# On the Optimal Size of Candidate Feature Set in Random forest
## 1 Introduction  
Classification is a predictive modeling whose target variable is categorical. There are several classical classification techniques such as logistic regressions, linear discriminant analysis and K-nearest neighbors. Modern popular methods include generalized additive models, decision trees, support vector machines and neural networks. A classification method that predicts the target variable with high accuracy is preferred. It is known that the best classifier for a dataset is domain dependent [1].  
Among many classifiers, a decision tree is a popular choice because the tree output is easy to understand and resembles more closely to human decision-making processes [2]. Moreover, it can be displayed graphically and can handle qualitative predictors without difficulty [2; 3]. However, a decision tree generally does not have the same level of predictive accuracy compared to many modern classification methods. Hence, ensemble methods using decision trees were actively proposed to improve the classification accuracy of a single tree classifier [4].  
Ensemble method is to combine many individual models to achieve better prediction accuracy for classification or regression. Popular ensemble methods for classification are to combine many weak classifiers, usually trees, to reach a final prediction [4]. Of course, ensemble methods can be applied using other classifiers such as neural networks [5].  
The well-known methodologies of classification ensemble are Boosting [6; 7; 8], Bagging [9] and Random forest [10]. Random forest, a variant version of bagging, is a tree-based ensemble method. As in bagging, Random forest uses bootstrapped samples at each stage of ensemble. However, it uses different subset of candidate features at each node when trees are constructed. The candidate feature set is a random subset of \(m\) predictors from the full set of \(p\) predictors. The value \(m\) is one of the parameters that a user can control. Several researches studied the effect of parameter values on the performance of Random forest. [PERSON] et al. [11] investigated the parameter sensitivity of Random forest. [PERSON] et al. [12] described a search method on the selection of parameters. [PERSON] et al. [13] explored the sensitivity of Random forest and stochastic gradient boosting to choices in tuning parameters.  
In this article, we will investigate the effect of candidate feature size \(m\) on the performance of Random forest. Furthermore, we will develop a novel algorithm to estimate a value \(m\) from the training dataset. We will demonstrate that the chosen value \(m\) by the proposed algorithm outperforms the default setting in terms of test accuracy.  
In Section 2, we will describe Random forest and provide a motivation for this research. In addition, the result of an extensive experiment using 58 datasets to figure out whether the size of candidate feature set affects the performance of Random forest will be reported. A novel algorithm for estimating an adequate size of candidate feature set will be proposed in Section 3. In Section 4, we will compare the performance of Random forest; using the estimated size of candidate feature set, and using the default size of candidate feature set. We will provide a conclusion in Section 5.