# On the Optimal Size of Candidate Feature Set in Random forest
## 3 Proposed Algorithm  
The beginning idea of our proposed algorithm is to use the out-of-bag (OOB) error as in [PERSON] et al. [12]. Recall in Algorithm 1 that the bootstrapped sample of the training data is used in each tree construction. The sample which are not selected in the bootstrapping is defined as out-of-bag (OOB) sample. The OOB error is the error rate calculated on the OOB sample [14]. Since the  
Figure 3: Confidence intervals of accuracy differences between the optimal \(m\) and the default \(m\). The difference is significant when the confidence interval does not contain zero. Figure 2: The difference between the optimal size and the default size. OOB sample is not used in the training stage, the OOB error is a good estimate of the true performance of ensemble methods. Similarly, an estimated size of candidate feature set using the OOB error can provide a good estimate for the optimal \(m\). The main characteristics of the proposed algorithm are as follows. Firstly, the algorithm compares the OOB errors starting from the smallest size of candidate feature set, which is one, step by step. The rationale is that there are more datasets in which the optimal \(m\) is smaller than the default \(m\) as shown in Figure 2. Another advantage is that as the value \(m\) gets smaller, training Random forest gets faster. Therefore, a simple and fast Random forest model can be produced. Secondly, since the bootstrapping has a randomness, the algorithm uses a statistical tolerance limit to compare the OOB errors. The definition of the statistical tolerance limit we used is \(e^{*}=\hat{e}-0.5\sqrt{\frac{\hat{e}(1-\hat{e})}{n}}\) where \(\hat{e}\) is the OOB error and \(n\) is the OOB sample size. The second characteristic means that if the OOB error of current size of candidate feature set (\(\hat{e}_{current}\)) is less than the statistical tolerance limit of the previous size of candidate feature set (\(e^{*}_{previous}\)), the algorithm moves on to the current size and continues to compare with the next size. For this reason, the algorithm prefers the smaller size of feature set if several sizes are in the same statistical significance range. Finally, the algorithm uses the concept of \(SearchSize\) to control the number of searches, where \(SearchSize\) is defined as \(ceiling(p/10)\). Note that the number of searches to be compared is different depending on the number of predictors of datasets. For example, consider a dataset with total of \(p=41\) variables. In this case, \(SearchSize=ceiling(41/10)=5\). Let \(e_{1}^{*}\) be the tolerance limit at \(m=1\). Let's now consider \(SearchSize=5\) as the number of contending sizes, i.e. \(m=2\), \(3\), \(4\), \(5\), \(6\). Let the number of features with the smallest OOB error be \(m=4\). Assume that the OOB error at \(m=4\) is less than \(e_{1}^{*}\). Then \(m=4\) becomes the provisional optimal value. Let \(e_{4}^{*}\) be the tolerance limit when \(m=4\). We again consider another five (\(SearchSize=5\)) OOB errors beyond \(m=4\). That is, \(m=5\), \(6\), \(7\), \(8\), \(9\) become new contending candidates. Let the smallest OOB error be \(m=8\) among the candidates. However, if the OOB error at \(m=8\) is not less than \(e_{4}^{*}\), we will still choose \(m=4\) and stop the search. If the OOB error at \(m=8\) is less than \(e_{4}^{*}\), the new tentative optimal value will be \(m=8\). Then we will search for additional five candidates, \(m=9\), \(10\), \(11\), \(12\), \(13\). The algorithm is shown in Algorithm 2. The prediction performance of the proposed algorithm will be evaluated in the next section.