## 4 Methodology
### Land Cover Classification Framework
#### 4.2.3 RF with Bayesian Optimization  
An appropriate classifier can build the implicit relationship between feature information (e.g., band information) and target information (e.g., five classes in this study) by supervised learning from training datasets. Given the trained classification model, prediction can be made on unseen data to generate the corresponding class labels. A number of classifiers have been used in the literature for supervised learning, such as SVM, RF, decision trees, nearest neighbor, and neural network. It has been shown that RF possesses more advantages in avoiding over-fitting while with a relatively low computation load [28]. RF is an ensemble learning based classification approach with a large number of decision trees constructed in the training process, where the final output integrates the outcome class of individual decision trees [15; 28]. Such a method is able to avoid over-fitting and at the same time is much more robust than a single decision tree. It is also shown in the existing studies that the RF method is able to achieve a high accuracy, a  
Figure 4: Band reflectance for five different classes including building, trees, roads, water, and crops. Figure 3: Labeled map for training data generation. good robustness, and less computation load [29; 30]. However, some hyperparameters in this method are necessary to be tuned according to the tasks of interest so that a better classification performance can be fully realized. According to [17], Bayesian optimization can be adopted to automatically tune the hyperparameters, where the details are summarized in Algorithm 1. Due to the lack of space, the basic RF algorithm is referred to the existing studies [31]. To demonstrate the advantages of the proposed RF with Bayesian hyperparameter optimization, its performance is compared against the conventional SVM and the RF with default parameters. 1. **Initial settings**: RF is composed of a large number of decision trees. Therefore, the number of decision trees need to be first determined (e.g., 150) because a large number of individual trees can improve the discrimination ability. At the same time, certain stopping rule needs (e.g., here the rule is maximum evaluation time) can also be defined to end the optimization iteration. 2. **Objective function**: Set the hyperparameters \(\mathcal{H}\in\Omega\) that should be tuned such as MinLeafSize (mls), NumPredictorsToSample (npts). The objective function in Bayesian optimization is then defined as the mean of the out-of-bag error ('oobErr') to avoid overfitting, so that the optimization problem is given by \[\mathcal{H}_{opt}=\operatorname*{arg\,min}_{\mathcal{H}\in\Omega}oobErr( \mathcal{H}).\] (1)
3. **Bayesian optimization**: The Bayesian optimization method aims to automatically and optimally suggest new parameters by fitting a Gaussian process model \(G\) for the existing data points \(\{\mathcal{H}_{i},oobErr(\mathcal{H}_{i})\}\) and find a new point after updating to minimize the objective function based on the posterior distribution function \(G\). 4. **Optimized hyperparameters**: The optimization process will end when the stopping criterion in Step (a) is satisfied and these optimized hyperparameters will be put into the process classifier. 5. **Optimized RF classifier**: By using the optimized hyperparameters, the optimized random forest can be used for performance assessment (e.g., confusion matrix calculation) and land cover classification applications (e.g., to the whole image of interest). All algorithms (SVM, conventional RF, and RF with a Bayesian optimization method) involved in this study are implemented in Matlab of version 2017b. For the proposed method, there are a total of two hyperparameters being tuned including minimum leaf size (\(mls\)) and the number of predictors to sample (\(npts\)), where \(mls\) is to control the depth of the trees and \(npts\) determines the amount of predictors to sample at each node when growing the trees.