## 5 Performance Evaluation of Classification Algorithms
### How well can a Trained Model be Transferred between Different Image Types and Image Scenes? Table 1 shows the properties of different VHR and MR image types. Despite differences in spatial resolution, it can be seen that there is a clear superimposition of the spectral resolution between the different VHR and MR image types, respectively. Since the feature vectors derived for different image types indicate that the most important features for classification of \"built-up areas\" are of the spectral and textural domain, it should be possible to transfer a trained classifier from one image type to another, given that spatial and spectral resolution of the image types are in a comparable range. The goal of the following experiments is therefore to evaluate the performance of a classifier when trained on one image type and applied to another. In a first experiment, the transferability between different VHR image types (QB, WV2 and IK) is evaluated. From the manually labeled reference datasets, ten unique training (1400 instances) and testing (350 instances) datasets have been randomly sampled for each image type. The number of training instances has been chosen based on the results of the experiment described in Subsection 5.3. and has been set large enough to not cause a negative influence on the performance of any of the tested classifiers. The size of the feature vector has been adjusted for each classifier individually based on the results of the experiment described in Subsection 5.2. Therefore, each instance has a dimensionality of 21 features for SVM, 37 features for RT, 33 features for KNN and 6 features for NB. The actual experiment iteratively trains a classifier on one image and tests it on the others. Only comparable  
Figure 10: Influence of the training set size on the classification accuracy for MSS. Mean and range of accuracy measures from the ten-fold cross-validation are shown for each iteration. spectral bands between the different image types have been used in order to compose identical feature vectors and to allow a transfer of trained learning machines. Figure 11 shows the performance of the classification algorithms for different training-testing scenarios on VHR image types. The same experiment has also been carried out for MR image types (TM and ETM). Ten unique training (1200 instances) and testing (300 instances) datasets have been randomly sampled for each image type from the reference datasets. Each instance has a dimensionality of 27 features for SVM, 40 features for RT, 18 features for KNN and 26 features for NB. The number of training instances and size of feature vectors have been adjusted based on the results of the experiments described in Subsections 5.2 and 5.3. Figure 12 shows the results for the different classifiers tested on the MR images. Figure 11: Classifier performance for different training-testing scenarios on VHR image types. For the VHR image types, all classifiers show an overall good performance when trained and tested on the same image type. The best overall performance for all classifiers can be observed on the WV2 image (Figure 11, WV2 WV2). On the IK (Figure 11, IKIK) and QB (Figure 11, QBQB) datasets, KNN performs slightly worse than the others. Trained on IK and applied to QB (Figure 11, IKQB), SVM shows a stable performance, RT performs slightly worse, KNN experiences even a slight performance improvement, and NB shows almost random classification behavior. Applying the same trained models to WV2 (Figure 11, IKWV2) leads to a comparable behavior of NB, SVM and RT, with RT showing even improved performance with respect to the image it has been trained on, while a performance decrease can be observed for KNN. When trained on QB and tested on IK (Figure 11, QBIK), SVM and RT show comparable performance with respect to when they are trained and tested on the same image type. Tested on WV2 (Figure 11, QBWV2), both learning machines show a performance increase on both the IK and the WV2 image.