## 5 Performance Evaluation of Classification Algorithms
### What are the Most Important Image Features for Class Separability? A large set of image features used to describe segments in the spectral, textural and geometrical domain has been implemented based on custom C/C++ solutions and OpenCV functions. A large number of features may be beneficial in describing the segments with respect to classes of interest. However, redundant or irrelevant features that provide little information for a specific classification task can have a negative effect on machine learning models and can lead to a decrease in classification performance [15]. Moreover, the size of the feature vector clearly influences the computation time for feature calculation and for training the classifier. During the training stage of a classifier, machine learning algorithms themselves already select features and ignore irrelevant or redundant ones. However, it has been shown that in practice, the classification performance can be improved by a preselection of features before training [15]. In the following, the instance-based feature ranking algorithm ReliefF [40] was utilized to rank sets of features derived for different image types based on how well they distinguish between instances that are near to each other. An instance in this context is composed of a feature vector and an associated class label. ReliefF iteratively selects randomly \(m\) times an instance \(R\) from the data and finds its \(k\) nearest neighbors \(H\) from the same class (nearest hits) and \(k\) nearest neighbors \(M\) from the other classes (nearest misses). The feature values of \(R\) are compared to \(H\) and \(M\) and a quality score is adjusted with every iteration for the features depending on how well they separate the classes. If the feature values of \(R\) and \(H\) are different and therefore the feature separates instances from the same class, the quality score for this feature is decreased since this is not desirable. On the contrary, if the feature values of \(R\) and \(M\) are different and therefore the feature separates instances from different classes, which is desirable, the quality score is increased for this feature [14]. The ReliefF algorithm which is implemented in the open-source data mining software WEKA 3 [41] was used on the QB, TM and MSS reference datasets. The number of nearest neighbors has been set to \(k=\) _10_ as suggested in [42] and the number of iterations has been set to \(m=\)_all instances_. Figure 2 gives an overview of the derived features for the different reference datasets grouped into spectral, textural and geometrical features and ranked by their importance in distinguishing between the different landcover classes. A description of the implemented image features can be found in Tables 2 and 3,where the total number of features per image type is dependent on the number of spectral bands. Therefore, 72 spectral, textural and geometrical features were calculated for 1750 reference segments of the QB image. For the 1500 reference segments of the Landsat TM image, 109 features were calculated, and for the 900 reference segments of Landsat MSS, 73 features were derived. The feature ranking reveals that for all image types, clearly the spectral features are the most important, followed by textural features, whereas geometrical features show the least importance for class separability (Figure 2). Expressed in terms of an average quality score per feature class, spectral  
Figure 2: Features for the Landsat Thematic Mapper (TM), Landsat Multi-Spectral Scanner (MSS) and Quickbird (QB) reference dataset grouped into spectral, textural and geometrical features and ranked by their importance for distinguishing built-up from not built-up areas. For each image type, the ten most important features are highlighted.