# Comparing Three Machine Learning Techniques for Building Extraction from a Digital Surface Model
## 3 Results  
We reported pixel-based metrics to evaluate the prediction error for each of the presented models for the building detection task in different urban areas. Figure 6 shows the contingency maps resulting from the pixel-wise results of the binary classification with the SVM, EML, and FCN superimposed on the aerial images of the considered areas: the building pixels correctly detected (TPs) are colored in yellow, FPs are colored in red, and FNs are colored in blue. As seen from examples in Figure 6, ELM tended to underestimate the pixels belonging to the building class, since a fairly high rate of FNs was produced. The resulting segmentation images were not close to reality in the Vaihingen and Toronto patches: in the first case, the smaller footprints were undetected; in the second case, there were \"holes\" in some footprints, having building pixels surrounding non-building areas, which is a characteristic of urban fabrics formed by specific architectural typologies (e.g., courtyard, sieveduan, and pati buildings). However, the SVM seemed in the Vaihingen case more prone to overestimating the number of  
building pixels due to the higher occurrence of FPs. The contingency maps produced by the FCN model showed a better correspondence of the predicted footprint position and size to ground-truth masks in all subsets. For quantitative evaluation of the different classifiers, we reported pixel-wise classification accuracy (Table 1) in terms of sensitivity, specificity, precision, the NPV, F1 scores, the MSE, the MCC, and the AUC using both the aggregate metrics and the metrics split by area. Considering the Vaihingen test set, MCC values of the HOG-SVM and HOG-ELM models (0.657 and 0.513, respectively) were significantly lower than the value of the FCN model (0.833); compared with the ELM classifier, the SVM was more prone to detecting building pixels as it disclosed higher sensitivity (77.3% vs. 41.5%), but the precision was lower (44.6% vs. 60.8%). The metrics relative to non-building pixels were not found to differ markedly. Considering the Potsdam test set, the HOG-SVM and HOG-ELM models showed similarly better outcomes in terms of sensitivity but were slightly outperformed by the FCN model; the difference became greater considering the MCC, a metric that provides a balanced \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & & **Sensitivity** & **Specificity** & **Precision** & **NPV** & **F1 Score** & **MSE** & **MCC** & **AUC** \\ \hline \multirow{4}{*}{HOG-SVM} & Vaihingen & 0.773 & 0.767 & 0.446 & 0.933 & 0.566 & 0.044 & 0.657 & 0.782 \\  & Potsdam & 0.737 & 0.876 & 0.718 & 0.886 & 0.728 & 0.079 & 0.620 & 0.828 \\  & Toronto & 0.887 & 0.871 & 0.742 & 0.949 & 0.808 & 0.033 & 0.826 & 0.914 \\  & Total & 0.817 & 0.854 & 0.687 & 0.923 & 0.746 & 0.058 & 0.725 & 0.867 \\ \hline \multirow{4}{*}{HOG-ELM} & Vaihingen & 0.415 & 0.935 & 0.608 & 0.868 & 0.493 & 0.114 & 0.513 & 0.710 \\  & Potsdam & 0.720 & 0.888 & 0.735 & 0.881 & 0.727 & 0.084 & 0.620 & 0.840 \\  & Toronto & 0.654 & 0.909 & 0.749 & 0.863 & 0.698 & 0.102 & 0.644 & 0.771 \\  & Total & 0.653 & 0.906 & 0.731 & 0.870 & 0.690 & 0.099 & 0.626 & 0.809 \\ \hline \multirow{4}{*}{FCN} & Vaihingen & 0.741 & 0.966 & 0.872 & 0.923 & 0.801 & 0.061 & 0.833 & 0.948 \\  & Potsdam & 0.801 & 0.948 & 0.784 & 0.899 & 0.792 & 0.060 & 0.834 & 0.964 \\ \cline{1-1}  & Toronto & 0.834 & 0.947 & 0.877 & 0.925 & 0.855 & 0.052 & 0.821 & 0.973 \\ \cline{1-1}  & Total & 0.785 & 0.956 & 0.883 & 0.914 & 0.831 & 0.063 & 0.834 & 0.960 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Pixel-wise performance by area for the three models: test evaluation metrics. Figure 6: Contingency maps obtained from a pixel-to-pixel comparison between the masks outputted by the models and the ground truth by urban area: (**a**) Vaihingen test set, (**b**) Potsdam test set, and (**c**) Toronto test set. True positives are shown in yellow, false positives in red, and false negatives (missed building pixels) in blue. measure of the relationship between reality and prediction.