# Comparing Three Machine Learning Techniques for Building Extraction from a Digital Surface Model
## 2 Materials and Methods
### Test Evaluation  
The three implemented methods were fitted on a common dataset and then used to generate predictions on the same hold-out test set to obtain an unbiased estimate of each model's accuracy.  
Given the building label as the positive class and non-building label as the negative class assigned to every single pixel of the raster images, the true positives (TPs), false positives (FPs), true negatives (TNs), and false negatives (FNs) were counted to obtain the pixel-wise metrics for the evaluation of the binary classifiers [78; 79] and a contingency map from a pixel-to-pixel comparison [80; 32].  
The TPs were the pixels correctly identified as building pixels; the FPs were non-building pixels wrongly labeled as belonging to the building class. Similarly, the TNs were non-building pixels correctly classified, whereas the FNs were building pixels wrongly classified as non-building pixels (undetected building pixels). Totaling the number of TPs, TNs, FPs, and FNs added up to the total pixels of the test set.  
Sensitivity, or recall, is the proportion of TPs, pixels that actually are positive (building class). Sensitivity takes values in the range (0, 1); with higher sensitivity, fewer building pixels are undetected (larger footprints).  
\[\text{Sensitivity or Recall}= \frac{\text{TP}}{\text{TP}+\text{FN}}\, \tag{3}\]Specificity is the proportion of TNs, pixels that actually are negative (non-building class). Precision takes values in the range (0, 1); higher specificity leads to fewer pixels mislabeled as building.  
\[\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}\;, \tag{4}\]  
The relationship between sensitivity and specificity was visualized using the receiver operating characteristic (ROC) curve and the area under the curve (AUC) [82] to quantify the performance of each classifier over its range of possible cut-offs (classification thresholds).  
Precision or positive predicted value (PPV) is the proportion of relevant pixels (TP) among all the pixels classified as building pixels (positive) in the test. The PPV varies from 0 to 1, corresponding respectively to the value of the worst and the best classifier.  
\[\text{Precision or PPV}=\frac{\text{TP}}{\text{TP}+\text{FP}}\;, \tag{5}\]  
Similarly, the negative prediction value (NPV) represents the proportion of pixels with accurate non-building test results (TN) among all the pixels identified as non-building pixels (negative class). NPV reference values are 0 for the worst classification and 1 for the best classification possible.  
\[\text{NPV}=\frac{\text{TN}}{\text{TN}+\text{FN}}\;, \tag{6}\]  
The F1 score combines the precision and recall values by taking their harmonic mean. The F1 score takes values in the range (0, 1); higher F1 values correspond to better model performance.  
\[\text{F1 score}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}}\;, \tag{7}\]  
The mean-square error (MSE) evaluates the mean of the quadratic prediction errors; lower values indicate better performance.  
\[\text{MSE}=\frac{1}{\text{N}_{\text{pixels}}}(\text{imgref}\;-\;\text{imgout} )^{2}\;, \tag{8}\]  
The Matthews correlation coefficient (MCC) effectively and reliably measures the quality of binary classifications [83], even if the classes are unbalanced, by taking into account true and false positives and negatives. MCC values range from the worst value \(-1\) to the best value \(+1\).  
\[\text{MCC}=\frac{\text{TP}\;\times\;\text{TN}\;-\text{FP}\;\times\;\text{FN}} {\sqrt{(\text{TP}+\text{FP})(\text{TP}+\text{FN})(\text{TN}+\text{FP})( \text{TN}+\text{FN})}}\;, \tag{9}\]