# Comparing Three Machine Learning Techniques for Building Extraction from a Digital Surface Model
## 2 Materials and Methods
### Models' Implementation and Training  
#### 2.3.1 Shallow ML-Based Building Detection  
A shallow ML-based model that can classify pixels as building or non-building requires discriminative and computable engineered features for image segmentation and pixels classification.  
Among the existing edge- and gradient-based descriptors, the HOG descriptors appeared especially suitable for detecting the construction footprints, as HOG detectors cue mainly on contours, can be computed quickly, and are fairly invariant to geometric transformations and occlusions [23].  
The HOG feature extraction chain computes occurrences of gradient orientation in the detection window, or the region of interest (ROI) across the input, on a regular cell grid and uses overlapping local contrast normalization to enhance the accuracy. For every pixel in the input, the gradient vectors contain information on pixel value changes in x (g\({}_{\mathrm{x}}\)) and y (g\({}_{\mathrm{y}}\)) directions with respect to its four neighbors.  
The attributes of the gradient are its magnitude  
\[\mathrm{M(x,y)}=\sqrt{\mathrm{g}_{\mathrm{x}}{}^{2}{+}\mathrm{g}_{\mathrm{y}} {}^{2}} \tag{1}\]  
and its direction  
\[\mathrm{\theta}=\mathrm{arctan}\Big{(}\mathrm{g}_{\mathrm{y}}/\mathrm{g}_{ \mathrm{x}}\Big{)}. \tag{2}\]  
Gradient information is then pooled into a 1-D histogram of orientations that can be used as input for ML algorithms.  
Thus, the feature extractor encoded the raster inputs into feature vectors to feed a classifier for the object/non-building instance detection, namely ELM and SVM classifiers.  
The first method for building detection under examination used HOG as an input to the SVM [60], a supervised ML algorithm that produces accurate classifications of remotely sensed data [61; 62; 28; 63]. The binary SVM classifier models a data point as a multi-dimensional vector belonging to one of two classes and constructs a hyperplane or set of hyperplanes to separate the classes [64] with the maximal margin (a space containing no observations) and the lowest misclassification [65; 66; 67].  
The SVM classifier was fitted on the aforementioned processed training set to find the best-separating hyperplane (i.e., the decision boundary) using as input the predictor vectors \(\mathrm{X}_{\mathrm{j}}\) along with their class labels \(\mathrm{Y}_{\mathrm{i}}\in\{+1,\ -1\}\). During the training process, sequential minimal optimization [68] solved the constrained optimization problem by breaking it into a series of sub-problems that could be solved analytically. The final model was then saved to perform prediction on the independent test set in the evaluation phase.  
The second method combines the HOG and an ELM [47] classifier instead of the SVM [46]. The ELM trains a shallow feedforward neural network with a single hidden layer, i.e., the feature mapping, which does not need parameter tuning [69; 70; 71]. The main advantages of the ELM method compared to the previous one include better scalability and similar generalization at a faster learning speed [72].  
The ELM classifier was fitted on the same training subset with a single hidden layer of 1000 neurons and a training ratio value of 0.9 to find the combination of nodes, weights, and biases minimizing the error between the actual output of the network (predictions) and the expected one (the ground truth) and obtain a learned model to be used for the test evaluation. The net started with random input weights and calculated the best values using the root-mean-square error to assess the prediction accuracy.