## 2 Materials and Methods
### Step 2: Fagade Image (Orthophoto) Generation
Considering the estimated **S**, the keypoint match lying on the plane **S** was denoted as **X**. In this way, numerous orthophotos could be generated for every plane that existed in the TOC. Users could set the number of planes to be estimated; thus, orthophotos were generated only for a true facade in the TOC. Here, the number of facades to be estimated, \(j\), could either be assigned or was set through an appropriate threshold in the process. Considering such factors, the equation of the plane in vector form is:  
\[\textbf{S}_{j}\textbf{X}_{j}=\textbf{0} \tag{2}\]  
where the facade is expressed as \(\textbf{S}=\left[ \begin{array}{cccc}a&b&c&1\end{array} \right]\), which is the model for MLESAC. A major plane was estimated based on counting the number of **X**\({}_{j}\) satisfying Equation (2). Here, the \(j\) number of major planes were finally chosen from the point cloud. Third, to project each image on the major planes detected, a homography matrix, **H**, was computed from the projection matrix, **P**, and the estimated plane, **S**. The homography matrix was a 3 \(\times\) 3 matrix providing a planar projection transformation between two 2D images. This matrix thus provided a one-to-one mapping between the images and the estimated plane [27]. To do this, the homography relationship needed to transform between a _pixel_ coordinate (frame) and _ortho_ coordinate (**S**) system (expressed in the \(x\)-\(y\) coordinate system). Computing a rotation coefficient, \(r\), enabled homography from the projection matrix. This derivation was described in detail in [45]. Considering each image, \(i\), and each estimated plane, \(j\), this relationship can be represented as:  
\[\textbf{H}_{ij}=\left[ \begin{array}{cccc}\textbf{p}_{i}^{1}&\textbf{p}_{i}^ {2}&r_{i}\textbf{p}_{i}^{3}+\textbf{p}_{i}^{4}\end{array} \right] \tag{3}\]  
where \(\textbf{p}_{i}^{k}\) is the \(k^{\text{th}}\) column of \(\textbf{p}_{i}\). Finally, the set of images was projected onto the estimated planes using the homography matrices, which formed the orthophoto. In Equation (3), each image has \(j\) homography matrices, as we have \(j\) estimated planes. Thus, we needed to determine the proper facade plane (or, the homography matrix) onto which each image had to be projected. This process may result in regions, especially in oblique images, that extend beyond a given facade (which are not in the plane) and would give the appearance of having large distortions when the perspective distortion is removed with respect to the facade plane. To avoid such cases, we used only a subset of the images that included those that were captured relatively parallel and close to the facade plane. We denoted this subset of images used as the _orthoSet_. A threshold value was set to automatically identify these images. The images in the _orthoSet_ were chosen based on the angle, defined here as \(\Theta\), as well as a distance, defined as \(\delta\), between each image plane and the estimated plane. Based on the experimental verification we performed in Section 3, we suggested that these thresholds be set such that that \(\Theta\) be smaller than 15\({}^{\circ}\) and \(\delta\) be less than 5 m. From this point on, we followed the procedure used by [PERSON] et al. to blend the images and generate an orthophoto using gain compensation and multiband blending [29].