# Deep Learning-Based Automated Background Removal for Structural Exterior Image Stitching
## 2 Structural Exterior Image Stitching through Automated Background Removal
### Mesh-Based Digital Image Stitching for Structural Exterior Map Establishment  
In order to establish the precise structural exterior map, the local warp with a grid mesh is often used. In this study, a mesh-based digital image stitching method, called natural image stitching with the global similarity prior (NISwGSP), was employed [21]. Once the optimal images (\(I_{j}\) and \(I_{j+1}\)) were selected in Step 2, the homography matrix \(\mathbf{H}\)  
Figure 3: Optimal image selection algorithm: \(\mu\) and \(\
u\) are the averaged magnitude of the inlier vectors along the \(x\) and \(y\) directions. \(R\) is the overlap ratio. \(p\) and \(q\) are the iteration variables. \(n\) is the total number of the FOV images. \(I_{p}\) and \(I_{q}\) are the background-removed ROI images.  
(\(\mathbf{H}\in\mathbb{R}^{3\times 3}\)), which is reshaped from \(\hat{h}\), was estimated between each optimal image using the following equation:  
\[\hat{h}\ =\ \underset{h}{\text{argmin}}\ \sum_{i\ =\ 1}^{N}\ ||a_{i}h||^{2}, \tag{3}\]  
\[h\ =\ [h_{1}\ h_{2}\ h_{3}]^{\text{T}},\ a_{i}\ =\ \left[\begin{array}{cccccccc}-x_{i} &-y_{i}&-1&0&0&0&x_{i}x_{i}^{\prime}&x_{i}^{\prime}y_{i}&x_{i}^{\prime}\\ 0&0&0&-x_{i}&-y_{i}&-1&x_{i}y_{i}^{\prime}&y_{i}y_{i}^{\prime}&y_{i}^{\prime} \end{array}\right]\]  
where \([x_{i}\ y_{i}]\) and \([x_{i}^{\prime}\ y_{i}^{\prime}]\) are the matched feature points of \(I_{j}\) and \(I_{j+1}\), respectively. \(N\) is the number of the matched feature points, and \(h_{m}\) is the \(m_{th}\) row components of \(\mathbf{H}\). The elements of \(\mathbf{H}\) was be obtained using direct linear transformation (DLT), and the solution became the least significant right singular vector of \(\{a_{i}\}_{i=\ 1}^{N}\). Given \(\mathbf{H}\), an arbitrary pixel position \(\mathbf{X}_{*}\) on \(I_{j}\) was warped to the pixel position \(\mathbf{X}_{*}\) on \(I_{j+1}\), using location dependent homography matrix \(\mathbf{H}_{*}\)[22].  
\[\widetilde{\mathbf{X}}_{*}^{\prime}\ =\ \mathbf{H}_{*}\widetilde{\mathbf{X}}_{*},\ \mathbf{H}_{*}\ =\ \underset{h}{\text{ argmin}}\ \sum_{i\ =\ 1}^{N}\ ||\omega_{*}^{i}a_{i}h||^{2}, \tag{4}\]  
\[\omega_{*}^{i}\ =\ exp\Bigg{(}-\frac{||\mathbf{X}_{*}-\mathbf{X}_{i}||^{2}}{ \rho^{2}}\Bigg{)}\]  
where \(\widetilde{\mathbf{X}}_{*}\) and \(\widetilde{\mathbf{X}}_{*}^{\prime}\) are the homogeneous coordinates of \(\mathbf{X}_{*}\) and \(\mathbf{X}_{*}^{\prime}\), and \(\omega_{*}^{i}\) is the scalar weight. \(\rho\) is the scale parameter, and \(\mathbf{X}_{i}\) is the position of extracted feature point in \(I_{j}\). Similarly, Equation (4) was solved by using DLT. However, solving Equation (4) with respect to all pixel position \(\mathbf{X}_{*}\) on \(I_{j}\), is not effective in terms of computational cost, because neighboring pixel positions often produce the same \(\mathbf{H}_{*}\). Thus, \(I_{j}\) and \(I_{j+1}\) were divided into the mesh composed of \(C_{x}\ \times\ C_{y}\) cells [23]. For each cell, the center point was chosen as \(\mathbf{X}_{*}\), and all pixels within the cell were warped by the same \(\mathbf{H}_{*}\) as shown in Figure 4. Finally, the structural exterior map was constructed through mesh optimization and image mapping [21].