## 2 Materials and Methods
### Unsupervised Homography Estimation Model
#### 2.1.3 Spatial Transformation Layer  
This layer applies the 3 \(\times\) 3 homography matrix, \(H\), obtained in the previous stage, to each pixel coordinate, \(x_{i}\), of the target image, \(I_{A}\), resulting in the corresponding coordinates, \(H(x_{i})\), after the homographic transformation. These transformed coordinates are necessary for the unsupervised method to compute the pixel intensity loss function between the two images. This layer comprises three parts: the normalization inverse computation of homography estimation, the parametric sampling grid generator, and the differential image sampling. Additionally, it must be differentiable to allow the error gradient to backpropagate for network training.  
* Normalized Inverse Computation of Homography Estimation  
The purpose of the spatial transformation layer is to align the target image with the reference image using the homography matrix. Let \(I_{A}\) and \(I_{B}\) be a pair of images with a homography relationship, where \(I_{A}\) is the target image to be transformed and \(I_{B}\) is the reference image. First, the pixel coordinates of input images \(I_{A}\) and \(I_{B}\) are normalized to the range \([-1,\ 1]\). Then, the inverse homography matrix, \(\overset{\sim}{H}_{inv}\), is calculated.  
\[\overset{\sim}{H}_{inv}=M^{-1}\overset{\sim}{H}^{-1}M \tag{7}\]  
where M = \(\begin{bmatrix}\frac{W^{\prime}}{2}&0&\frac{W^{\prime}}{2}\\ 0&\frac{H^{\prime}}{2}&\frac{H^{\prime}}{2}\\ 0&0&1\end{bmatrix}\) and W\({}^{\prime}\) and H\({}^{\prime}\) are the width and height of image I\({}_{\text{B}}\).  
* Parametric Sampling Grid Generator Create a grid, \(G=\{G_{i}\}\), of the same size as \(I_{B}\) for sampling, where each grid element, \(G_{i}=\left(u_{i}^{\prime},v_{i}^{\prime}\right)\), corresponds to the pixel coordinates of image \(I_{B}\). Apply the previously computed normalized inverse homography matrix, \(\widetilde{H}_{inv}\), to the grid, \(G\), to obtain the pixel grid of predicted image \(I_{A}\).  
\[\begin{bmatrix}\mathbf{u}_{i}\\ \mathbf{v}_{i}\\ 1\end{bmatrix}=\widetilde{H}_{inv}(\mathbf{G}_{i})=\widetilde{H}_{inv}\begin{bmatrix} \mathbf{u}_{i}^{\prime}\\ \mathbf{v}_{i}^{\prime}\\ 1\end{bmatrix} \tag{8}\]
* Differential Image Sampling Based on the sampled points, \(\widetilde{H}_{inv}(G_{i})\), obtained in the previous step, generate a transformed image, \(V\) with height \(h^{\prime}\), width \(w^{\prime}\), and \(C\) channels, where \(V(x_{i})=I_{A}(\widetilde{H}(x_{i}))\). The image, \(V\), is defined as: \[V_{i}^{C}=\sum_{m}^{h}\sum_{m}^{w}I_{nm}^{c}k(u_{i}-m;\varnothing_{u})k(v_{i}- n;\varnothing_{v})\ \ \ \forall i\in[1\dots h^{\prime}w^{\prime}],\ \forall c\in[1\dots c]\] (9) where \(h\) and \(w\) are the height and width of input image \(I_{A}\), \(k(\cdot)\) denotes the sampling kernel, \(\varnothing_{u}\) and \(\varnothing_{v}\) are parameters of the sampling kernel, \(I_{nm}^{c}\) is the pixel value of the input image at channel \(c\) and coordinates \((n,\ m)\), and \(V_{i}^{C}\) is the pixel value of the output image at channel \(c\) and coordinates \((u_{i},\ v_{i})\). Due to the potential non-integer values resulting from image transformation, bilinear interpolation is employed to obtain integer values. Additionally, to ensure that the loss function can be backpropagated during network training, it needs to be differentiable. Therefore, the gradient of the image after bilinear interpolation is defined as: \[\frac{\partial V_{i}^{C}}{\partial I_{nm}^{c}}=\sum_{n}^{h}\sum_{m}^{w}max(0,1 -|u_{i}-m|)max(0,1-|v_{i}-n|)\] (10)