# STN-Homography: Direct Estimation of Homography Parameters for Image Pairs
## 2 Dataset  
To compare the homography estimation accuracy of our proposed model with [22; 24], we also used the Microsoft common objects in context 2014 dataset (COCO 2014) [27]. First, all the images were converted to grayscale and were downsampled to a resolution of \(320\times 240\) pixels. To prepare training and test samples, we choose 118,000 images from the trainval set of the COCO 2014 dataset and 10,000 images from the test set of the COCO 2014 dataset. Subsequently, three samples from each image (denoted as image_a) were generated to increase the dataset size. To achieve this, three random rectangles with a size of \(128\times 128\) pixels (excluding the boundary region comprising 32 pixels) were chosen from each image. For each rectangle, a random perturbation was added (within the range of the 32 pixels) to each corner point of the rectangle. This provided us with the target four-point homography values. Target homography was used with the OpenCV library to warp image_a to image_b, where image_b had the same size as image_a. Finally, original corner point coordinates were used within the warped image pair (image_a and image_b) to extract the warped patches patch_a and patch_b. Accordingly, the normalized homography matrix \(\overline{H}_{ba}\) can be calculated with the following equation,  
\[\overline{H}_{ba}=MH_{ba}M^{-1} \tag{1}\]  
where \(H_{ba}\) is the homography matrix calculated from the previously generated four-point homography values and \(M=\begin{bmatrix}\frac{2}{w}&0&-1\\ 0&\frac{2}{n}&-1\\ 0&0&1\end{bmatrix}\), where \(w\) and \(h\) denote the width and height of patch_b (patch_a and patch_b have the same sizes equal to \(128\times 128\) pixels).  
We also used the target homography with the OpenCV library to warp patch_a to get patch_a_t with the same size as patch_a, and patch_a_t was used to calculate the L1 pixelwise photometric loss. The quadruplet data of patch_a, patch_b, \(\overline{H}_{ba}\), and patch_a_t, is our training sample, and are fed as inputs to the network. Please note that the prediction of the network is the normalized \(\overline{H}_{ba}\), and we need to use Equation (1) to transform the prediction result to obtain the nonnormalized homography matrix \(H_{ba}\).  
Given that the homography matrix can be multiplied by an arbitrary nonzero scale factor without altering the projective transformation, only the ratio of the matrix elements is significant, thus leaving \(\overline{H}_{ba}\) eight independent ratios corresponding to eight degrees-of-freedom. Furthermore, we always set the last element of \(\overline{H}_{ba}\) to be equal to 1.0. In the training sample of the quadruplet data, we flattened \(\overline{H}_{ba}=\begin{bmatrix}h_{11}&h_{12}&h_{13}\\ h_{21}&h_{22}&h_{23}\\ h_{31}&h_{32}&1.0\end{bmatrix}\) and took the first eight elements as the training input. Figure 1 shows the value histogram of \(\overline{H}_{ba}\) in the training samples after pixel coordinate normalization, as depicted in Equation (1). From Figure 1 we can clearly observe that after normalization, the magnitude difference and variance of the eight independent elements of \(\overline{H}_{ba}\) is very small, which means that \(\overline{H}_{ba}\) can be easily regressed with CNN.  
Figure 1: Histograms of \(\overline{H}_{ba}\) of the training dataset.