# STN-Homography: Direct Estimation of Homography Parameters for Image Pairs
## 1 Introduction  
Estimating a 2D homography (or projective transformation) from a pair of images is a fundamental task in computer vision. A homography is a mapping between any two images of the same planar surface acquired from different perspectives. They play a vital role in robotics and computer vision applications, such as image stitching [1; 2; 3], simultaneous localization and mapping (SLAM) [4; 5; 6], three-dimensional (3D) camera pose reconstruction [7; 8; 9], and optical flow [10; 11]. The basic approach used to tackle a homography estimation is to use two sets of corresponding points and a direct linear transform (DLT) method. However, finding the corresponding set of points in images is not always an easy task. In this regard, a significant amount of research has been conducted on this topic. Various feature extraction methods, such as the scale-invariant feature transform (SIFT) [12] and oriented fast and rotated brief (ORB) [13] are used to identify the interest points. Accordingly, by employing a matching framework, point correspondences are achieved. Commonly, a random sample consensus (RANSAC) [14] approach is applied on the correspondence set to avoid incorrect associations. Additionally, after an iterative optimization process, the best estimate is chosen. One of the major problems of these methods, such as ORB+RANSAC, is that the achievement of an accurate homography estimation heavily relies on accurate localization and evenly distribution of the detected hand-crafted feature points, which is challenging in low-textured scenes. Convolutional neural networks (CNNs) automate feature extraction and provide much powerful features than conventional approaches. Their superiority has been demonstrated on numerousoccasions in the past in various tasks [15; 16; 17; 18; 19]. Recently, attempts have been expended to solve the problem of matching with CNN. Flownet [20] achieves optical flow estimation by using a parallel convolutional network model to independently extract features from each image. A correlation layer is used to locally match the extracted features against each other and aggregate them with responses. Finally, a refinement stage consisting of deconvolutions is used to map optical flow estimates back to the original image coordinates. Flownet 2.0 [21] uses Flownet models as building blocks to create a hierarchical framework to solve the same problem. In view of the powerful feature extraction and matching capabilities of the CNNs, some studies focused on the solution of homography estimation using CNN and achieved higher accuracies compared with the ORB+RANSAC method. HomographyNet [22] estimated the homography between two images based on the relocation of a set of four points, also known as four-point homography parameterization. This model is based on the VGG architecture [23] with eight convolutional layers, a pooling layer after every two convolutions, two fully connected layers and an L2 loss function that results from the difference between the predicted and the true four-point coordinate values. [PERSON] et al. [24] used the hierarchy of twin convolutional regression networks to estimate the homography between a pair of images, and improved the prediction accuracy of four-point homography compared with that proposed in [22]. [PERSON] et al. [25] proposed an unsupervised learning algorithm that trained a deep CNN to estimate planar homography which was also based on four-point homography parameterization. All these studies chose the four-point homography parameterization because the \(3\times 3\) homography matrix H mixes the rotation, translation, scaling, and shear components of the homography transformation. The rotation and shear components tend to have a much smaller magnitude than the translation component. Even though an error in their values can have a major impact on H, it will have a minor effect on the L2 loss function of the elements of H that is detrimental for training the neural network. The four-point homography parameterization does not suffer from these problems.