# Deep Homography Estimation and Its Application to Wall Maps of Wall-Climbing Robots
## 1 Introduction  
In the past several decades, considerable research has been dedicated to the development of mobile systems that can traverse vertical surfaces. This research on wall-climbing robots has predominantly been motivated by increases in safety legislation and economic efficiency improvements over existing solutions. Wall-climbing robots can replace humans in the execution of highly dangerous tasks [1; 2; 3; 4; 5; 6; 7], such as evaluating and diagnosing storage tanks in petrochemical facilities and nuclear power plants, inspecting and cleaning high-rise buildings, performing the maintenance and welding of ship hulls and application in open-pit mine walls. It can be seen that wall-climbing robots are widely used in a variety of engineering applications. Accurate location detection is important in the controlling of wall-climbing robots, and visual methods are commonly used in the positioning of wall-climbing robots [8; 9; 10]. In these methods, the wall-climbing robot is first detected visually, and the pixel position of the robot must then be converted for use in a spatial coordinate system of the wall according to the calibrated externalparameters of the camera. To convert the pixel position to a spatial coordinate, the rotation matrix \(\mathbf{R}\) and translation matrix \(\mathbf{t}\) of the coordinate system of the camera are calibrated against a coordinate system of the wall. However, when the camera is far away from the wall (e.g., 20 m), it is difficult to calibrate the external parameters of the camera. A simple and effective alternative is to prepare a wall map (using the front view of the wall), and to then locate and control the climbing robot using the pixel coordinate of the wall map, as shown in Figure 1. To locate and control wall-climbing robots using a wall map, the pixel coordinate of the wall-climbing robot, as detected in image_a (Figure 1, left), can be translated as a pixel coordinate in wall_map (Figure 1, right) using a homography matrix (or projective transformation matrix) \(H\), which eliminates the need to calibrate the internal and external parameters of the camera. Obstacles can also be labeled on the wall to assist in the design of a safe walking route for the wall-climbing robot. The estimation accuracy of \(H\) between the camera image and the wall map directly impacts the pixel positioning accuracy of the wall-climbing robot in the wall map. This paper focuses on the homography estimation between the camera image and the wall map. Direct Linear Transformation (DLT) based on two sets of corresponding points including at least four pairs of corresponding points is the basic method of homography estimation, but finding corresponding points from a given pair of images is difficult. The usual practice is to use artificially designed features (e.g., Scale-Invariant Feature Transform (SIFT) [11] and Oriented FAST and Rotated BRIEF (ORB) [12]) to look for points of interest in the image first, and then to match the points of interest of the image pairs using a matching method to get two sets of corresponding points. Finally, Random Sample Consensus (RANSAC) [13] is used to process any incorrect matching point pairs that may exist in the point set. The best homography estimation is then selected after iterative optimization. The requirement of artificially designed features and an exhaustive matching step are major issues with methods such as ORB+RANSAC. The deep learning model automatically extracts features and provides more powerful features than traditional methods. The superiority of feature extraction using deep learning models has been validated in a variety of tasks [14; 15; 16; 17]. Recently, attempts have been made to solve the problem of matching using a Convolutional Neural Network (CNN). As an example, Flownet [18] solves the optical flow estimation task by using a parallel convolutional network model and a correlation layer. Flownet 2.0 [19] proposes a hierarchical model that is stacked by Flownet and achieves higher precision than Flownet.