Abstract. Estimating fundamental matrices is a classic problem in computer vision. Traditional methods rely heavily on the correctness of estimated key-point correspondences, which can be noisy and unreliable. As a result, it is difficult for these methods to handle image pairs with large occlusion or significantly different camera poses. In this paper, we propose novel neural network architectures to estimate fundamental matrices in an end-to-end manner without relying on point correspondences. New modules and layers are introduced in order to preserve mathematical properties of the fundamental matrix as a homogeneous rank-2 matrix with seven degrees of freedom. We analyze performance of the proposed models using various metrics on the KITTI dataset, and show that they achieve competitive performance with traditional methods without the need for extracting correspondences. Keywords: Fundamental Matrix · Epipolar Geometry · Deep Learning · Stereo. The Fundamental matrix (F-matrix) contains rich information relating two stereo images. The ability to estimate fundamental matrices is essential for many computer vision applications such as camera calibration and localization, image rectification, depth estimation and 3D reconstruction. The current approach to this problem is based on detecting and matching local feature points, and using the obtained correspondences to compute the fundamental matrix by solving an optimization problem about the epipolar constraints [27, 16] . The performance of such methods is highly dependent on the accuracy of the local feature matches, which are based on algorithms such as SIFT [28] . However, these methods are not always reliable, especially when there is occlusion, large translation or rotation between images of the scene. In this paper, we propose end-to-end trainable convolutional neural networks for F-matrix estimation that do not rely on key-point correspondences. The main challenge of directly regressing the entries of the F-matrix is to preserve its mathematical properties as a homogeneous rank-2 matrix with seven degrees of freedom. We propose a reconstruction module and a normalization layer (Sec. 2.2) to address this challenge. We demonstrate that by using these layers, we can accurately estimate the fundamental matrix, while a simple regression approach does not yield good results. Our detailed * Indicates equal contribution
Another work, also focused on estimating the fundamental matrix, introduces a CNN model which explicitly takes into account the mathematical properties of the fundamental matrix as a homogeneous rank-2 matrix with seven degrees of freedom REF .
Deep Fundamental Matrix Estimation without Correspondences
ArXiv
ArXiv
Computer Science Mathematics