DeepVIO: Selfsupervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints This paper presents an selfsupervised deep learning network for monocular visual inertial odometry (named DeepVIO). DeepVIO provides absolute trajectory estimation by directly merging 2D optical flow feature (OFF) and Inertial Measurement Unit (IMU) data. Specifically, it firstly estimates the depth and dense 3D point cloud of each scene by using stereo sequences, and then obtains 3D geometric constraints including 3D optical flow and 6DoF pose as supervisory signals. Note that such 3D optical flow shows robustness and accuracy to dynamic objects and textureless environments. In DeepVIO training, 2D optical flow network is constrained by the projection of its corresponding 3D optical flow, and LSTMstyle IMU preintegration network and the fusion network are learned by minimizing the loss functions from egomotion constraints. Furthermore, we employ an IMU status update scheme to improve IMU pose estimation through updating the additional gyroscope and accelerometer bias. The experimental results on KITTI and EuRoC datasets show that DeepVIO outperforms stateoftheart learning based methods in terms of accuracy and data adaptability. Compared to the traditional methods, DeepVIO reduces the impacts of inaccurate CameraIMU calibrations, unsynchronized and missing data.

PoseConvGRU: A Monocular Approach for Visual Egomotion Estimation by Learning Abstract   Visual egomotion estimation is one of the longstanding problems which estimates the movement of cameras from images. Learning based egomotion estimation methods have seen an increasing attention since its desirable properties of robustness to image noise and camera calibration independence. In this work, we propose a datadriven approach of learning based visual egomotion estimation for a monocular camera. We use an endtoend learning approach in allowing the model to learn a map from input image pairs to the corresponding egomotion, which is parameterized as 6DoF transformation matrix. We introduce a twomodule Longterm Recurrent Convolutional Neural Networks called PoseConvGRU. The featureencoding module encodes the shortterm motion feature in an image pair, while the memorypropagating module captures the longterm motion feature in the consecutive image pairs. The visual memory is implemented with convolutional gated recurrent units, which allows propagating information over time. At each time step, two consecutive RGB images are stacked together to form a 6channel tensor for featureencoding module to learn how to extract motion information and estimate poses. The sequence of output maps is then passed through the memorypropagating module to generate the relative transformation pose of each image pair. In addition, we have designed a series of data augmentation methods to avoid the overfitting problem and improve the performance of the model when facing challengeable scenarios such as highspeed or reverse driving. We evaluate the performance of our proposed approach on the KITTI Visual Odometry benchmark and Malaga 2013 Dataset. The experiments show a competitive performance of the proposed method to the stateoftheart monocular geometric and learning methods and encourage further exploration of learningbased methods for the purpose of estimating camera egomotion even though geometrical methods demonstrate promising results.