Vehicle State Estimation for Autonomous Driving
Creators: name: Kaygusuz, Nimet
Publisher: name: University of Surrey
Year: 2023
Subjects: Visual Odometry
Deep Sensor Fusion

description: Robust perception of the environment around a vehicle through onboard sensors is a
description: major hurdle in the quest to realise autonomous driving. To ensure safety, accurate
description: and robust systems are required that can operate within any environment. One of
description: the most fundamental components of a moving agent is ego-motion estimation and
description: Visual Odometry (VO) is a popular choice to achieve it. This is because cameras are a
description: cost-effective solution and are often already installed on modern vehicles. Traditional
description: approaches to VO have been studied for decades and typically work by detecting and
description: matching hand-crafted features. The main disadvantage of this is they are susceptible
description: to failure in poor lighting and low-textured environments, where feature detection and
description: matching tend to fail. In recent years, learning-based VO approaches have become
description: popular and achieved competitive performance due to their ability to extract high-level
description: representations without relying on hand-crafted features. Despite encouraging
description: results, there are several shortcomings that need to be addressed to deploy such motion
description: estimation systems into real-life mobile agents. This thesis proposes novel architectures
description: for learning-based VO and fusion techniques to address these drawbacks.<br>
description: <br>
description: Uncertainty of the estimated motion provides a measure of its credibility and is often
description: crucial for systems that depend upon motion output, such as path planning, vehicle state
description: estimation, sensor fusion etc. Unfortunately, estimating uncertainty for VO is especially
description: challenging as modelling the environmental factors, such as lighting, occlusions and
description: weather, is often not feasible. In the first contribution of this thesis, we propose a robust
description: VO model to estimate 6-DoF poses and uncertainties from video. A learning-based
description: algorithm, MDN-VO, is built which utilises a CNN-RNN hybrid model to learn feature
description: representations from image sequences. A Mixture Density Network (MDN) is then
description: employed to regress the camera motion as a mixture of Gaussians. We then employ
description: a multi-camera approach to provide robustness to individual estimation failures and
description: propose two multi-camera fusion techniques.<br>
description: <br>
description: Any fusion approach must have the ability to balance incoming signals based on
description: their uncertainties. In the second contribution of this thesis, we introduce a naive
description: fusion approach, MC-FUSION, which learns to fuse multi-view motion estimations
description: considering their uncertainties. MC-FUSION outperforms single-camera estimations
description: and the state of the art, validating the benefits of a multi-view VO approach. The
description: approach expects estimations to be synchronised which is problematic for real-world
description: applications. This is because acquiring images from multiple cameras at the same time
description: requires gen-locked or synchronised hardware. Also, sufficient bandwidth is required
description: to capture and process those images simultaneously which in turn has cost implications.
description: Therefore, it is advantageous that a fusion system has the ability to fuse asynchronous
description: signals.<br>
description: <br>
description: In the third contribution of this thesis, we introduce AFT-VO, a novel transformer-based
description: deep fusion framework. Our framework combines predictions from asynchronous
description: multiview cameras and accounts for the time discrepancies of measurements coming from
description: different sources. It can fuse signals from any number of sources and, more importantly,
description: it does not require those sources to be synchronised or to be at the same frequency. We
description: evaluate our model under severe driving scenarios, such as rain and night-time scenes,
description: with promising results. However, the nature of VO is that small errors in the estimation
description: of each frame accumulate over time which results in drift between estimated and real
description: trajectories.<br>
description: <br>
description: In the last contribution of this thesis, we employ a pose graph optimisation technique to
description: reduce the VO drift. Unlike other approaches, we utilise multi-view cameras to detect
description: loops which gather information from different views and provide a more robust loop
description: detection pipeline. In this chapter, we also collect a large-scale multi-view dataset for the
description: evaluation of VO and SLAM systems. Finally, we conclude with thorough experiments
description: demonstrating the drift in VO can be reduced via the optimisation pipeline.<br>