Multi-hypothesis Incremental Smoothing and Mapping for Ambiguity-aware Passive and Active SLAM

Simultaneous localization and mapping (SLAM) is the problem of estimating the state of a moving agent with sensor(s) on it while simultaneously reconstructing amap of its surrounding environment, which has been a popular research field due to its wide applications, such as inspection and reconstruction, autonomous transportationand delivery, virtual/augmented/mixed reality (VR/AR/MR), search and rescue, and all kinds of service robotics that involve moving platforms. As many state-of the- art SLAM algorithms can already achieve high accuracy in both state estimation and mapping, improving the robustness of SLAM systems has become a research focus in both academia and the industry in recent years. The most common challenge to robustness is insufficient information. Whenno information is observed by the adopted sensors, e.g.: the view of a camera is fully blocked, and the signal of the Global Positioning System (GPS) is denied in indoor or underwater environments, it is impossible to estimate the state or the map correctly. As a result, most real world SLAM systems adopt multiple sensors in order to achieve better robustness. However, occasionally the observations from all the sensors might still be insufficient to determine a unique solution. For example, when the visual odometry (VO) estimation does not agree with the pose estimation of theGPS, it can be hard to tell which one is correct, or maybe both of them are wrong. In the case that more than one interpretation could be plausible for the same observations,which is known as the ambiguity problem, it is theoretically possible to keep track of all the highly likely interpretations until more information is observed later to disambiguate the ambiguities. However, most of the state-of-the-art SLAM systems only estimate a single solution (and possibly unimodal uncertainties) without considering the impact from ambiguous measurements in the entire pipeline. Therefore, in this thesis, a novel multi-hypothesis back-end optimizer called MH-iSAM2 is introduced to take ambiguities into account and output multi-hypothesis solutions when the ambiguities are temporarily unsolvable. Our novel optimizer allows nonlinear incremental updates in all hypotheses while avoiding redundant computations across different hypotheses, which results in better efficiency than computing each hypothesis individually. Then, an ambiguity-aware planar-inertial SLAM (API-SLAM) system is developed based on MH-iSAM2 to reconstruct dense 3Dmodels of indoor environments in real-time, which provides an example of applying MH-iSAM2 in a multi-hypothesis SLAM (MH-SLAM) framework for better robustness.Finally, an ambiguity-aware active SLAM framework is proposed to make use of the multi-hypothesis state and map estimates from the MH-SLAM system in decisionmaking and path planning, which demonstrates a complete and interactive usage of the multi-hypothesis estimations in a real-world robotic system. The experimental results show that MH-iSAM2 can be applied properly to improve the robustness of a SLAM or active SLAM system, especially for handling the ambiguity problem in real-world tasks.