# A Robust Localization System Fusion Vision-CNN
## 1 Introduction
[11] proposed to use the simultaneous localization and mapping (SLAM) method to actively record the Oriented FAST and Rotated BRIEF (ORB) feature points, and then the collected landmarks were trained to generate a database of bag-of-words (DboW) through the kmeans++ method. When the robot's laser is blocked, the bag of words is used to perform pose regression; ORBSLAM [12] and VINS-Mono [13] all use the bag-of-words model for loop detection. The use of the bag-of-words model for pose regression usually requires large overlap between the current pose of the robot and the pose in the pre-training set, which means that in order to be able to perform better relocalization the pre-training set must cover large enough environmental regions which will result in larger bag-of-words; but, the retrieval time based on landmarks will be longer if the bag-of-words is too large. Different from extracting artificially designed feature points, the relocalization of the vision-based convolutional neural network can increase the stability of positioning because the convolutional neural network can extract the high-dimensional features of the environment. In 2015, [PERSON] et al. proposed PoseNet [14], the first deep learning network framework for relocalization by using a single image. Inspired by the PoseNet framework, many improved versions including posenetLSTM [15] have emerged for network structure, loss function, increase constraint relations, multi-task modeling, etc. Both algorithms use the trained model to directly estimate the six degree of freedom (6-DoF) pose of the two-dimensional image. This relocalization algorithm is mainly aimed at outdoor or large scenes, and it has high robustness to illumination changes, weather changes, and motion blur. However, the relocation accuracy for small indoor scenes is not high. In summary, it can be seen that combining the relocation capability of vision with the high-precision capability of laser positioning has achieved many results. However, due to the low accuracy of visual relocation, how to use this positioning information to perform rapid and high-precision positioning is still worth studying. This paper proposes an indoor robot positioning system based on an improved posenetLSTM visual neural network and a particle filter that can realize fast and high-precision relocation and dynamic tracking. The methods and the main contributions of this paper are:  
1. Based on posenetLSTM, the laser measurement and the prior map information are used as geometric constraint relationships and they are added to the loss function of the neural network to improve the accuracy of visual relocation;
2. The use of particle filtering to sample the rotation and the translation of particles separately on the pre-calculated, multi-resolution grid map; and the use of the branch and bound method to accelerate the high-precision relocation of the robot;
3. The pose tracking stage is based on the original high-resolution map, using an extended Kalman filter (EKF) to fuse the IMU, the wheel odometer information for initial pose estimation, and correlated particle sampling to obtain the particle with the highest score. The system also introduces a nonlinear optimization method to optimize the pose, which further improves the tracking accuracy and real-time performance of the system;
4. Simulations and physical experiments were carried out in Gazebo and real environments, respectively, which verified that the system could: realize relocation quickly and accurately and track trajectory in real time with high accuracy. The distribution of the paper is as follows. In Section 2, the related works of laser-based, vision-based, and their fusion localization is introduced. Section 3 introduces the system and the method of this article. In Section 4, a simulation environment in Gazebo to perform algorithm simulation analyses is set up. In Section 5, the algorithm verification is carried out on the physical Mir robot platform.