# Tightly-Coupled GNSS/Vision Using a Sky-Pointing Camera for Vehicle Navigation in Urban Areas
## 2 Background and Related Works  
The research presented in this paper involves many fields including image processing and computer vision, GNSS, as well as estimation for navigation (Kalman filtering). The image-based NLOS satellites exclusion used in this research heavily relies on the accuracy of the segmentation algorithm [14; 30; 31]. Let us assume that due to a segmentation error, a building is classified as sky. A satellite that is projected in that area will not be excluded, i.e., will be considered as a Line-Of-Sight (LOS) satellite (but actually it is not). Given that even a single GNSS signal affected by multipath can cause a significant position error, this could significantly degrade our GNSS-only solution. Few segmentation algorithms have been evaluated in the literature [12; 14]. In [PERSON] et al. [12], a set of supervised and unsupervised (pixel) classifiers were compared, and it was concluded that the proposed method (i.e., the Geodesic Reconstruction by Dilatation (GRD) coupled to the Fisher clustering algorithm) was more efficient than the other evaluated classifiers. However, the Fisher classifier takes about 1.03 s to process one image. This is computationally heavy compared to the method proposed in [14], where it was found that the Otsu method outperforms the other considered algorithms (Meanwhile, HMRF-EM, graph-cut) for this specific upward-pointing camera application. The Otsu method segments an image with a resolution of \(1288\times 728\) in 0.015 s and provides an accuracy of 94.7%. To increase the accuracy of the segmentation algorithms, [PERSON] et al. and [PERSON] et al. [23; 30] used infrared cameras. However, to date, these types of cameras are not yet available on mass market devices. This leads us to choose a visual spectrum standard camera with a narrow Field Of View (FOV) and suggests a need to improve the segmentation algorithm. A fast and accurate segmentation method is presented in Section 4. Obtaining the position of a robot or vehicle using visual features is one of the most studied problems in the computer vision community. The subject is extensively discussed in [32]. There are several approaches that are usually classified based on how feature points are matched. These include 2D-to-2D matching, 3D-to-3D and 3D-to-2D [32]. An example of monocular visual odometry using 3D-to-2D matching is presented in [33] where a database is used to store a set of point clouds (feature database) registered with intensity information. 2D features are detected in an image captured by the camera, and the match is done with the content of the database. Then, by applying the PnP algorithm, the camera's rotation and translation are obtained. In contrast, in the present paper, we propose a 2D-to-2D approach where 2D feature points are detected on successive images and matching and tracking is used to obtain the position of the camera with respect to the tracked features. Other references such as [20; 34; 35; 36] integrate visual odometry into SLAM applications. This not only provides an advantage of allowing the methods to be used for multiple purposes (positioning, navigation and mapping) but also increases the robustness of these methods. In contrast, in this work, visual odometry based on feature point detection, matching and tracking is used since there is no need in our application (so far) to build a map as the vehicle navigates. GNSS fundamentals are highly documented in the literature [37]. In open-sky, GNSS performs well. However, in harsh environments, it performs poorly. In such areas where some weak GNSS signals are available, high sensitivity receivers using massive numbers of parallel correlators and longer integration times can be used. However, this approach involves higher processing, complexity and power costs [38; 39]. Different GNSS receiver architectures that can improve the receiver performance are also introduced in [40; 41].