# Tightly-Coupled GNSS/Vision Using a Sky-Pointing Camera for Vehicle Navigation in Urban Areas
## 1 Introduction  
Autonomous vehicles rely on navigation sensors such as GNSS receivers, inertial navigation systems (INS), odometers, LiDAR, radar, etc. However, none of these sensors alone is able to provide satisfactory position solutions in terms of accuracy, availability, continuity and reliability all the time and in all environments. For example, INS are immune to interference. However, their performance degrades quickly when updates from other systems such as GNSS are not available. This is generally the typical scenario that is observed in deep downtown areas as presented in [1; 2; 3]. Moreover, integrating INS and GNSS might not be enough to ensure the availability of the position solutions with a certain accuracy and reliability. To avoid these kinds of situations, [PERSON] et al. [4] proposed the integration of GNSS, INS and LiDAR. This is done to alternatively take advantage of the good performance of GNSS in open-sky and the rich LiDAR measurements in occluded outdoor environments for integration with the INS. The same reason drove [5] to combine GNSS, INS, an odometer and an omnidirectional camera. In open-sky scenarios (with no interference or spoofing), relying on GNSS for lane-level navigation is feasible. However, in challenging environments such as deep urban canyons, the GNSS signals are often blocked, reflected or shadowed. This significantly degrades the solution provided by the GNSS receiver. To overcome the disadvantages of GNSS inurban canyons, fusion with map data and other sensors is often employed (e.g., the Google self driving car [6]). However, combining sensors is expensive both in terms of cost and the computational load. Besides sensors fusion, many multipath mitigation methods have been implemented in the literature, including: (i) antenna-based multipath detection/mitigation (antenna design [7], choke rings, controlled reception pattern antenna [8], angle of arrival [9], multiple antennas, dual-polarization antenna); (ii) receiver based (code discriminator design [10], early-late correlator comparison, Doppler domain, carrier smoothing, vector tracking, etc. ); (iii) \(C/N_{0}\)-based multipath detection (\(C/N_{0}\)-based selection and weighting [11], multi-frequency \(C/N_{0}\)-based multipath detection, etc. ); (iv) NLOS detection using a sky-pointing camera [12; 13; 14] and a 3D building model [15; 16; 17]. The research proposed in this paper uses a sky-pointing camera setup similar to what is used in [12; 13; 14]. The proposed method differs from the previous works in two ways. Previous works use a fish-eye camera in order to observe satellites at all elevations. However, low elevation satellites are not very beneficial for the position solution computation since most of the time they are corrupted by the tropospheric [18], ionospheric and multipath errors (causing the receiver to lose the lock on them frequently). Moreover, fish-eye cameras are not yet widely available on mass market products such as cellphones or car-mounted cameras. This paper uses a narrow field of view camera. This is practical since the method can be implemented on mass-market products that already incorporate such cameras. Furthermore, this setup helps to automatically exclude low elevation satellites. The second contribution regarding the multipath mitigation in this work is the segmentation algorithm proposed. A fast and robust segmentation algorithm tailored for navigation (with an upward-facing camera) is introduced. Since almost all modern vehicles are equipped with multiple cameras, they constitute an attractive option for navigation. Some research has focused on using only the camera for navigation purposes. In [19], simultaneous localization and mapping (SLAM) methods are proposed using different camera configurations (monocular versus stereo), as well as different types of cameras (monocular/stereo visual spectral cameras versus RGB-D cameras). The proposed methods were tested on multiple publicly available datasets and showed consistency in terms of accuracy.