# Tightly-Coupled GNSS/Vision Using a Sky-Pointing Camera for Vehicle Navigation in Urban Areas
## 4 Camera-Based Non-Line-Of-Sight Effect Mitigation
### Satellite Projection and Rejection  
The method used in this paper consists of projecting all the tracked satellites onto the successive images and classifying them into LOS satellites (in the sky area) and NLOS satellites (in non-sky area). To project the satellites on an image, we need:  
* The distance from the centre of the image in pixels (\(d_{\rm{c_{pix}}}\)): this corresponds to the elevation angle of the satellite (\(\varepsilon_{sat}\)),
* The azimuth within an image: for this, the heading of the platform is required.  
The geometry shown in Figure 8 is used to determine the distance (\(d_{\rm{c_{pix}}}\), in pixels) from the centre of a calibrated image to a satellite's projection onto the image plane.  
To determine \(d_{\rm{c_{pix}}}\) corresponding to \(\varepsilon_{sat}\), we defined a mapping function, which is a function of the focal length \(\mathbf{f}\) and the angle \(\Psi\) (see Figure 8). Assuming that the optical centre is constantly zenith pointing, the angle \(\Psi\) is defined as:  
\[\Psi=\frac{\pi}{2}-\varepsilon_{sat} \tag{9}\]  
Having a calibrated camera and from Figure 8 and Equation (9), we obtain:  
\[d_{\rm{c_{pix}}}=\rm{f_{pix}}.\tan(\frac{\pi}{2}-\varepsilon_{sat}) \tag{10}\]  
where \(\rm{f_{pix}}\) is the focal length in pixels.  
Thus, given the centre of the image coordinates (\(x_{c}\),\(y_{c}\)), the platform heading, as well as the satellite azimuth, the coordinates of the projection of the satellite on the image (in pixel) are given by Equations (11a) and (11b) below:  
\[x_{Sat_{\rm{Img}}} =x_{c}+d_{\rm{c_{pix}}}.\cos(\alpha_{\rm{platform}}+\alpha_{sat}) \tag{11a}\] \[y_{Sat_{\rm{Img}}} =y_{c}-d_{\rm{c_{pix}}}.\sin(\alpha_{\rm{platform}}+\alpha_{sat}) \tag{11b}\]  
where:  
\((x_{Sat_{\rm{Img}}},y_{Sat_{\rm{Img}}})\) are the projected satellite coordinates on the image plane;  
Figure 7: Flood-fill-based image segmentation steps. Following the arrow direction we have the following: first box: original image; second box: filtered RGB image; third box: luminance channel extracted from the RGB to Luv colour space conversion; fourth box: detected edges; fifth box: image segmentation result.  
\(\alpha_{\text{platform}}\) is the heading of the platform;  
\(\alpha_{sat}\) is the satellite azimuth.  
An example of the satellite projection result at a given vehicle location is depicted in Figure 9. The non-obstructed satellites, also called LOS satellites, are those that lie in the sky area represented with the green circles. In Figure 9, they are PRN6, PRN17, PRN19 and PRN28. The obstructed satellite (NLOS satellite, red circle) in this case is PRN9.  
Thus, at this specific epoch, PRN9 will be rejected, and PRNs 6, 17, 19 and 28 will be used for further processing.  
Figure 8: Modelling of the satellite projection on the image.  
Figure 9: Example of satellites’ classification based on their obstruction status. Left: original image; right: satellites projection and classification. Obstructed, NLOS (PRN9), and non-obstructed, LOS (PRN6, PRN17, PRN19, PRN28), satellites are represented in red and green, respectively.