# Tightly-Coupled GNSS/Vision Using a Sky-Pointing Camera for Vehicle Navigation in Urban Areas
## 3 Vehicle Motion Estimation Using an Upward-Facing Camera
### Motion Estimation Process  
Once the correspondences are established between the detected features, the next step is the estimation of the essential matrix **E**. The matrix **E** represents the geometric relations between two  
Figure 1: FAST feature detection illustration. (**a**) \(p\) is the candidate corner; pink-highlighted squares are sixteen pixels used in the feature detection; the dashed line passes through twelve consecutive pixels that are brighter than the pixel at \(p\). (**b**) Same as (a) with the four pixels at Indexes 1, 5, 9 and 13 filled in dark orange used to speed up the threshold test. consecutive images \(I_{k-1}\) and \(I_{k}\). It contains the camera motion parameters up to an unknown scale factor for the translation [32] in the form:  
\[\mathbf{E}_{k}\simeq\mathbf{\hat{t}}_{c_{k}}\mathbf{r}_{c_{k}} \tag{5}\]  
where:  
\(k\) denotes the image frame number;  
\(\mathbf{\hat{t}}_{c_{k}}= \begin{bmatrix}0&-t_{z}&t_{y}\\ t_{z}&0&-t_{x}\\ -t_{y}&t_{x}&0\end{bmatrix} \);  
\(\mathbf{r}_{c_{k}}\) is the rotation matrix;  
and \(\mathbf{t}_{c_{k}}= \begin{bmatrix}t_{x}&t_{y}&t_{z}\end{bmatrix} ^{\mathsf{T}}\), where \(t_{x}\), \(t_{y}\) and \(t_{z}\) are the relative translations following the camera axes. Equation (5) is resolved based on the five-point algorithm solver [53], which provides a certain robustness in the presence of outliers. The camera translation up to a scale, as well as its rotation are thus obtained for two consecutive images and can be used in the subsequent steps in our integration algorithm. One of the most important properties of the motion estimation in this paper is the epipolar constraint that can be described by the essential or the fundamental matrix between two camera positions. The epipolar constraint provides that the projection of a point on the actual image frame \(I_{k}\) must fall in the intersection of an epipolar plane with the image plane [53; 54]. By this process, it is possible to verify if two 2D feature points correspond to the same 3D point. Thus, the epipolar constraint determines the line on which the feature point on the image \(I_{k}\) falls in the image \(I_{k-1}\). This constraint is formulated as:  
\[p^{\mathsf{T}}\mathbf{E}p=0 \tag{6}\]  
The step by step determination of the essential matrix is presented in [53]. Once \(\mathbf{E}\) is estimated from the matched feature points, the relative rotation and translation of the camera are recovered by using the cheirality check [53]. The essential matrix is first decomposed using Singular Value Decomposition (SVD) [54] to obtain four possible poses (rotation/translation), then the verification of the possible pose hypotheses by doing the cheirality check is performed. The cheirality check consists of verifying that the feature point lies in front of the camera (i.e., the triangulated feature point should have positive depth). By this process, the correct rotation \(\mathbf{r}_{c}\) and translation \(\mathbf{t}_{c}\) can be identified and used for the GNSS and vision integration (see Section 5). However, as mentioned previously, using a monocular system, the magnitude of the translation between two consecutive images suffers from scale ambiguity. The integration with GNSS helps to overcome this issue.