## 4 FAST-FUSION Approach  
### Omnidirectional Visual Odometry  
As referenced before, the central module of our system is an omnidirectional VO method that is an extension of the state-of-the-art VO approach Libviso2 [11]. Figure 2 is a high-level representation of our approach. As can be observed, it is divided into three main steps:  
1. Application of a camera model that converts 2-D feature pixels in the omnidirectional image in 3-D unit vectors. 2. A Random sample consensus (RANSAC) approach to select the inliers from the entire set of 3-D unit vectors. Figure 1: High-level system architecture. 3. Motion estimation using the epipolar constraint and linear triangulation. To deal with the distortion imposed by an omnidirectional image, a suitable camera model is required. We chose the unified camera model [34] proposed by [PERSON] et al. This model uses the calibration parameters obtained from the Matlab toolbox [35] provided by the authors. Figure 3 represents the two transformations provided by this model. It allows us to convert a 2-D pixel in a 3-D unit vector but also to use the inverse model to transform a world point in a 2-D pixel. The camera model is represented by a polynomial. The polynomial and the inverse polynomial coefficients are obtained during the calibration procedure. This being said, the model is described as follows. **Definition 1**. : _Let \(X=[x~{}y~{}z]^{T}\) be a scene point observed by the omnidirectional camera, \(x^{\prime\prime}=[u^{\prime\prime}~{}v^{\prime\prime}]^{T}\) its projection into the sensor plane, \(x^{\prime}=[u^{\prime}~{}v^{\prime}]^{T}\) in the camera plane and \(v\) be the unit vector that emanates from the viewpoint to the scene point. The camera plane refers to the image plane and it is expressed in pixels. The sensor plane is an hypothetical plane orthogonal to the mirror axis or fish-eye lens, with the origin located at the camera optical center expressed in metric coordinates, i.e., pixels in relation with the image center. The projection of a point in the camera plane into the unit sphere is given by:_  
\[\frac{\left[u^{\prime}~{}v^{\prime}~{}f(u^{\prime},v^{\prime})\right]^{T}}{|| \left[u^{\prime}~{}v^{\prime}~{}f(u^{\prime},v^{\prime})\right]^{T}||},\,f(u^{ \prime},v^{\prime})=a_{0}+a_{1}r^{\prime}+...+a_{N}r^{\prime N} \tag{1}\]  
Figure 3: Camera model overview. Figure 2: Omnidirectional visual odometry scheme. _where \(x^{\prime}=A^{-1}(x^{\prime\prime}-t)\) is the affine transformation that converts points in the sensor plane into the camera plane and \(f(u^{\prime},v^{\prime})\) is the polynomial function that represents the mirror/lens distortion and gives information about the direction of the 3-D ray that emanates from the viewpoint to the 3-D scene point in function of the euclidean distance \(r^{\prime}=\sqrt{u^{\prime 2}+v^{\prime 2}}\) of the image point to its respective center._  
**Definition 2**. : _Let \(X=[x\ y\ z]^{T}\) be a scene point observed by the omnidirectional camera where \(z\) represents its depth and \(h(u^{\prime},v^{\prime})\) the inverse polynomial of \(f(u^{\prime},v^{\prime})\). To project this 3-D point into the image the following transformation is performed_  
\[x^{\prime\prime}= \begin{bmatrix}x\frac{\theta h_{1}+\theta^{2}h_{2}+...+ \theta^{N}h_{N}}{\sqrt{x^{2}+y^{2}}}\\ y\frac{\theta h_{1}+\theta^{2}h_{2}+...+\theta^{N}h_{N}}{\sqrt{x^{2}+y^{2}}} \end{bmatrix} A+ \begin{bmatrix}x_{c}\\ y_{c}\end{bmatrix} ,\ \theta=tan(\frac{z}{\sqrt{x^{2}+y^{2}}}) \tag{2}\]  
_where \(h_{i}\) is the \(i\)th coefficient of the inverse polynomial \(h(u^{\prime},v^{\prime})\), \(A\) is the affine matrix of the camera model and \([x_{c}\ y_{c}]^{T}\) is the image center._  
After having a model \(M\) that converts the 2-D features in 3-D unit vectors dealing with the camera distortion, we propose an adaptation of Libviso2 to estimate the camera motion. To do so, we reuse the matching procedure from the original approach and recreate the epipolar geometry approach to work with omnidirectional cameras.