# Pose Estimation Utilizing a Gated Recurrent Unit Network for Visual Localization
## 2 System Overview  
### Visual Odometry  
The process of the feature-based VO method is shown in Figure 2. Features are detected by feature detectors such as Smallest Univalue Segment Assimilating Nucleus (SUSAN) and Features from Accelerated Segment Test (FAST). The detected feature points are tracked in sequential frames. Local optimization to obtain optimized pose can be executed by sparse bundle adjustment. The VO dealt with in this work is 2D-to-2D motion estimation involved with rotation and translation. The pipeline of the VO in Figure 2 can be described by the pseudo-code in Algorithm 1.  
In 2D-to-2D VO, the transformation \(T_{k}=\left[\begin{array}{cc}R&t\\ 0&1\end{array}\right]\in\mathbb{R}^{4\times 4}\) is estimated in a way that minimizes reprojection error \(\underset{T_{k}}{argmin}\sum\limits_{i}\parallel P_{k}^{i}-\hat{P}_{k-1}^{i} \parallel^{2}\) where \(T_{k}\) represents motion estimation between two images \(I_{k-1}\), \(I_{k}\) obtained with a calibrated camera, and \(\hat{p}_{k-1}^{i}=f\big{(}T_{k},X_{k-1}^{i}\big{)}\) represents the reprojection of corresponding feature point \(X_{k-1}^{i}\) onto image \(I_{k}\) following transformation \(T_{k}\) by reprojection function \(f(\cdot)\). Furthermore, \(p_{k}^{i}\) is the observed feature point in the current frame \(I_{k}\).  
The essential matrix \(E\) defined according to the rotation matrix \(R\in SO(3)\) where \(SO(3)=\left\{R\in\mathbb{R}^{3\times 3}\big{|}RR^{T}=I,\det(R)=1\right\}\) and the translation vector \(t\in\mathbb{R}^{3}\) is given as follows  
\[E=\hat{t}R \tag{1}\]  
where the translation vector is defined as \(t=\left[\begin{array}{cc}t_{x}\\ t_{y}\\ t_{z}\end{array}\right]\) and the skew-symmetric matrix of the translation vector \(t\) is defined as \(\hat{t}=\left[\begin{array}{cc}0&-t_{z}&t_{y}\\ t_{z}&0&-t_{x}\\ -t_{y}&t_{x}&0\end{array}\right]\). Furthermore, the essential matrix \(E\) can be represented by Singular Value Decomposition (SVD) as follows  
\[E=U\Sigma V^{T} \tag{2}\]  
Figure 2: Pipeline (process) of feature-based VO method [2].  
where \(U\) is the unitary matrix, \(\sum\) is the orthogonal matrix, and \(V^{T}\) is the transpose of conjugate matrix. Then, the rotation matrix \(R\) and \(\hat{t}\) can be extracted as follows  
\[R=U\Big{(}\pm W^{-1}\Big{)}V^{T}, \tag{3}\]  
\[\hat{t}=U(\pm W)\Sigma U^{T} \tag{4}\]  
where \(W=\left[\begin{array}{ccc}0&\mp 1&0\\ \pm 1&0&0\\ 0&0&1\end{array}\right]\). Among the possible 4 combinations in Equations (3) and (4), \(R\) and \(\hat{t}\) are selected by judging whether the depth by the corresponding feature points is positive in both camera coordinate systems when the frame is \(k-1\) and \(k\). By getting the rotation matrix \(R\), and the translation vector \(t\), the ego-motion of the camera is estimated using the following equation  
\[R_{traj,t}=RR_{traj,t-1}, \tag{5}\]  
\[t_{traj,t}=t_{traj,t-1}+\lambda\Big{(}R_{traj,t-1}t\Big{)} \tag{6}\]  
where \(\lambda\) is the scale information obtained from ground truth data. \(R_{traj}\) and \(t_{traj}\) are the accumulated rotation and translation calculated by the rotation matrix \(R=\left[\begin{array}{ccc}R_{11}&R_{12}&R_{13}\\ R_{21}&R_{22}&R_{23}\\ R_{31}&R_{32}&R_{33}\end{array}\right]\) and the translation vector \(t=\left[\begin{array}{c}t_{x}\\ t_{y}\\ t_{z}\end{array}\right]\).