# Mixed-Degree Cubature \(H_{\infty}\) Information Filter-Based Visual-Inertial Odometry
## 2 Model of Visual-Inertial Odometry
### Motion Model of Camera-IMU System
\[\prescript{SC}{\mathbf{p}}{}_{\mathrm{P}_{j}}=\prescript{C}{I}{R}^{\mathrm{T}}\left( \prescript{SC}{\mathbf{p}}{}_{\mathrm{P}_{j}}-\prescript{C}{\mathbf{p}}\right) \tag{8}\]  
where \(\prescript{C}{I}{R}\) and \(\prescript{C}{\mathbf{p}}\) represent the transformation between the ICS and the CCS. We assumed that the two sensors were mounted on a rigid body so that the transformation did not change over time. The camera-IMU transformation and the intrinsic parameters were calibrated in advance and the calibration methods are not included in this paper. The absolute intensity gradient threshold \(g_{\mathrm{thres}}\) was designed according to the intensity characteristic of each patch and evaluated adaptively online. We treated \(g_{\mathrm{thres}}\) as the sum of two parts:  
\[g_{\mathrm{thres}}=\overline{g}+g_{\mathrm{b}} \tag{9}\]  
where \(\overline{g}\) is the average intensity of the image patch and \(g_{\mathrm{b}}\) is a positive value to ensure that the absolute intensity gradient of the candidate is sufficiently high. It is easy to understand that the absolute intensity gradient of the candidate should exceed more pixels in a larger patch, which means that \(g_{\mathrm{thres}}\) is higher with a larger patch and \(g_{\mathrm{b}}\) has a positive correlation with the patch size \(b\). We used a simple linear function to represent the positive correlation and formulated the lower-bound constraint of the absolute intensity gradient as follows:  
\[g\geq\overline{g}+\lambda(b-1),\lambda>0 \tag{10}\]  
where \(\lambda\) is a positive constant. Particularly in the case of \(b=1\), i.e., only one pixel exits in each patch, the above lower bound constraint degrades into \(g\geq g\), which means that all points with valid depth measurements are constructed. Figure 1 shows an example of point sampling on a 640 \(\times\) 480 image. In the example, the above method was able to sample points not only from the edges of the objects, but also from regions with weak intensity variations such as the computer screen and the floor. The points were distributed more densely with a smaller patch. The numbers of sampled points with different patch sizes in this example are shown in Table 1. \begin{table}
\begin{tabular}{c c} \hline \hline \(\mathbf{b}\) & **Number of Sampled Points** \\ \hline
2 & 55752 \\
4 & 14445 \\
8 & 3707 \\
16 & 964 \\
32 & 242 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The number of sampled points with different patch sizes. The above point sampling method adjusts the number of the points by tuning \(b\) and guarantees evenly distributed points. The usage of images approximated the dense visual odometry with a small \(b\). The usage of images approximated the sparse visual odometry with a large \(b\). The keyframe may not sufficient to estimate the egomotion as the camera moves, so we used two judging criteria to change the keyframe:  
1. A new keyframe is needed if there are not enough points in the current view. This is measured via the ratio of points in the current view. 2. Observing the same area from different locations far from each other is likely to lead to large differences in occlusion and illumination. In such cases, new keyframes are needed even if enough points are in the current view. This criterion is quantified with the mean square optical flow ignoring rotational motion \(f=\sqrt{\frac{1}{m}\sum_{i=1}^{m}\Bigl{(}(u_{i,0}-u_{i,t})^{2}+(v_{i,0}-v_{i,t} )^{2}\Bigr{)}}\). \([u_{i,0},v_{i,0}]\) is the pixel coordinate of point \(i\) in the keyframe, and \([u_{i,t},v_{i,t}]\) is the pixel coordinate transformed only by the translational motion. A new keyframe is constructed with the current RGBD images if any of the two criteria are satisfied. After a new keyframe is constructed, the previous keyframe will be replaced by the new one, and the reference coordinate systems are changed in turn. The estimations of \({}^{\mathrm{SI}}\boldsymbol{v}\) and \({}^{\mathrm{SI}}_{\mathrm{E}}\boldsymbol{R}\) are transformed with \({}^{\mathrm{I}}_{\mathrm{SI}}\boldsymbol{\rho}\).