# A Lightweight Visual Odometry Based on LK Optical Flow Tracking
## 2 System Description
### Pyramid-Based Sparse Optical Flow Tracking Matching  
The descriptors of the normal frames cannot be reused in the local optimization and loopback detection threads, which causes a serious waste of computing resources. Therefore, the motion model is combined with the pyramid-based LK optical flow method without calculating the descriptors of normal frames in this paper, and the key points in the reference frame (previous frame) image are tracked to establish a robust key point-matching relationship between the adjacent images. In LK optical flow, the image can be viewed as a function of time and position, and the function range is the grayscale value of the pixels in the image. Due to the motion of the camera, the position of the same spatial point is different in adjacent images. To estimate the position of this point in the adjacent image, the assumptions of grayscale invariance and neighborhood motion consistency are introduced into the optical flow method. Combined with Taylor expansion, it can be obtained via  
\[\big{[}I_{x}I_{y}\big{]} \begin{bmatrix}u\\ v\end{bmatrix} =-I_{t} \tag{4}\]  
where, \(u\),\(v\) are the motion velocities of a pixel on the \(x\)-axis and \(y\)-axis, respectively, \(I_{x}\), \(I_{y}\) are the gradient in the \(x\) and \(y\) direction of the image at that point, and \(I_{t}\) is the change of gray value with time. Because two unknown variables \(u\),\(v\) are included, the equation cannot be solved. We can use the hypothesis of motion invariance in the optical flow method, select all pixels within the window \(n\times n\), and establish \(n^{2}\) equations. Then, the equations can be solved via the least squares method. Suppose that the pixel coordinates of a key point \(P\) in the reference image (previous image) \(I_{r}\) are (\(x\),\(y\)), the predicted pixel coordinates of point \(P\) in the current image \(I_{c}\) can be calculated as \((x^{*},y^{*})\). However, due to the movement of the camera, point \(P\) moves to the position \(\big{(}x^{*}+d_{x},y^{*}+d_{y}\big{)}\) in the current image. The key-point-matching relationship between the adjacent images can be established by solving the motion vector \(\big{(}d_{x},d_{y}\big{)}\) of this point. Based on the assumptions of the optical flow method, the optical flow calculation problem can be converted to solve \(\big{(}d_{x},d_{y}\big{)}\) to minimize the change of the pixel gray value in the neighborhood window of the key points in the adjacent image. The grayscale variation function is defined as  
\[\delta\big{(}d_{x},d_{y}\big{)}=\sum_{x=-w_{x}}^{w_{x}}\sum_{y=-w_{y}}^{w_{y}} \big{(}I_{r}(x,y)-I_{c}\big{(}x^{*}+d_{x},y^{*}+d_{y}\big{)}\big{)} \tag{5}\]  
Figure 3: Prediction of key point correspondence. where, \(w_{x}w_{y}\) are the neighborhood window ranges expanded around key point \(P\). The neighborhood window of point \(P\) is \((2w_{x}+1)\times(2w_{y}+1)\). In this paper, both \(w_{x}\) and \(w_{y}\) are set to 2, and there are a total of 25 pixels in the window. Therefore, the objective function for solving the motion vector can be defined as  
\[\left(d_{x},d_{y}\right)=\underset{d_{x},d_{y}}{arg\min}\sum_{i=1}^{m}\left\| I_{t}(x_{i},y_{i})-I_{c}\left(x_{i}^{*}+d_{x},y_{i}^{*}+d_{y}\right)\right\|^{2} \tag{6}\]  
where, \(m\) represents the number of pixels in the neighborhood window of key points, which is set as \(m=25\) in this paper. Through the above derivation, it can be seen that the solution of the objective function is a nonlinear optimization problem. The closer the initial value of the optimization is to the optimal value, the easier the algorithm converges. When the camera movement is large, the assumptions of the optical flow method are not valid, which makes the algorithm easy to fall into minimum and fail. To solve this problem, a pyramid structure similar to that used in the ORB feature extraction is introduced.