## 4 The Proposed Method
### Joint Keypoint Detector  
To accelerate the speed of feature tracking, we propose a Joint Keypoint detector (JKD) that is based on FAST detector, as described in [39]. The JKD consists of two major stages: learning keypoint and superpixel-based keypoint location--each of which, in turn, takes several steps. In the stages of the learning keypoint, the input image is first convoluted. The output of convolution, known as the integral image, is then used as the basis of the scale-space analysis. The responses obtained from the scale-space analysis are utilized to detect the keypoints, \(kp_{i}(x,y)\). In the stage of superpixel-based keypoint location, the SLIC is used to segment the input image to several labels, and then those labels have their center position, \(cp_{i}(x,y)\). Finally, combine the \(kp_{i}(x,y)\) and \(cp_{i}(x,y)\), we can get the final keypoints, \(k_{i}(x,y)\), via non-maximal suppression. The pipeline of the JKD keypoint detector is shown in Figure 3.  
Let \(O(x,y)\) represents a candidate keypoint, and \(N_{O}(x,y)\) represents the \(7\times 7\) neighbors of \(O(x,y)\). Compute the DOG image of \(R_{O}(x,y)\) to get \(DOG_{O}(x,y)\) by Equation (1)  
\[DOG_{O}(x,y)=G(x,y,k\alpha)-G(x,y,\alpha) \tag{1}\]  
Figure 3: Joint keypoint detection. Figure 2: Flowchart of the superpixel-based feature tracking (SPFT) method. where \(k\) is a constant, \(G(x,y,\sigma)=\frac{1}{2\pi\alpha}exp\left(-\frac{x^{2}+y^{2}}{2\sigma^{2}}\right)\) represents Gaussian density function with variance \(\sigma\). Changing the value of \(\sigma\), a set of DOG image is obtained as \(DOG_{set}(x,y)=~{}\{dog_{1},\cdots,dog_{5}\}\) where 5 DOG images is constructed only for saving computation time. For each location on the given DOG image \(I\), the pixel at that position relative to \(O\) can have one of three states:  
\[S_{O\to N}=\left\{ \begin{array}{c}d,I_{o\to n}\leq I_{o}-t\\ s,I_{o}-t<I_{o\to n}<I_{o}+t\\ b,I_{o\to n}>I_{o}+t\end{array} \right. \tag{2}\]  
where, \(S_{O\to N}\) is a correlation between pixel \(o\) and \(n\). \(d\) denotes darker, s denotes similar, and \(b\) denotes brighter. \(t\) is a threshold with a tiny value. For all \(O\in N_{o}\), the \(N_{o}\) can be divided into three subsets \(N_{d}\), \(N_{s}\), and \(N_{b}\) by computing \(S_{O\to N}\). Use ID3 [67] algorithm to choose the first pixel \(n\) to compare with the candidate keypoint \(O(x,y)\), and decide whether \(O(x,y)\) is keypoint or not according to the entropy \(H(O)\) of \(K_{O}\). \[\mathrm{H}(O)=~{}(c+\overline{c})\log_{2}(c+\overline{c})-c\log_{2}c-\overline {c}\log_{2}\overline{c} \tag{3}\]  
where \(c=\left|\left\{p|K_{p}\text{ {is true}}\right\}\right|\) represents the number of keypoints and \(\overline{c}=\left|\left\{p|K_{p}\text{ {is true}}\right\}\right|\) represents the number of non-keypoints. If the selected \(n\) belongs to \(O_{d}\) and produce the max value of \(H(O)\), then \(O_{d}\) can be further divided into the following five categories: \(O_{dd}\), \(O_{\bar{d}\bar{d}^{\prime}}\), \(O_{ds}\), \(O_{\bar{d}b}\), \(O_{\bar{d}\bar{b}}\). For \(O_{s}\), divide it into \(O_{sd}\), \(O_{s\overline{d}^{\prime}}\), \(O_{ss}\), \(O_{gb}\), \(O_{s\overline{b}}\). The process is applied recursively on all five subsets until \(H(O)\) equals to zero. The candidate keypoint can be detected according to the value of \(K_{O}\). \[\mathrm{O}(x,~{}y)=\left\{ \begin{array}{c}true,K_{O}=1\\ false,K_{O}=0\end{array} \right. \tag{4}\]  
where \(O\) is a keypoint if \(K_{o}\) is one. Repeat above process until all input images processed over, then a set of FAST keypoints can be obtained as follows:  
\[K_{fast}=\{k_{i}|i=1,\cdots,n\} \tag{5}\]  
However, the keypoints detected by FAST are often distributed not average, then the resulting point-cloud models are discontinuous. To avoid the in-averaging distributed of FAST keypoints, we use superpixel segmentation approach as a post-process step to find many small regions. Thus, the centers of the regions are selected as the candidate keypoints.