# Deep Learning-Based Ground Target Detection and Tracking for Aerial Photography from UAVs
## 3 Detection-Tracking Framework
### Target Matching Strategy Based on SIFT  
The purpose of target matching is to select the preset target from the potential target extracted from the target detection module. In this paper, the invariant feature transform algorithm (SIFT) is used to calculate features and perform feature matching.  
The method to match features is to calculate the Euclidean distance between feature vectors. After the SIFT features of the two images are generated, the feature vectors of image 1 and image 2 are extracted respectively, and the descriptors are calculated. For each feature vector \(f\) of the image 1, find two feature vectors \(k_{1}\) and \(k_{2}\) closest to it in the image 2, and calculate the Euclidean distances \(d_{1}\) and \(d_{2}\) between \(f\) and \(k_{1}\), \(f\) and \(k_{2}\), if \(d_{1}/d_{2}<\delta_{SIFT}\)\(f\) and \(k_{1}\) are a pair of matching points. An example of the invariant feature  
Figure 1: Target detection-tracking framework.  
Figure 2: Deep feature extracted from CNN. (**a**) corresponds to YOLO v3. (**b**) corresponds to Faster R-CNN.  
transform algorithm (SIFT) match for the same target with different scenario is shown in Figure 3.  
Since the conventional SIFT feature calculation is based on grayscale images, color information is lost. But color information in color images is very important. Therefore, it is necessary to improve the SIFT calculation method and use the color information in the image.  
Considering that the RGB histogram does not have invariance for the linear changes of gray scale and color, the three channels of RGB are normalized separately. The normalization process is  
\[\left(\begin{array}{c}R^{\prime}\\ G^{\prime}\\ B^{\prime}\end{array}\right)=\left(\begin{array}{c}\frac{R-\mu_{R}}{\sigma_{ B}}\\ \frac{G^{-1}M_{G}}{\sigma_{G}}\\ \frac{B-\mu_{B}}{\sigma_{B}}\end{array}\right) \tag{1}\]  
where \(\left(R\ G\ B\right)^{T}\) and \(\left(R^{\prime}\ G^{\prime}B^{\prime}\right)^{T}\) are the values of the original and the normalized RGB channels of each pixel, \(\mu_{*}(*=\left\{R,G,B\right\})\) and \(\sigma_{*}(*=\left\{R,G,B\right\})\) are the mean and variance of each color channel for the entire image respectively.  
Each color channel is subtracted from the mean to offset the change in color value, divided by the variance to offset the scale change. The normalized histogram is invariant to the change in color space.  
According to the experiment, when the number of matching feature points accounts for more than 50% of the number of key points, we believe that the target matching on the two images is successful.  
Target tracking is a continuous process. In this paper, the sample real-time update strategy is used to adapt to the shape and size changes of the target during the tracking process. SIFT features of the target region are calculated and recorded during tracking to achieve the goal of updating the target features in real time. To prevent the sample base from expanding, we delete the previously recorded samples from the sample base and record the new samples. This allows the framework to remember the recent features of the target, facilitates target matching and retrieval, and prevents model drift.  
Figure 3: Invariant feature transform algorithm (SIFT) match for the same target with different scenario. (**a**) shows the origin figures. (**b**) shows the SIFT match result for the same target.