# A New Real-Time Detection and Tracking Method in Videos for Small Target Traffic Signs
## 2 Materials and Methods
### The Improvement of Detection Method with Multi-Object Tracking  
A challenge for video detection when the bounding boxes become tiny due to frequent flickering and drifting results in missed and false detections. Significant external factors such as camera movement, occlusion, blur, etc. are also inherent issues that may occur with traffic signs. To address this problem, multi-object tracking of Deep-Sort, which has been verified validity by [51] in 2017, is adopted after YOLOv3. It is capable of improving the stability of the continuous frame target trajectory and eliminating the long-term mismatches of detection targets with recursive Kalman filtering and frame-by-frame data association. Figure 5 shows the structure for the tracking-by-detection used in this paper and the specific process of the method is described with pseudo-code in the Algorithm 1. There are mainly two parts of Deep-Sort: appearance descriptor and data association. The matching measure is divided into Mahalanobis distance represents the motion matching metric, and the smallest cosine distance represents the appearance matching metric. When performing detection on a video, the bounding box of a target in the t frame can be obtained by the object detection algorithm or modeling and predicting the target's bounding box in previous t frames by Kalman filtering. Finally, Kalman filtering can combine two bounding boxes to get the best estimate of the target in the current frame. Each target has its trajectory in the video; thus, the 8-dimensional space is introduced as follows to describe the state of the target trajectory at a specific time. \[(u,v,r,h,x^{*},y^{*},r^{*},h^{*}) \tag{5}\]  
where \(u,v\) represent center coordinates of the bounding box, \(r\) and \(h\) represent ratio and height, \(x^{*},y^{*},r^{*},h^{*}\) respectively represent target speed information. The Mahalanobis distance is defined as follows:  
\[d^{(1)}(i,j)=(d_{j}-y_{i})^{T}S_{i}^{-1}(d_{j}-y_{i}) \tag{6}\]  
In (6), \(S_{i}\) represent the covariance matrix of the observation space at the current moment predicted by the Kalman filtering, represent the prediction of the trajectory in the current frame to the next frame, \(d_{j}\) represent the observed variable of the \(j\)-th target. It reflects the matching degree between \(j\)-th target an \(i\)-th trajectory. If the Mahalanobis  
Figure 5: Improved structure for detection-and-tracking. distance between the bounding box of the target in the current frame and the previous trajectory prediction observation is less than 9.4877, expressed as \(t^{(1)}\), the corresponding target and trajectory are related, expressed as \(b_{i,j}^{(1)}\), as follows:  
\[b_{i,j}^{(1)}=1[d^{(1)}(i,j)\leq t^{(1)}] \tag{7}\]  
While the Mahalanobis distance is capable of ensuring the continuity of the bounding box during driving, the motion of the onboard camera will cause a large number of mismatches in the Mahalanobis distance. Therefore, the smallest cosine distance is introduced as follows to match different target trajectories:  
\[d^{(2)}(i,j)=\min\{1-r_{j}^{T}r_{k}^{(i)}|r_{k}^{(i)}\in R_{i}\} \tag{8}\]  
where \(|r_{j}|=1\), which represents the feature description factor of \(j\)-th target, \(r_{j}\) represents the appearance of the tracking target in different frames, \(R_{i}\) represents the entire trajectory of tracking, \(r_{k}^{i}\) represents the description factors for all trajectories. If the two trajectories are related if the smallest cosine distance between two trajectories is less than \(b(2)\), \(i\),\(j\) which can be obtained by training:  
\[b_{i,j}^{(2)}=1[d^{(2)}(i,j)\leq t^{(2)}] \tag{9}\]  
And combine both two metrics:  
\[c_{i,j}=\lambda d^{(i)}(i,j)+(1-\lambda)d^{(2)}(i,j) \tag{10}\]  
where \(\lambda\) represents the weight, the Mahalanobis distance measure is particularly valid for short-term prediction and matching. In contrast, the smallest cosine distance measure is more effective for long-term mismatched trajectories.