# A Review of Deep Learning-Based Visual Multi-Object Tracking Algorithms for Autonomous Driving
## 4 TBD Algorithms Based Deep Learning
### TBD Algorithm Based on Deep Learning Object Detection  
Obtaining detection results is the first step in TBD, as already discussed. The results of the object detection have a significant impact on how well the TBD framework tracks objects. There are currently many object detection research findings that are rather advanced [35]. Traditional object detection techniques frequently rely on manually created feature operators to describe images. Examples include SIFT features [36], histogram of gradient directions HOG features [37], etc. However, the results of many related works of literature have demonstrated that compared to traditional algorithms for learning features, Convolutional Neural Networks (CNN) have the strongest feature representation capability and contain the most feature information compared to traditional algorithms learning features.  
The mainstream object detectors are R-FCN [38], SSD [39], Faster R-CNN [40], and YOLO series [41; 42; 43; 44], which have demonstrated good performance for the detection of objects, such as cars, people, and various obstacles. Neural networks have been used more and more in recent years in the recognition and classification of images. The benefit of using neural networks is their capacity to train on relevant data and learn the depth properties of the tracked objects, allowing for extremely precise detection and categorization. Based on the improved single-frame image detection capabilities, MOT tasks exhibit a trend away from the initial emphasis on computationally demanding data association optimization algorithms, such as Joint Probabilistic Data Association (JPDA) [45] and Multi Hypothesis Tracking (MHT) [46], and toward a TBD framework that depends on detection results.  
One of the earliest multi-object tracking algorithms, SORT, uses convolutional neural networks to identify pedestrians. The Faster R-CNN object detection network is used in place of the Aggregate Channel Feature (ACF) detection in the technique, which is based on the conventional Hungarian association approach. The multi-object tracking accuracy was improved on the dataset by an astounding 18.9%, and the algorithm now operates at up to 60 Hz. This algorithm's exceptional performance has garnered a lot of study interest. Aiming at the problem of poor detection quality of Faster R-CNN, Jin et al. [47] extracted multi-scale features and combined three Faster R-CNN models with various backbone structures, which further improved the accuracy and speed of the detection module. They also demonstrated that, after obtaining high-quality detection results, it is possible to simplify the multi-object tracking data association part and obtain comparable multi-object tracking results.  
Some researchers have also introduced deep network detectors for visual multi-object tracking, including SSD and YOLO. To complete multi-object tracking tasks, Zhao et al. [48] used a single-stage SSD as a detector and multi-scale augmentation approach training data. The later YOLO version has managed to strike a compromise between object detection accuracy and detection speed and is frequently employed as a detection module for visual multi-object tracking since the YOLO series has a faster detection speed. A multi-object tracking technique based on YOLO was proposed by Li et al. [49]. The video stream was first subjected to multi-object detection using the YOLO algorithm. To exclude extraneous details from the image, depth feature extraction was performed after acquiring the object's  
Figure 4: The main procedures of TBD framework, which consists of four core components. The four core components contain object localization, feature extraction, data association, and track management. (Source: This image is from the MOT16 dataset [25]).  
size, location, and other details. The region's noisy data reduces feature extraction's computing difficulty and processing time.