# A Review of Deep Learning-Based Visual Multi-Object Tracking Algorithms for Autonomous Driving
## 5 JDT Algorithms Based Deep Learning
### Fusion Detection and Tracking Module  
The front-end object detection network significantly affects how well multi-object tracking functions, as can be shown from the analysis above. In order to further increase algorithm simplicity, weight sharing between the two phases of detection and tracking is realized, and the high-performance object detection network is improved to support multiple objects tracking tasks. Recently, research on object tracking algorithms has turned its attention to fusion of multi-object tracking algorithm modules.  
Feichtenhofer et al. [11] initially proposed to include the object detection network in the tracking branch in 2017. They then implemented the main line detection task using an improved R-FCN algorithm, interacted with the multiscale feature maps of the first stage based on the properties of two-stage object detection. This method is based on the traditional twin network framework for single-object tracking, but the original twin network uses 1:1 correlation filtering, whereas the D&T framework uses n:n correlation filtering. The experimentally verified algorithm significantly increases the accuracy and speed of multi-object tracking, but it is still essentially a two-stage tracking algorithm to further integrate the detection and tracking modules. Bergmann et al. [70] proposed a new class of joint detection and tracking framework, Tracktor++, whose core lies in using the tracking frame and observation frame instead of the original RPN module to obtain the true observation frame. Next, data correlation is used to establish the matching of the tracking frame and observation frame. It has been empirically proven that improving the object detection network not only improves the tracking effect but also increases the weight of influence of the fused detection module on the final tracking effect. Inspired by the Tracktor++ framework, Zhang et al. [71] further improved the detection network by adding the predicted optical flow feature module, which turned the tracking frame and  
Figure 5: Three Algorithmic Frameworks of JDT. Three different JDT algorithms are created by fusing the sub-modules encompassed by the purple trapezoidal curve, green elliptic curve, and red rectangular curve. (Source: This image is from the MOT16 dataset [25]).  
observation frame in Tracktor++ into an optical flow prediction frame and observation frame. After enhancing the motion model, epistatic model, and data association section, Huang et al. [72] also worked to enhance the tracking effect.  
When monitoring small objects or a large number of objects, the tracking efficiency is poor, the quantity of calculation is excessive, and the detection speed is too sluggish for automatic driving. The Tracktor++ framework's object recognition techniques all use anchor boxes as their foundation. Given this minimal input to locate objects and predict their associations with the previous frame, Zhou et al.'s multi-object tracking algorithm CenterTrack [73] is applied to the detection of a pair of images and the previous frame. The tracking problem is transformed into the tracking based on the object center point, and the two-dimensional and three-dimensional multi-object tracking of pedestrians and vehicles is realized at the same time.