# A Review of Deep Learning-Based Visual Multi-Object Tracking Algorithms for Autonomous Driving
## 3 Overview of Deep Learning-Based Visual Multi-Object Tracking
### MOT Evaluating Indicator  
Setting realistic and accurate evaluation metrics is essential for comparing the effectiveness of visual multi-object tracking algorithms in an unbiased and fair manner. The three criteria that make up the multi-object tracking assessment indicators are if the object detection is real-time, whether the predicted position matches the actual position, and whether each object maintains a distinct ID [33]. MOT Challenge offers recognized MOT evaluation metrics. MOTA (Multi-Object-Tracking Accuracy): the accuracy of multi-object tracking is used to count the accumulation of errors in tracking, including the number of tracking objects and whether they match:  
\[\text{MOTP}=\frac{\sum\left(\text{FN}+\text{FP}+\text{IDSW}\right)}{\sum \text{GT}} \tag{1}\]  
where FN (False Negative) is the number of detection frames that do not match the prediction frame; FP (False positive) is the number of prediction frames that do not match the detection frame; IDSW (ID Switch) is the object ID change the number of times; GT (Ground Truth) is the number of tracking objects. MOTP (Multi-Object-Tracking Precision): the accuracy of multi-object tracking, which is used to evaluate whether the object position is accurately positioned. \[\text{MOTP}=\frac{\sum\text{Bt}(i)}{\sum\text{C}t} \tag{2}\]  
where C_t_ is the number of matches between the object and the predicted object in the t-th frame; Bt(\(i\)) is the distance between the corresponding position of the object in the t-th frame and the predicted position, also known as the matching error. AMOTA (Average Multiple Object Tracking Accuracy): summarize MOTA over all object confidence thresholds instead of using a single threshold. Similar to mAP for object detection, it is used to evaluate the overall accurate performance of the tracking algorithm under all thresholds to improve algorithm robustness. AMOTA can be calculated by \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Ref.** & **Datasets** & **Year** & **Feature** & **DOI/URL** \\ \hline
[25, 26, 32] & MOT15, 16,17,20 & 2016–2020 & Sub datasets containing multiple different camera angles and scenes & [[https://dot.org/10.48550/arXiv:1504.01942](https://dot.org/10.48550/arXiv:1504.01942)]([https://dot.org/10.48550/arXiv:1504.01942](https://dot.org/10.48550/arXiv:1504.01942)) \\
[27, 28] & KITTI-Tracking & 2012 & Provides annotations for cars and pedestrians, scene objects are sparse & [[https://dot.org/10.1177/0278364913491297](https://dot.org/10.1177/0278364913491297)]([https://dot.org/10.1177/0278364913491297](https://dot.org/10.1177/0278364913491297)) \\
[29] & NuScenes & 2019 & Dense traffic and challenging driving conditions & [[https://www.cvlibs.net/datasets/kitti/eval_tracking.php](https://www.cvlibs.net/datasets/kitti/eval_tracking.php)]([https://www.cvlibs.net/datasets/kitti/eval_tracking.php](https://www.cvlibs.net/datasets/kitti/eval_tracking.php)) \\
[30] & Waymo & 2020 & Environment, dense label information & [[https://www.nuscenes.org/](https://www.nuscenes.org/)]([https://www.nuscenes.org/](https://www.nuscenes.org/)) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary of Visual Multi-object Tracking Datasets. integrating MOTA under the recall curve, using interpolation to approximate the integral in order to simplify the calculation. \[\text{AMOTA}= \frac{1}{\text{L}}\sum_{r\in\{\frac{1}{L},\frac{2}{L},\dots,1\}} \left(1-MOTA_{r}\right) \tag{3}\]  
where L represents the number of recall values (integration confidence threshold), the higher the L, the more accurate the approximate integral. AMOTA represents the multi-object tracking accuracy at a specific recall value r.  
AMOTP (Average Multi-object Tracking Precision): The same calculation method as AMOTA, with recall as the abscissa and _MOTP_ as the ordinate, use the interpolation method to obtain AMOTP. \[\text{AMOTP}= \frac{1}{\text{L}}\sum_{r\in\{\frac{1}{L},\frac{2}{L},\dots,1\}} \left(1-MOTP_{r}\right) \tag{4}\]  
IDF1 (ID F1 score): measures the difference between the predicted ID and the correct ID.