# Multiple Object Tracking in Robotic Applications: Trends and Challenges
## 3 Mot Benchmark Datasets and Evaluation Metrics
### Benchmark Datasets  
Most research done in multiple object tracking uses standard datasets for evaluating the state of art techniques. In this way, we have a better view of what criteria, the new methodologies, have shown superiority. For this application, everyday moving objects could be pedestrians, vehicles, cyclists, etc. The most common datasets that provide a variation of those objects in the streets are the MOTChallenge collection and KITTI.  
* MOTChallenge: The most common datasets in this collection are the MOT15 [19], MOT16 [92], MOT17 [92], and MOT20 [93]. There is a newly created set, MOT20, but it has not yet become a standard for evaluation in the research community to our current knowledge. The MOT datasets contain some data from existing sets such as PETS and TownCenter and others that are unique. Examples of the data included are presented in Table 4, where the amount of variation included in the MOT15 and MOT16 can be observed. Thus, the dataset is useful for training and testing using static and dynamic backgrounds and for 2D and 3D tracking. An evaluation tool is also given with the set to measure all features of the multiple object tracking algorithm, including accuracy, precision, and FPS. The ground truth data samples are shown in Figure 10.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Dataset** & **Sequences** & **Length** & **Tracks** & **FPS** & **Platform** & **ViewPoint** & **Density** & **Weather** \\ \hline \multirow{4}{*}{MOT 2015} & TUD-Crossing & 201 & 13 & 25 & static & horizontal & 5.5 & cloudy \\  & PETS09-S212 & 436 & 42 & 7 & static & high & 22.1 & cloudy \\  & ETH-jelmoi & 440 & 45 & 14 & moving & low & 5.8 & sunny \\  & KITTI-16 & 209 & 17 & 10 & static & horizontal & 8.1 & sunny \\ \hline \multirow{4}{*}{MOT 2016} & MOT16-01 & 450 & 23 & 30 & static & horizontal & 14.2 & cloudy \\  & MOT16-03 & 1500 & 148 & 30 & static & high & 69.7 & night \\  & MOT16-06 & 1194 & 221 & 14 & moving & low & 9.7 & sunny \\  & MOT16-12 & 900 & 86 & 30 & moving & horizontal & 9.2 & indoor \\ \hline \hline \end{tabular}
\end{table}
Table 4: Examples of the types of data included in MOT15 and MOT16.  
Figure 10: Samples from MOT 15-17 ground truth dataset. Samples of MOT15 (**top image**), MOT16 (**middle image**), and MOT17 (**bottom image**).  
* KITTI [94]: This dataset is created specifically for autonomous driving. It was collected by a car driven through the streets with multiple sensors mounted for data collection. The set includes PointCloud data collected using LIDAR sensors and RGB video sequences captured by monocular cameras. It has been included in multiple research related to 2D and 3D multiple object tracking. Samples of the pointcloud and RGB data included in the KITTI dataset are shown in Figure 11.
* UA-DETRAC [96; 97; 98]: The dataset includes videos sequences captured from static cameras looking at the streets at different cities. A huge amount of labeled vehicles can assist in training and testing for static background multiple object tracking in surveillance and autonomous driving. Samples of the UA-DETRAC dataset at different illumination conditions can be shown in Figure 12.