# A Review of Video Object Detection: Datasets, Metrics and Methods
## 2 Datasets and Evaluation Metrics
### Evaluation Metrics  
The metric mean Average Precision (mAP) is extensively used in conventional object detection, which provides a performance evaluation in terms of regression and classification accuracies [9; 10; 11; 12; 13; 14; 15; 17]. The evaluation metric mAP represents the mean Average Precision. The definition is the mean of the Average Precision of each category. As per the PASCAL Visual Object Classes Challenge 2012 (VOC2012) Development Kit, it is computed as follows:  
1. The Precision/Recall curve is obtained first. For the Recall (r), the Precision is set to the maximum Precision achieved for any Recall \(\text{r}^{\prime}\geq\text{r}\). 2. The area under the Precision/Recall curve is considered to be the Average Precision (AP). The mean of AP in each category is mAP. Prior to 2010, AP used to be computed by sampling the curve at a set of uniformly spaced Recall \((0,0.1,0.2,\ldots,1)\) and then computing the average of the corresponding Precision value. More specifically, \(\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}\) and \(\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}\), where the definitions of TP, FP and FN appear in Table 1.  
When the IoU is larger than a set threshold, the prediction is true. That is, \(\text{IoU}=\frac{\text{area}\left(\text{B}_{\text{gt}}\cap\text{B}_{\text{p} }\right)}{\text{area}\left(\text{B}_{\text{gt}}\cap\text{B}_{\text{p}}\right)}\), where \(\text{B}_{\text{gt}}\) and \(\text{B}_{\text{p}}\) indicate the ground truth and prediction box, respectively. More details are stated in the following example. The detection results of one category are presented in Table 2, and the number of the objects is 3, which means that TP + FN = 3. Confidence represents the confidence level of the prediction boxes. The definition of confidence score is stated in Equation (1) with \(\text{P}_{\text{r}}\) denoting precision and \(\text{IoU}_{\text{pred}}^{\text{truth}}\) the confidence level of the box surrounding the entire object,  
\[\text{confidence score}=\text{P}_{\text{r}}(\text{object})\times\text{ IoU}_{\text{pred}}^{\text{truth}}, \tag{1}\] \[\text{P}_{\text{r}}(\text{object})\in[0,1].\] \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Label is True.** & **Label is False.** \\ \hline Prediction is true. & True positive (TP) & False positive (FP) \\ Prediction is false. & False negative (FN) & True negative (TN) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Definitions of TP, FP, TN and FN. The detection results are ranked according to the confidence score, which are shown in Table 3. The Precision and Recall are computed by the Equation noted above. The Precision/Recall curve of this category is shown in Figure 1. As a result, \(\text{AP}=\text{A1}+\text{A2}+\text{A3}=\left(\frac{1}{3}\times 1\right)+ \left(\frac{2}{3}-\frac{1}{3}\right)\times\frac{3}{4}+\left(1-\frac{2}{3} \right)\times\frac{3}{4}=83.33\%\) and mAP is the mean of the AP in each category. For video object detection, mAP is also directly used as an evaluation metric in [22; 25; 28; 72; 74]. Based on the object speed, it is labeled as mAP (slow), mAP (medium) and mAP (fast) [22]. This is done using the average score of IoU (Intersection over Union) of a current frame and 10 frames ahead and past as follows: slow (score \(>\) 0.9), medium (score \(\in\) [0.7, 0.9]) and fast (score \(<\) 0.7). In [110], it was pointed out that performance cannot be sufficiently evaluated using only Average Precision (AP), since the temporal nature of video snippets do not get captured by it. In the same paper, a new metric named Average Delay (AD) was introduced based on the number of frames taken to detect an object starting from the frame it first appears in. A subset of the ImageNet VID dataset, named ImageNet VIDT, was considered to verify the effectiveness of AD. It has been reported that most methods with higher ADs still had good APs or good average detection accuracies. However, higher ADs also mean that the detection delay is large.