# A Review of Video Object Detection: Datasets, Metrics and Methods
## 5 Future Trends  
Challenges still remain for further improving the accuracy and speed of the video object detection methods. This section presents the major challenges and possible future trends as related to video object detection.  
At present, there is a lack of a comprehensive benchmark dataset containing the labels of each frame. The most widely used dataset, that is ImageNet VID, does not include complex real-world conditions as compared to the static image dataset COCO. The number of objects in each frame in the ImageNet VID dataset is limited, which is not the case under real-world conditions. In addition, in many real-world applications, videos include a large field of view and in some cases high resolution images. Lack of a well-annotated dataset representing actual or real-world conditions remains a challenge for the purpose of advancing video object detection. Hence, the establishment of a comprehensive benchmark dataset is considered a future trend of importance.  
Up to now, the most widely used evaluation metric in video object detection is mAP, which is derived from static image object detection. This metric does not fully reflect the temporal characteristics in video object detection. Although Average Delay (AD) is proposed to reflect the temporal characteristics, it is still not a fully developed metric. For example, the stability of detection in video is not reflected by it. Therefore, novel evaluation metrics to reflect detection stability which are more suitable for video object detection are considered to be another future trend of importance.  
Most of the methods covered in this review paper only utilize the local temporal information or global information separately. There are only a few methods, such as MEGA, which have used the local and global temporal information at the same time and achieved a benchmark mAP of 85.4%. As demonstrated by MEGA, it is worth developing future frameworks which utilize both the local and global temporal information. Furthermore, for most of the existing video object detection algorithms, the number of frames used is too small to fully utilize the video information. Hence, as yet another future trend, it is of importance to develop methods that utilize the long-term video information.  
As can be observed from Tables 4 and 5, the attention-based frameworks achieved a relatively high accuracy. However, such methods pose difficulties for real-time applications demanding very powerful GPUs. Although the Looking Fast and Slow method [70] achieved 72.3 fps on Pixel 3 phones, the accuracy is only 59.3%, which poses challenges for actual deployment. Indeed, the trade-off between accuracy and speed needs to be further investigated. Real-time performance is important for practical applications such as autonomous driving and video surveillance. It is significant to pay more attention to the methods to make a light model, while ensuring that the accuracy will not drop too much. Some light network structure design methods like Depthwise Separable Convolution [115] and channel shuffle [124] used in the classification application can be used for reference in video object detection. In addition, model compression methods like [125] can be considered as well.