# A Review of Video Object Detection: Datasets, Metrics and Methods
## 4 Comparison of Video Object Detection Methods  
The great majority of video object detection approaches use the ImageNet VID dataset [5] for performance evaluation. In this section, the timeline of video object detection methods in recent years is shown in Figure 11 together with a group listing of the methods in Figure 12. Then, a comparison is provided between the methods covered in the previous section. The comparison is presented in Tables 4 and 5, which correspond to with and without post-processing, respectively. The methods in Figure 11 belong to different groups but the same time, whereas the methods in Figure 12 belong to different times but the same groups. As can be seen from Figures 11 and 12, the methods based on optical flow were proposed earlier. During the same period, video object detection methods were assisted by tracking due to the effectiveness of tracking in utilizing the temporal-spatial information. The optical flow-based methods needed a large number of parameters and they were only suitable for small motions. In recent years, the methods based on attention have achieved much success, such as MEGA [25]. Using LSTM for feature propagation and aggregation is becoming a hot research topic and many new methods are being proposed, such as STSN [83] using deformable convolution to align the feature maps. The latest research is mostly based on attention, LSTM or a combination of methods such as Flow&LSTM [72]. Table 4 provides the outcomes without post processing. In this table, the methods are divided into different groups according to the way temporal and spatial information are utilized. Flow-guided groups propagate and align the feature maps according to the flow field obtained by optical flow. Both accuracy and speed of various frameworks are reported in this table. For example, DFF provides high computational efficiency and achieves a runtime of 20.25 fps using a Titan K40 GPU. FGFA achieves a high accuracy producing 76.3% mAP with 1.36 fps. Obviously, DFF is faster than FGFA. Flow-guided methods are intuitive and well understood to propagate features. Optical flow is deemed suitable for small movement estimation. In addition, since optical flow reflects pixel level displacement, it has difficulties when it is applied to high-level feature maps. One pixel movement on feature maps may correspond to 10 to 20 pixels movement. Figure 11: Timeline of video object detection methods. Figure 12: Video object detection methods sorted in different groups. Inspired by the LSTM-based solutions in natural language processing, LSTM methods are used to incorporate the sequence information. In the LSTM group, Flow&LSTM [72] achieved the highest accuracy of 75.5%. Looking Fast and Slow [70] generated high speed but with low accuracy. LSTM captures the long-term information with a simple implementation. Since the sigmoid activation of the input and forget gates are rarely completely saturated, a slow state decay and thus loss of long-term dependence is resulted. In other words, it is difficult to retain the complete previous state in the update. Attention-based methods also show the ability to perform video object detection effectively. In the attention-related group, MEGA [25] with ResNeXt-101 as backbone achieved the highest accuracy of 84.1% mAP. As described, it achieved a very high accuracy with a relatively fast speed. Attention-based methods aggregate the features within proposals that are generated. This decreases the computation